Evaluative Analysis of Deep Learning and Conventional Machine Learning for Outlier Identification in Financial Transactions
- Victor Amadi Oluikpe & Nnali-Uroh, Emmanuel
- DOI: 10.5281/zenodo.19927545
- ISA Journal of Engineering and Technology (ISAJET)
The exponential growth of digital financial services has led to a corresponding rise in fraudulent transactions, highlighting the limitations of traditional rule-based and shallow machine learning fraud detection systems. This study presents a comparative framework for anomaly detection in financial transactions using both deep learning and traditional machine learning models, with a focus on real-time deployment, model interpretability, and practical utility. The core objective was to design and evaluate an intelligent fraud detection system capable of learning temporal transaction patterns and providing transparent predictions. Three deep learning architectures—Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), and Transformer—were developed alongside traditional classifiers including Logistic Regression, Support Vector Machine (SVM), and Random Forest. The models were trained on an imbalanced real-world dataset. Class imbalance was addressed using AIF360 and SMOTE. Evaluation metrics included accuracy, precision, recall, F1-score, and ROC-AUC. Results indicated that Random Forest achieved best classification performance, while LSTM excelled in recall and temporal pattern recognition. Both models were selected for deployment. The LSTM model was quantized using TensorFlow Lite (TFLite) for real-time inference on resource-constrained devices. Model transparency was ensured using SHapley Additive exPlanations (SHAP), revealing that features such as transaction amount and account balances significantly influenced predictions. A custom Streamlit dashboard was developed to operationalize the models, supporting batch and real-time transaction evaluation with integrated SHAP visualizations. This accessible interface demonstrated practical applicability for fraud analysts and compliance teams. Overall, this research contributes a scalable, interpretable, and deployable fraud detection solution, combining high predictive accuracy with explainability. The findings support the adoption of AI-driven systems in modern financial infrastructures and offer a foundation for future research in ethical, real-time fraud prevention.
