Machine Learning and Counterfactual Fairness Analysis to Detect and Counter Bias in Credit Analysis for Loan Granting Systems

Abstract

Creditworthiness analysis and the loan granting process are core services of the finance sector. With the shifts towards Machine Learning and Artificial Intelligence based solutions, the finance sector has also incorporated them into their creditworthiness analysis and loan granting processes over the years. As the finance sector is a heavily regulated industry, machine learning solutions require transparency, interpretability, and fairness. In this research, the questions of how to detect bias in the dataset before training, how to train a fair model, and how to assess the fairness of a trained model are explored. The purpose of this thesis is to derive a modular approach that addresses inherited bias from the dataset in the model training itself. This research uses existing literature for defining bias, to determine the machine learning models that work best for creditworthiness analysis, and methodologies for determining bias in trained models, along with determining how bias can be tackled at the model training level. The core methodology that this thesis suggests and focuses on is a Hybrid SVM and Random Forest, which utilizes custom decision trees based on the leave-group-out concept. The dataset utilized in the thesis is the German Credit Dataset from the UCI repository. It has several features, including sensitive features like gender and foreign worker status. The thesis determines the importance of all features for quantifying the effect of sensitive features, analyzes the dataset, trains models including the proposed model for comparison of bias in the model outcome and accuracy. Evaluation metrics show the proposed model performs better than or as well as the other models. Counterfactual analysis and statistical tests for fairness show that the proposed solution significantly reduces bias and is much more fair in its predictions. The thesis lastly sheds light on the practical use of the proposed model.