Unveiling Accuracy-Fairness Trade-Offs
Investigating Machine Learning Models in Student Performance Prediction
DOI:
https://doi.org/10.18608/jla.2025.8543Keywords:
accuracy-fairness trade-offs, algorithmic bias, machine learning, Virtual Learning Environments (VLEs), bias mitigation, research paperAbstract
While high-accuracy machine learning (ML) models for predicting student learning performance have been widely explored, their deployment in real educational settings can lead to unintended harm if the predictions are biased. This study systematically examines the trade-offs between prediction accuracy and fairness in ML models trained on the widely used Open University Learning Analytics Dataset (OULAD). We evaluated the relationship between model accuracy and fairness across various student demographic subgroups and investigated the extent to which fairness can be improved without significantly sacrificing accuracy. Our analysis revealed that standard ML models often exhibit bias; however, applying bias mitigation techniques can reduce these disparities while maintaining acceptable accuracy. Our findings emphasize the importance of auditing ML models for fairness to ensure that predictive insights are equitable across diverse student populations. We also discuss implications for best practices and challenges in achieving fair ML models for student performance prediction.
References
Baker, R. S. & Hawn, A. (2022). Algorithmic bias in education. International Journal of Artificial Intelligence in Education, 32(4), 1052–1092. https://doi.org/10.1007/s40593-021-00285-9
Bayer, V., Hlosta, M., & Fernandez, M. (2021). Learning analytics and fairness: Do existing algorithms serve everyone equally? In I. Roll, D. McNamara, S. Sosnovsky, R. Luckin, & V. Dimitrova (Eds.), Artificial intelligence in education: 22nd international conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, proceedings, part II (pp. 71–75). Springer. https://doi.org/10.1007/978-3-030-78270-2_12
Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., Nagar, S., Ramamurthy, K. N., Richards, J., Saha, D., Sattigeri, P., Singh, M., Varshney, K. R., & Zhang, Y. (2018). AI fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv. https://doi.org/10.48550/arxiv.1810.01943
Brodersen, K. H., Ong, C. S., Stephan, K. E., & Buhmann, J. M. (2010). The balanced accuracy and its posterior distribution. 2010 20th international conference on pattern recognition (pp. 3121–3124). IEEE. https://doi.org/10.1109/ICPR.2010.764
Carey, A. N., & Wu, X. (2023). The statistical fairness field guide: Perspectives from social and formal sciences. AI and Ethics, 3(1), 1–23. https://doi.org/10.1007/s43681-022-00183-3
Caton, S., & Haas, C. (2023). Fairness in machine learning: A survey. ACM Computing Surveys, 56(7), Article 166. https://doi.org/10.1145/3616865
Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In B. Krishnapuram, M. Shah, A. Smola, C. Aggarwal, D. Shen, & R. Rastogi (Eds.), KDD ’16: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (pp. 785–794). https://doi.org/10.1145/2939672.2939785
Chinta, S. V., Wang, Z., Yin, Z., Hoang, N., Gonzalez, M., Le Quy, T., & Zhang, W. (2024). FairAIED: Navigating fairness, bias, and ethics in educational AI applications. ArXiv. https://doi.org/10.48550/arxiv.2407.18745
Corbett-Davies, S., Gaebler, J. D., Nilforoshan, H., Shroff, R., & Goel, S. (2023). The Measure and Mismeasure of Fairness. Journal of Machine Learning Research, 24(312), 1–117. https://www.jmlr.org/papers/volume24/22-1511/22-1511.pdf
Deho, O. B., Zhan, C., Li, J., Liu, J., Liu, L., & Duy Le, T. (2022). How do the existing fairness metrics and unfairness mitigation algorithms contribute to ethical learning analytics? British Journal of Educational Technology, 53(4), 822–843. https://doi.org/10.1111/bjet.13217
Dutta, S., Wei, D., Yueksel, H., Chen, P.-Y., Liu, S., & Varshney, K. R. (2020). Is there a trade-off between fairness and accuracy? A perspective using mismatched hypothesis testing. ArXiv. doi.org/10.48550/arxiv.1910.07870
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2011). Fairness through awareness. ArXiv. doi.org/10.48550/arxiv.1104.3913
Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874. doi.org/10.1016/j.patrec.2005.10.010
Fazil, M., Rísquez, A., & Halpin, C. (2024). A novel deep learning model for student performance prediction using engagement data. Journal of Learning Analytics, 11(2), 23–41. doi.org/10.18608/jla.2024.7985
Feldman, M., Friedler, S., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015). Certifying and removing disparate impact. ArXiv. doi.org/10.48550/arxiv.1412.3756
Fenu, G., Galici, R., & Marras, M. (2022). Experts’ view on challenges and needs for fairness in artificial intelligence for education. In M. M. Rodrigo, N. Matsuda, A. I. Cristea, & V. Dimitrova (Eds.), Artificial intelligence in education: 23rd international conference, AIED 2022, Durham, UK, July 27–31, 2022, proceedings, part I (pp. 243–255). Springer. doi.org/10.1007/978-3-031-11644-5_20
Gardner, J., Brooks, C., & Baker, R. (2019). Evaluating the fairness of predictive student models through slicing analysis. In S. Hsiao, J. Cunningham, K. McCarthy, G. Lynch, C. Brooks, R. Ferguson, & U. Hoppe (Eds.), LAK19: Proceedings of the 9th international conference on learning analytics & knowledge (pp. 225–234). ACM Press. doi.org/10.1145/3303772.3303791
Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. ArXiv. doi.org/10.48550/arxiv.1610.02413
Hu, Q., & Rangwala, H. (2020). Towards fair educational data mining: A case study on detecting at-risk students. In A. N. Rafferty, J. Whitehill, C. Romero, & V. Cavalli-Sforza (Eds.), Proceedings of the 13th international conference on educational data mining (EDM 2020) (pp. 431–437). International Educational Data Mining Society. https://files.eric.ed.gov/fulltext/ED608050.pdf
Idowu, J. A. (2024). Debiasing education algorithms. International Journal of Artificial Intelligence in Education, 34(4), 1510–1540. doi.org/10.1007/s40593-023-00389-4
Johnston, L. J., Griffin, J. E., Manolopoulou, I., & Jendoubi, T. (2024). Uncovering student engagement patterns in Moodle with interpretable machine learning. ArXiv. doi.org/10.48550/arxiv.2412.11826
Kamiran, F., & Calders, T. (2012). Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems, 33(1), 1–33. doi.org/10.1007/s10115-011-0463-8
Kamiran, F., & Calders, T. (2009). Classifying without discriminating. In 2009 2nd international conference on computer, control and communication (pp. 1–6). IEEE. doi.org/10.1109/IC4.2009.4909197
Khoudi, Z., Hafidi, N., Nachaoui, M., & Lyaqini, S. (2025). New approach to enhancing student performance prediction using machine learning techniques and clickstream data in virtual learning environments. SN Computer Science, 6, Article 139. https://doi.org/10.1007/s42979-024-03622-6
Kizilcec, R. F., & Lee, H. (2022). Algorithmic fairness in education. In W. Holmes & K. Porayska-Pomsta (Eds.), The ethics of artificial intelligence in education: Practices, challenges, and debates (pp. 174–202). Routledge. doi.org/10.4324/9780429329067-10
Köchling, A., Riazy, S., Wehner, M. C., & Simbeck, K. (2021). Highly accurate, but still discriminatory. Business & Information Systems Engineering, 63(1), 39–54. doi.org/10.1007/s12599-020-00673-w
Kuzilek, J., Hlosta, M., & Zdrahal, Z. (2017). Open University learning analytics dataset. Scientific Data, 4, Article 170171. doi.org/10.1038/sdata.2017.171
Lallé, S., Bouchet, F., Verger, M., & Luengo, V. (2024). Fairness of MOOC completion predictions across demographics and contextual variables. In A. M. Olney, I.-A. Chounta, Z. Liu, O. C. Santos, I. I. Bittencourt (Eds.), Artificial intelligence in education: 25th international conference, AIED 2024, Recife, Brazil, July 8–12, 2024, proceedings, part I (pp. 379–393). Springer. doi.org/10.1007/978-3-031-64302-6_27
Leite, W. L., Jing, Z., Kuang, H., Kim, D., & Huggins-Manley, A. C. (2021). Multilevel mixture modeling with propensity score weights for quasi-experimental evaluation of virtual learning environments. Structural Equation Modeling: A Multidisciplinary Journal, 28(6), 964–982. doi.org/10.1080/10705511.2021.1919895
Le Quy, T., Nguyen, T. H., Friege, G., & Ntoutsi, E. (2023). Evaluation of group fairness measures in student performance prediction problems. In I. Koprinska, P. Mignone, R. Guidotti, S. Jaroszewicz, H. Fröning, F. Gullo, P. M. Ferreira, D. Roqueiro, G. Ceddia, S. Nowaczyk, J. Gama, R. Ribeiro, R. Gavaldà, E. Masciari, Z. Ras, E. Ritacco, F. Naretto, A. Theissler, P. Biecek, … S. Pashami (Eds.), Machine learning and principles and practice of knowledge discovery in databases: International workshops of ECML PKDD 2022, Grenoble, France, September 19–23, 2022, proceedings, part I (pp. 119–136). Springer. doi.org/10.1007/978-3-031-23618-1_8
Liu, S., & Vicente, L. N. (2022). Accuracy and fairness trade-offs in machine learning: A stochastic multi-objective approach. arXiv. doi.org/10.48550/arxiv.2008.01132
Liu, Z., Jiao, X., Li, C., & Xing, W. (2024). Fair prediction of students’ summative performance changes using online learning behavior data. Proceedings of the 17th international conference on educational data mining (pp. 686–691). International Educational Data Mining Society. https://doi.org/10.5281/zenodo.12729918
Li, C., Xing, W., & Leite, W. (2024). Using fair AI to predict students’ math learning outcomes in an online platform. Interactive Learning Environments, 32(3), 1117–1136. doi.org/10.1080/10494820.2022.2115076
Martinez, A. L. J., Sood, K., & Mahto, R. (2025). Early detection of at-risk students using machine learning. In H. R. Arabnia, L. Deligiannidis, S. Amirian, F. Ghareh Mohammadi, & F. Shenavarmasouleh (Eds.), Foundations of computer science and frontiers in education: Computer science and computer engineering: 20th international conference, FCS 2024, and 20th international conference, FECS 2024, held as part of the world congress in computer science, computer engineering and applied computing, CSCE 2024, Las Vegas, NV, USA, July 22–25, 2024, revised selected papers (pp. 396–406). Springer. https://doi.org/10.1007/978-3-031-85930-4_36
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022). A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6), Article 115. https://doi.org/10.1145/3457607
Morik, M., Singh, A., Hong, J., & Joachims, T. (2020). Controlling fairness and bias in dynamic learning-to-rank. In J. Huang, Y. Chang, X. Cheng, J. Kamps, V. Murdock, J.-R. Wen, & Y. Liu (Eds.), SIGIR ’20: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval (pp. 429–438). ACM Press. doi.org/10.1145/3397271.3401100
Pei, B., & Xing, W. (2022). An interpretable pipeline for identifying at-risk students. Journal of Educational Computing Research, 60(2), 380–405. doi.org/10.1177/07356331211038168
Peng, L., & Jeang, B. (2023). Prediction model of students’ learning behavior on learning effect in online live class based on machine learning algorithm. In Y. Zhong (Ed.), Fifth international conference on computer information science and artificial intelligence (CISAI 2022) (Article 1256650). SPIE. doi.org/10.1117/12.2669155
Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., & Weinberger, K. Q. (2017). On fairness and calibration. ArXiv. doi.org/10.48550/arxiv.1709.02012
Raftopoulos, G., Davrazos, G., & Kotsiantis, S. (2025). Evaluating fairness strategies in educational data mining: A comparative study of bias mitigation techniques. Electronics, 14(9), Article 1856. https://doi.org/10.3390/electronics14091856
Romero, C., & Ventura, S. (2020). Educational data mining and learning analytics: An updated survey. WIREs: Data Mining and Knowledge Discovery, 10(3), Article e1355. doi.org/10.1002/widm.1355
Shin, J., Bulut, O., & Pinto, W. N., Jr. (2022). E-learning preparedness: A key consideration to promote fair learning analytics development in higher education. Proceedings of the 15th international conference on educational data mining (pp. 673–678). International Educational Data Mining Society. doi.org/10.5281/zenodo.6853111
Song, Y., Li, C., Xing, W., Li, S., & Hannah, H. (2024). A fair clustering approach to self-regulated learning behaviors in a virtual learning environment. In B. Flanagan, B. Wasson, & D. Gašević (Eds.), LAK ’24: Proceedings of the 14th learning analytics and knowledge conference (pp. 771–778). ACM Press. https://doi.org/10.1145/3636555.3636863
Tukey, J. W. (1949). Comparing individual means in the analysis of variance. Biometrics, 5(2), 99–114. doi.org/10.2307/3001913
Verger, M., Fan, C., Lallé, S., Bouchet, F., & Luengo, V. (2024). A comprehensive study on evaluating and mitigating algorithmic unfairness with the MADD metric. Journal of Educational Data Mining, 16(1), 365–409. doi.org/10.5281/zenodo.12180668
Verma, S., & Rubin, J. (2018). Fairness definitions explained. In Y. Brun, B. Johnson, & A. Meliou (Eds.), FairWare ’18: Proceedings of the international workshop on software fairness, 1–7. doi.org/10.1145/3194770.3194776
Wang, Y., Wang, X., Beutel, A., Prost, F., Chen, J., & Chi, E. H. (2021). Understanding and improving fairness–accuracy trade-offs in multi-task learning. In F. Zhu, B. C. Ooi, C. Miao, H. Wang, I. Skrypnyk, W. Hsu, & S. Chawla (Eds.), KDD ’21: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining (pp. 1748–1757). ACM Press. doi.org/10.1145/3447548.3467326
Wei, Q., & Dunbrack, R. L., Jr. (2013). The role of balanced training and testing data sets for binary classifiers in bioinformatics. PLOS ONE, 8(7), Article e67863. doi.org/10.1371/journal.pone.0067863
Wongvorachan, T., Bulut, O., Liu, J. X., & Mazzullo, E. (2024). A comparison of bias mitigation techniques for educational classification tasks using supervised machine learning. Information, 15(6), Article 326. doi.org/10.3390/info15060326
Xing, W., & Du, D. (2018). Dropout prediction in MOOCs: Using deep learning for personalized intervention. Journal of Educational Computing Research, 57(3), 547–570. doi.org/10.1177/0735633118757015
Xu, J., Xiao, Y., Wang, W. H., Ning, Y., Shenkman, E. A., Bian, J., & Wang, F. (2022). Algorithmic fairness in computational medicine. eBioMedicine, 84, Article 104250. doi.org/10.1016/j.ebiom.2022.104250
Yu, H.-F., Huang, F.-L., & Lin, C.-J. (2011). Dual coordinate descent methods for logistic regression and maximum entropy models. Machine Learning, 85(1–2), 41–75. doi.org/10.1007/s10994-010-5221-8
Zhang, F., Xing, W., & Li, C. (2023). Predicting students’ Algebra I performance using reinforcement learning with multi-group fairness. In I. Hilliger, H. Khosravi, B. Rienties, & S. Dawson (Eds.), LAK23: 13th international learning analytics and knowledge conference (pp. 657–662). ACM Press. doi.org/10.1145/3576050.3576104
Zhao, C., Mi, F., Wu, X., Jiang, K., Khan, L., & Chen, F. (2024). Dynamic environment responsive online meta-learning with fairness awareness. ACM Transactions on Knowledge Discovery from Data, 18(6), Article 153. doi.org/10.1145/3648684
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution 4.0 International License.
