Toward Reliable Estimation of Algorithmic Bias for Minority Groups
DOI:
https://doi.org/10.18608/jla.2026.9001Keywords:
algorithmic bias, fairness, group bias, bias audit, estimation of bias, reliability, predictive modelsAbstract
While predictive models are widely used in learning analytics, several studies have shown that the performance of these models can vary significantly across different demographic groups of students. The first step to audit for and mitigate these group biases is to accurately estimate them. However, the current practices for identifying and measuring group bias often suffer from reliability issues. In this paper, we use simulations and real-world data analysis to explore statistical factors that impact the reliable estimate of group bias and suggest approaches to improve their statistical robustness. Our analysis revealed that small group sizes lead to high variability in group bias estimation due to sampling error -- an issue that is more likely to impact students from historically marginalized communities. We then suggest statistical approaches, such as bootstrapping, to construct confidence intervals for a more reliable estimation of group bias. Based on our findings, we encourage future learning analytics researchers to ensure sufficiently large group sizes, construct confidence intervals rather than relying on p-values, use at least two metrics, and move beyond the dichotomy of the presence or absence of bias for a more comprehensive evaluation of group bias.
References
Binns, R. (2020). On the apparent conflict between individual and group fairness. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* 2020), 27–30 January 2020, Barcelona, Spain (pp. 514–524). ACM. https://doi.org/10.1145/3351095.3372864
Blazkova, T., Kizilcec, R. F., Nielsen, M. L., Lassen, D. D., & Bjerre-Nielsen, A. (2025). Fairness over time: A nationwide study of evolving bias in dropout prediction. In Proceedings of the 12th ACM Conference on Learning at Scale (L@S 2025), 21–23 July 2025, Palermo, Italy (pp. 246–250). ACM. https://doi.org/10.1145/3698205.3733933
Blodgett, S. L., Barocas, S., Daume III, H., & Wallach, H. (2020). Language (technology) is power: A critical survey of “bias” in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020), 5–10 July 2020, online (pp. 5454–5476). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.485
Bornstein, M. H., Jager, J., & Putnick, D. L. (2013). Sampling in developmental science: Situations, shortcomings, solutions, and standards. Developmental Review, 33(4), 357–370. https://doi.org/10.1016/j.dr.2013.08.003
Brandt, A. M. (1978). Racism and research: The case of the Tuskegee Syphilis Study. Hastings Center Report, 8(6), 21–29. https://doi.org/10.2307/3561468
Brown, L., & Li, X. (2005). Confidence intervals for two sample binomial distribution. Journal of Statistical Planning and Inference, 130(1-2), 359–375. https://doi.org/10.1016/j.jspi.2003.09.039
Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the First International Conference on Fairness, Accountability and Transparency (FAT* 2018), 23–24 February 2018, New York, New York, USA (pp. 77–91). PMLR. https://proceedings.mlr.press/v81/buolamwini18a.html
Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365–376. https://doi.org/10.1038/nrn3475
Carey, S. (2004). Bootstrapping & the origin of concepts. Daedalus, 133(1), 59–68. https://doi.org/10.1162/001152604772746701
Chen, S., Fang, Y., Shi, G., Sabatini, J., Greenberg, D., Frijters, J., & Graesser, A. C. (2021). Automated disengagement tracking within an intelligent tutoring system. Frontiers in Artificial Intelligence, 3, 595627. https://doi.org/10.3389/frai.2020.595627
Chicco, D., & Jurman, G. (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 21(6). https://doi.org/10.1186/s12864-019-6413-7
Chouldechova, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2), 153–163. https://doi.org/10.1089/big.2016.0047
Chouldechova, A., & Roth, A. (2020). A snapshot of the frontiers of fairness in machine learning. Communications of the ACM, 63(5), 82–89. https://doi.org/10.1145/3376898
Codiroli Mcmaster, N., & Cook, R. (2019). The contribution of intersectionality to quantitative research into educational inequalities. Review of Education, 7(2), 271–292. https://doi.org/10.1002/rev3.3116
Dablain, D., Krawczyk, B., & Chawla, N. (2024). Towards a holistic view of bias in machine learning: Bridging algorithmic fairness and imbalanced learning. Discover Data, 2(1), 4. https://doi.org/10.1007/s44248-024-00007-1
Dawson, S., Jovanovic, J., Gasevic, D., & Pardo, A. (2017). From prediction to impact: Evaluation of a learning analytics retention program. In Proceedings of the Seventh International Conference on Learning Analytics and Knowledge (LAK 2017), 13–17 March 2017, Vancouver, British Columbia, Canada (pp. 474–478). ACM. https://doi.org/10.1145/3027385.3027405
Diciccio, T. J., & Romano, J. P. (1988). A review of bootstrap confidence intervals. Journal of the Royal Statistical Society Series B: Statistical Methodology, 50(3), 338–354. https://doi.org/10.1111/j.2517-6161.1988.tb01732.x
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012). Fairness through awareness. In Proceedings of the Third Innovations in Theoretical Computer Science Conference (ITCS 2012), 8–10 January 2012, Cambridge, Massachusetts, USA (pp. 214–226). ACM. https://doi.org/10.1145/2090236.2090255
Eccles, J. S. (2005). Influences of parents’ education on their children’s educational attainments: The role of parent and child perceptions. London Review of Education, 3(3), 191–204. https://doi.org/10.1080/14748460500372309
Fagerland, M. W., Lydersen, S., & Laake, P. (2015). Recommended confidence intervals for two independent binomial proportions. Statistical Methods in Medical Research, 24(2), 224–254. https://doi.org/10.1177/0962280211415469
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015). Certifying and removing disparate impact. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2015), 10–13 August 2015, Sydney, Australia (pp. 259–268). ACM. https://doi.org/10.1145/2783258.2783311
Gardner, M. J., & Altman, D. G. (1986). Confidence intervals rather than p values: Estimation rather than hypothesis testing. British Medical Journal (Clinical Research Edition), 292(6522), 746–750. https://doi.org/10.1136/bmj.292.6522.746
Gautheron, L., Habrard, A., Morvant, E., & Sebban, M. (2019). Metric learning from imbalanced data. In Proceedings of the 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI 2019), 4–6 November 2019, Portland, Oregon, USA (pp. 923–930). IEEE. https://doi.org/10.1109/ICTAI.2019.00131
Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, & R. Garnett (Eds.), Proceedings of the 2016 Conference on Advances in Neural Information Processing Systems (NIPS 2016), 5–10 December 2016, Barcelona, Spain. NeurIPS Proceedings. https://papers.neurips.cc/paper/6374-equality-of-opportunity-in-supervised-learning
Jeni, L. A., Cohn, J. F., & De La Torre, F. (2013). Facing imbalanced data—recommendations for the use of performance metrics. In 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII 2013), 2–5 September 2013, Geneva, Switzerland (pp. 245–251). IEEE. https://doi.org/10.1109/ACII.2013.47
Jeong, H., Wu, M. D., Dasgupta, N., Medard, M., & Calmon, F. (2022). Who gets the benefit of the doubt? Racial bias in machine learning algorithms applied to secondary school math education [Poster Presented at the 2021 Workshop Math AI for Education: Bridging the Gap between Research and Smart Education, 14 December 2021, online]. https://neurips.cc/virtual/2021/33829
Jo, E. S., & Gebru, T. (2020). Lessons from archives: Strategies for collecting sociocultural data in machine learning. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* 2020), 27–30 January 2020, Barcelona, Spain (pp. 306–316). ACM. https://doi.org/10.1145/3351095.3372829
Karumbaiah, S., Baker, R. S., & Shute, V. (2018). Predicting quitting in students playing a learning game. Proceedings of the 11th International Conference on Educational Data Mining (EDM 2018), 15–18 July 2018, Buffalo, New York, USA. https://educationaldatamining.org/files/conferences/EDM2018/papers/EDM2018_paper 39.pdf
Karumbaiah, S., & Brooks, J. (2021). How colonial continuities underlie algorithmic injustices in education. In Proceedings of the 2021 Conference on Research in Equitable and Sustained Participation in Engineering, Computing, and Technology (RESPECT 2021), 23–27 May 2021, online (pp. 1–6). IEEE. https://doi.org/10.1109/RESPECT51740.2021.9620605
Karumbaiah, S., Ocumpaugh, J., & Baker, R. S. (2022). Context matters: Differing implications of motivation and help-seeking in educational technology. International Journal of Artificial Intelligence in Education, 32(3), 685–724. https://doi.org/10.1007/s40593-021-00272-0
Kearns, M., Neel, S., Roth, A., & Wu, Z. S. (2018). Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. Proceedings of Machine Learning Research, 80, 2564–2572. https://proceedings.mlr.press/v80/kearns18a.html
Kimble, G. A. (1987). The scientific value of undergraduate research participation. American Psychologist, 42(3), 267–268. https://doi.org/10.1037/0003-066X.42.3.267.b
Kwegyir-Aggrey, K., Gerchick, M., Mohan, M., Horowitz, A., & Venkatasubramanian, S. (2023). The misuse of AUC: What high impact risk assessment gets wrong. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT 2023), 12–15 June 2023, Chicago, Illinois, USA (pp. 1570–1583). ACM. https://doi.org/10.1145/3593013.3594100
Lee, D. K. (2016). Alternatives to P value: Confidence interval and effect size. Korean Journal of Anesthesiology, 69(6), 555. https://doi.org/10.4097/kjae.2016.69.6.555
Lee, H., Belitz, C., Nasiar, N., & Bosch, N. (2025). XAI reveals the causes of attention deficit hyperactivity disorder (ADHD) bias in student performance prediction. In Proceedings of the 15th International Conference on Learning Analytics and Knowledge (LAK 2025), 3–7 March 2025, Dublin, Ireland (pp. 418–428). ACM. https://doi.org/10.1145/3706468.3706521
Li, W., Sun, K., Schaub, F., & Brooks, C. (2022). Disparities in students’ propensity to consent to learning analytics. International Journal of Artificial Intelligence in Education, 32(3), 564–608. https://doi.org/10.1007/s40593-021-00254-2
Lin, L. (2018). Bias caused by sampling error in meta-analysis with small sample sizes. PLoS One, 13(9), e0204056. https://doi.org/10.1371/journal.pone.0204056
Martins, M. V., Tolledo, D., Machado, J., Baptista, L. M., & Realinho, V. (2021). Early prediction of student’s performance in higher education: A case study. In A. Rocha, H. Adeli, G. Dzemyda, F. Moreira, & A. Ramalho Correia (Eds.), Trends and applications in information systems and technologies. WorldCIST 2021. Advances in intelligent systems and computing (pp. 166–175, Vol. 1365). Springer. https://doi.org/10.1007/978-3-030-72657-7_16
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6), 1–35. https://doi.org/10.1145/3457607
Mitchell, S., Potash, E., Barocas, S., D’Amour, A., & Lum, K. (2021). Algorithmic fairness: Choices, assumptions, and definitions. Annual Review of Statistics and Its Application, 8(1), 141–163. https://doi.org/10.1146/annurev-statistics-042720-125902
Newcombe, R. G. (1998). Two-sided confidence intervals for the single proportion: Comparison of seven methods. Statistics in Medicine, 17(8), 857–872. https://doi.org/10.1002/(SICI)1097-0258(19980430)17:8⟨857::AID-SIM777⟩3.0.CO;2-E
Nunez, A. - M., Mayhew, M. J., Shaheen, M., & McChesney, E. (2023). Critical quantitative intersectionality: Maximizing integrity in expanding tools and applications. In M. D. Young & S. Diem (Eds.), Handbook of critical education research (pp. 430–451). Routledge. https://doi.org/10.4324/9781003141464-25
Olsen, J. K., Aleven, V., & Rummel, N. (2015). Predicting student performance in a collaborative learning environment. In O. Santos, J. G. Boticario, C. Romero, M. Pechenizkiy, A. Merceron, P. Mitros, J. Luna, C. Mihaescu, P. Moreno, A. Hershkovitz, S. Ventura, & M. Desmarais (Eds.), Proceedings of the Eighth International Conference on Educational Data Mining (EDM 2015), 26–29 June 2015, Madrid, Spain (pp. 211–217). International Educational Data Mining Society. https://www.educationaldatamining.org/EDM2015/proceedings/edm2015_proceedings.pdf
Ranjeeth, S., Latchoumi, T. P., & Paul, P. V. (2020). A survey on predictive models of learning analytics. Procedia Computer Science, 167, 37–46. https://doi.org/10.1016/j.procs.2020.03.180
Ranstam, J. (2012). Why the P-value culture is bad and confidence intervals a better alternative. Osteoarthritis and Cartilage, 20(8), 805–808. https://doi.org/10.1016/j.joca.2012.04.001
Saleiro, P., Kuester, B., Hinkson, L., London, J., Stevens, A., Anisfeld, A., Rodolfa, K. T., & Ghani, R. (2018). Aequitas: A bias and fairness audit toolkit. arXiv preprint arXiv:1811.05577. https://doi.org//10.48550/arXiv.1811.05577
Salgado, J. F. (2018). Transforming the area under the normal curve (AUC) into Cohen’s d, Pearson’s rpb, odds-ratio, and natural log odds-ratio: Two conversion tables. European Journal of Psychology Applied to Legal Context, 10(1), 35–47. https://doi.org/10.5093/ejpalc2018a5
Scholtz, S. E. (2021). Sacrifice is a step beyond convenience: A review of convenience sampling in psychological research in Africa. SA Journal of Industrial Psychology, 47(1), 1–12. https://doi.org/10.4102/sajip.v47i0.1837
Sedgwick, P. (2012). What is sampling error? BMJ, 344. https://doi.org/10.1136/bmj.e4285
Sha, L., Rakovic, M., Das, A., Gasevic, D., & Chen, G. (2022). Leveraging class balancing techniques to alleviate algorithmic bias for predictive tasks in education. IEEE Transactions on Learning Technologies, 15(4), 481–492. https://doi.org/10.1109/TLT.2022.3196278
Singh, A. S., & Masuku, M. B. (2014). Sampling techniques & determination of sample size in applied statistics research: An overview. International Journal of Economics, Commerce and Management, 2(11), 1–22. https://ijecm.co.uk/wp-content/uploads/2014/11/21131.pdf
Smith, H. (2020). Algorithmic bias: Should students pay the price? AI & Society, 35(4), 1077–1078. https://doi.org/10.1007/s00146-020-01054-3
Smith, L. J. (2008). How ethical is ethical research? Recruiting marginalized, vulnerable groups into health services research. Journal of Advanced Nursing, 62(2), 248–257. https://doi.org/10.1111/j.1365-2648.2007.04567.x
Sullivan, G. M., & Feinn, R. (2012). Using effect size—or why the P value is not enough. Journal of Graduate Medical Education, 4(3), 279–282. https://doi.org/10.4300/JGME-D-12-00156.1
Suresh, K., & Chandrashekara, S. (2012). Sample size estimation and power analysis for clinical research studies. Journal of Human Reproductive Sciences, 5(1), 7–13. https://doi.org/10.4103/0974-1208.97779
Tempelaar, D. T., Heck, A., Cuypers, H., van der Kooij, H., & van de Vrie, E. (2013). Formative assessment and learning analytics. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (LAK 2013), 8–13 April 2013, Leuven, Belgium (pp. 205–209). ACM. https://doi.org/10.1145/2460296.2460337
Varoquaux, G. (2018). Cross-validation failure: Small sample sizes lead to large error bars. Neuroimage, 180, 68–77. https://doi.org/10.1016/j.neuroimage.2017.06.061
Wang, A., Ramaswamy, V. V., & Russakovsky, O. (2022). Towards intersectionality in machine learning: Including more identities, handling underrepresentation, and performing evaluation. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT 2022), 21–24 June 2022, Seoul, Korea (pp. 336–349). ACM. https://doi.org/10.1145/3531146.3533101
Wang, W., Dong, Y., Liu, X., Bai, Y., & Zhang, L. (2020). The effect of parents’ education on the academic and non-cognitive outcomes of their children: Evidence from China. Children and Youth Services Review, 117, 105307. https://doi.org/10.1016/j.childyouth.2020.105307
Zambrano, A. F., Zhang, J., & Baker, R. S. (2024). Investigating algorithmic bias on Bayesian knowledge tracing and carelessness detectors. In Proceedings of the 14th International Conference on Learning Analytics and Knowledge (LAK 2024), 18–22 March 2024, Tokyo, Japan (pp. 349–359). ACM. https://doi.org/10.1145/3636555.3636890
Zhang, J., Andres, J. M. A. L., Hutt, S., Baker, R. S., Ocumpaugh, J., Nasiar, N., Mills, C., Brooks, J., Sethuaman, S., & Young, T. (2022). Using machine learning to detect smart model cognitive operations in mathematical problem-solving process. Journal of Educational Data Mining, 14(3), 76–108. https://doi.org/10.5281/zenodo.7304763
Zollanvari, A., Kizilirmak, R. C., Kho, Y. H., & Hern ´andez-Torrano, D. (2017). Predicting students’ GPA and developing intervention strategies based on self-regulatory learning behaviors. IEEE Access, 5, 23792–23802. https://doi.org/10.1109/ACCESS.2017.2740980
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution 4.0 International License.
