Learning Analytics to Uncover Ethnic Bias in Educational Texts
An Ensemble Learning Approach
DOI:
https://doi.org/10.18608/jla.2026.8905Keywords:
learning analytics, ethnic bias, machine learning, online learning, open educational resources, research paperAbstract
Online learning platforms have expanded access to education but also raise concerns about biased content, particularly in text-based learning materials such as textbooks, lesson plans, and course excerpts. Such biases can perpetuate discrimination, can harm student outcomes, and can often be difficult to detect, as identification typically relies on time-consuming human review. Learning analytics (LA) can enhance this process by supporting human reviewers through automated detection, offering a scalable solution while retaining human judgment for nuanced evaluations. Accordingly, this LA study explores two research questions: RQ1: Which features might support the identification of ethnic bias in text-based online learning materials? and RQ2: Which classification approaches might be suitable for identifying ethnic bias in text-based online learning materials? First, we identified features signalling potential ethnic bias (presence or absence) in textual content using a dataset (N = 345) labelled by 193 students from diverse ethnic backgrounds. Then, we evaluated multiple machine learning (ML) models for their effectiveness in bias classification. The results suggest significant correlations between perceived bias and content from social sciences. Additionally, through bootstrap analysis, support vector machines and random forest classifiers showed consistent performance in bias identification (with F1-scores of 0.71 and 0.70 on the test set, respectively). In contrast, the naive Bayes (NB) model demonstrated the highest precision (0.75 on the test set). We discuss these findings and their implications for LA, emphasizing the importance of quality and inclusive educational tools. As an initial step toward automated bias classification in education, this study provides a foundation for spotting ethnic bias in learning content, supporting fairer technologies for more inclusive learning environments.
References
Albuquerque, J. (2023). Towards an automatic approach for uncovering ethnic bias in online learning texts [Doctoral dissertation, The Open University]. https://doi.org/10.21954/ou.ro.000170d9
Albuquerque, J., Bittencourt, I. I., Coelho, J. A. P. M., & Silva, A. P. (2017). Does gender stereotype threat in gamified educational environments cause anxiety? An experimental study. Computers & Education, 115, 161–170. https://doi.org/10.1016/j.compedu.2017.08.005
Albuquerque, J., Rienties, B., Holmes, W., & Hlosta, M. (2025). From hype to evidence: Exploring large language models for inter-group bias classification in higher education. Interactive Learning Environments, 33(3), 2332–2354. https://doi.org/10.1080/10494820.2024.2408554
Allen, I. E., & Seaman, J. (2007). Online nation: Five years of growth in online learning. ERIC. https://files.eric.ed.gov/fulltext/ED529699.pdf
Al-Zawqari, A., & Vandersteen, G. (2023). Fairness in predictive learning analytics: A case study in online STEM education. In Proceedings of the 2023 IEEE Frontiers in Education Conference (FIE 2023), 18–21 October 2023, College Station, Texas, USA (pp. 1–5). IEEE. https://doi.org/10.1109/FIE58773.2023.10343059
Ashong, C. Y., & Commander, N. E. (2012). Ethnicity, gender, and perceptions of online learning in higher education. MERLOT Journal of Online Learning and Teaching, 8(2), 98–110. https://jolt.merlot.org/vol8no2/ashong_0612.pdf
Baker, R. S., & Hawn, A. (2022). Algorithmic bias in education. International Journal of Artificial Intelligence in Education, 32(4), 1052–1092. https://doi.org/10.1007/s40593-021-00285-9
Balica, R. (2018). Big data learning analytics and algorithmic decision-making in digital education governance. Analysis and Metaphysics, 17, 128–133. https://doi.org/10.22381/AM1720187
Bansak, C., & Starr, M. (2021). Covid-19 shocks to education supply: How 200,000 US households dealt with the sudden shift to distance learning. Review of Economics of the Household, 19(1), 63–90. https://doi.org/10.1007/s11150-020-09540-9
Baumeister, R., & Vohs, K. (2007). Ingroup–outgroup bias. In Encyclopedia of social psychology (pp. 484–485). SAGE Publications, Inc. https://doi.org/10.4135/9781412956253.n286
Beaunoyer, E., Dupere, S., & Guitton, M. J. (2020). COVID-19 and digital inequalities: Reciprocal impacts and mitigation strategies. Computers in Human Behavior, 111, 106424. https://doi.org/10.1016/j.chb.2020.106424
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x
Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13(2), 281–305. https://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf
Berrar, D. (2019). Cross-validation. In S. Ranganathan, M. Gribskov, K. Nakai, & C. Schonbach (Eds.), Encyclopedia of bioinformatics and computational biology (pp. 542–545, Vol. 1). Academic Press. https://doi.org/10.1016/B978-0-12-809633-8.20349-X
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N., Chen, A., Creel, K., Davis, J. Q., Demszky, D., . . . Liang, P. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258. https://doi.org/10.48550/arXiv.2108.07258
Borchers, C., & Baker, R. S. (2025). ABROCA distributions for algorithmic bias assessment: Considerations around interpretation. In Proceedings of the 15th International Conference on Learning Analytics and Knowledge (LAK 2025), 3–7 March 2025, Dublin, Ireland (pp. 837–843). ACM. https://doi.org/10.1145/3706468.3706498
Brewer, M., & Yuki, M. (2007). Culture and social identity [PsycINFO ID: 2007-12976-012]. In S. Kitayama & D. Cohen (Eds.), Handbook of cultural psychology (1st ed., pp. 307–322). The Guilford Press.
Chandrashekar, G., & Sahin, F. (2014). A survey on feature selection methods. Computers & Electrical Engineering, 40(1), 16–28. https://doi.org/10.1016/j.compeleceng.2013.11.024
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321–357. https://doi.org/10.1613/jair.953
Chen, F., & Cui, Y. (2020). Utilizing student time series behaviour in learning management systems for early prediction of course performance. Journal of Learning Analytics, 7(2), 1–17. https://doi.org/10.18608/jla.2020.72.1
Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2016), 13–17 August 2016, San Francisco, California, USA (pp. 785–794). ACM. https://doi.org/10.1145/2939672.2939785
Chiu, C. - y., Hong, Y.- y., & Dweck, C. S. (1997). Lay dispositionism and implicit theories of personality. Journal of Personality and Social Psychology, 73(1), 19–30. https://doi.org/10.1037//0022-3514.73.1.19
Choi, J., Karumbaiah, S., & Matayoshi, J. (2025). Bias or insufficient sample size? Improving reliable estimation of algorithmic bias for minority groups. In Proceedings of the 15th International Conference on Learning Analytics and Knowledge (LAK 2025), 3–7 March 2025, Dublin, Ireland (pp. 547–557). ACM. https://doi.org/10.1145/3706468.3706540
Copur-Gencturk, Y., Thacker, I., & Cimpian, J. R. (2022). Teacher bias in the virtual classroom. Computers & Education, 191, 104627. https://doi.org/10.1016/j.compedu.2022.104627
Dai, S., Xu, C., Xu, S., Pang, L., Dong, Z., & Xu, J. (2024). Bias and unfairness in information retrieval systems: New challenges in the LLM era. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2024), 25–29 August 2024, Barcelona, Spain (pp. 6437–6447). ACM. https://doi.org/10.1145/3637528.3671458
Daumeyer, N. M., Onyeador, I. N., Brown, X., & Richeson, J. A. (2019). Consequences of attributing discrimination to implicit vs. explicit bias. Journal of Experimental Social Psychology, 84, 103812. https://doi.org/10.1016/j.jesp.2019.04.010
Dawkins, H., Hedgeland, H., & Jordan, S. (2017). Impact of scaffolding and question structure on the gender gap. Physical Review Physics Education Research, 13(2), 020117. https://doi.org/10.1103/PhysRevPhysEducRes.13.020117
Doube, W., & Lang, C. (2012). Gender and stereotypes in motivation to study computer programming for careers in multimedia. Computer Science Education, 22(1), 63–78. https://doi.org/10.1080/08993408.2012.666038
Dovidio, J. F., & Gaertner, S. L. (2010). Intergroup bias. In Handbook of social psychology (pp. 1084–1121). John Wiley & Sons, Inc. https://doi.org/10.1002/9780470561119.socpsy002029
Dutt, A., Ismail, M. A., & Herawan, T. (2017). A systematic review on educational data mining. IEEE Access, 5, 15991–16005. https://doi.org/10.1109/ACCESS.2017.2654247
Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6), 304–317. https://doi.org/10.1504/IJTEL.2012.051816
Fernandez, A., Garcia, S., Herrera, F., & Chawla, N. V. (2018). SMOTE for learning from imbalanced data: Progress and challenges, marking the 15-year anniversary. Journal of Artificial Intelligence Research, 61, 863–905. https://doi.org/10.1613/jair.1.11192
Greenwald, A. G., & Krieger, L. H. (2006). Implicit bias: Scientific foundations. California Law Review, 94(4), 945–967. https://doi.org/10.2307/20439056
Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3(Mar), 1157–1182. https://dl.acm.org/doi/10.5555/944919.944968
Hollister, B., Nair, P., Hill-Lindsay, S., & Chukoskie, L. (2022). Engagement in online learning: Student attitudes and behavior during COVID-19. Frontiers in Education, 7, 851019. https://doi.org/10.3389/feduc.2022.851019
Holstein, K., & Doroudi, S. (2019). Fairness and equity in learning analytics systems (FairLAK). In Workshop at the Ninth International Conference on Learning Analytics and Knowledge (LAK 2019), 4–8 March 2019, Tempe, Arizona, USA. https://sites.google.com/view/fairlak
Holstein, K., & Doroudi, S. (2022). Equity and artificial intelligence in education. In W. Holmes & K. Porayska-Pomsta (Eds.), The ethics of artificial intelligence in education (pp. 151–173). Routledge. https://www.taylorfrancis.com/chapters/edit/10.4324/9780429329067-9/equity-artificial-intelligence-education-kenneth-holstein-shayan-doroudi
Hutt, S., Baker, R. S., Ashenafi, M. M., Andres-Bray, J. M., & Brooks, C. (2022). Controlled outputs, full data: A privacy-protecting infrastructure for mooc data. British Journal of Educational Technology, 53(4), 756–775. https://doi.org/10.1111/bjet.13231
Jordano, M. L., & Touron, D. R. (2017). Stereotype threat as a trigger of mind-wandering in older adults. Psychology and Aging, 32(3), 307. https://doi.org/10.1037/pag0000167
Karumbaiah, S., & Brooks, J. (2021). How colonial continuities underlie algorithmic injustices in education. In C. Gardner-McCune, S. Grady, Y. Jimenez, J. Ryoo, R. Santo, & J. Payton (Eds.), Proceedings of the 2021 Conference on Research in Equitable and Sustained Participation in Engineering, Computing, and Technology (RESPECT 2021), 23–27 May 2021, online (pp. 1–6). IEEE. https://doi.org/10.1109/RESPECT51740.2021
Kizilcec, R. F., & Lee, H. (2022). Algorithmic fairness in education. In W. Holmes & K. Porayska-Pomsta (Eds.), The ethics of artificial intelligence in education (pp. 174–202). Routledge. https://www.taylorfrancis.com/chapters/edit/10.4324/9780429329067-10/algorithmic-fairness-education-ren
Kumar, D., Jain, U., Agarwal, S., & Harshangi, P. (2024). Investigating implicit bias in large language models: A large-scale study of over 50 LLMs. arXiv preprint arXiv:2410.12864. https://doi.org/10.48550/arXiv.2410.12864
Liao, Q. V., & Varshney, K. R. (2021). Human-centered explainable AI (XAI): From algorithms to user experiences. arXiv preprint arXiv:2110.10790. https://doi.org/10.48550/arXiv.2110.10790
Long, P., Siemens, G., Grainne, C., & Gasevic, D. (2011). 1st International Conference on Learning Analytics and Knowledge. In Proceedings of the First International Conference on Learning Analytics and Knowledge (LAK 2011), 27 February–1 March 2011, Banff, Alberta, Canada (pp. 3–4). ACM. https://dl.acm.org/doi/proceedings/10.1145/2090116
Maass, A. (1999). Linguistic intergroup bias: Stereotype perpetuation through language. In M. P. Zanna (Ed.), Advances in experimental social psychology (pp. 79–121, Vol. 31). Elsevier. https://doi.org/10.1016/S0065-2601(08)60272-5
Mayer, R. E. (2019). How multimedia can improve learning and instruction. In J. Dunlosky & K. A. Rawson (Eds.), The Cambridge handbook of cognition and education (pp. 460–479). Cambridge University Press. https://doi.org/10.1017/9781108235631.019
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6), 1–35. https://doi.org/10.1145/3457607
Mendelsohn, J., Tsvetkov, Y., & Jurafsky, D. (2020). A framework for the computational linguistic analysis of dehumanization. Frontiers in Artificial Intelligence, 3, 55. https://doi.org/10.3389/frai.2020.00055
Mohamed, S., Png, M.- T., & Isaac, W. (2020). Decolonial AI: Decolonial theory as sociotechnical foresight in artificial intelligence. Philosophy & Technology, 33, 659–684. https://doi.org/10.1007/s13347-020-00405-8
Namoun, A., & Alshanqiti, A. (2020). Predicting student performance using data mining and learning analytics techniques: A systematic literature review. Applied Sciences, 11(1), 237. https://doi.org/10.3390/app11010237
Nguyen, A., Ngo, H. N., Hong, Y., Dang, B., & Nguyen, B.- P. T. (2023). Ethical principles for artificial intelligence in education. Education and Information Technologies, 28(4), 4221–4241. https://doi.org/10.1007/s10639-022-11316-w
Nguyen, Q., Rienties, B., & Richardson, J. T. E. (2020). Learning analytics to uncover inequality in behavioural engagement and academic attainment in a distance learning setting. Assessment & Evaluation in Higher Education, 45(4), 594–606. https://doi.org/10.1080/02602938.2019.1679088
Osgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1957). The measurement of meaning. University of Illinois Press. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. https://jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf
Pennington, C. R., Heim, D., Levy, A. R., & Larkin, D. T. (2016). Twenty years of stereotype threat research: A review of psychological mediators. PLoS ONE, 11(1), 1–25. https://doi.org/10.1371/journal.pone.0146487
Pryzant, R., Martinez, R. D., Dass, N., Kurohashi, S., Jurafsky, D., & Yang, D. (2020). Automatically neutralizing subjective bias in text. Proceedings of the AAAI Conference on Artificial Intelligence, 34(01), 480–489. https://doi.org/10.1609/aaai.v34i01.5385
Richardson, J. T. E., Mittelmeier, J., & Rienties, B. (2020). The role of gender, social class and ethnicity in participation and academic attainment in UK higher education: An update. Oxford Review of Education, 46(3), 346–362. https://doi.org/10.1080/03054985.2019.1702012
Sabnis, S., Yu, R., & Kizilcec, R. F. (2022). Large-scale student data reveal sociodemographic gaps in procrastination behavior. In Proceedings of the Ninth ACM Conference on Learning at Scale (L@S 2022), 1–3 June 2022, New York, New York, USA (pp. 133–141). ACM. https://doi.org/10.1145/3491140.3528285
Sagi, O., & Rokach, L. (2018). Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8(4), e1249. https://doi.org/10.1002/widm.1249
Sboev, A., Gudovskikh, D., Rybka, R., & Moloshnikov, I. (2015). A quantitative method of text emotiveness evaluation on base of the psycholinguistic markers founded on morphological features. Procedia Computer Science, 66, 307–316. https://doi.org/10.1016/j.procs.2015.11.036
Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., & Kirschner, P. A. (2020). Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior, 107, 105512. https://doi.org/10.1016/j.chb.2018.05.004
Shahjahan, R. A., Estera, A. L., Surla, K. L., & Edwards, K. T. (2022). “Decolonizing” curriculum and pedagogy: A comparative review across disciplines and global higher education contexts. Review of Educational Research, 92(1), 73–113. https://doi.org/10.3102/00346543211042423
Singhal, Y., Jain, A., Batra, S., Varshney, Y., & Rathi, M. (2018). Review of bagging and boosting classification performance on unbalanced binary classification. In A. Goswami (Ed.), Proceedings of the 2018 IEEE Eighth International Advance Computing Conference (IACC 2018), 14–15 December 2018, Delhi, India (pp. 338–343). IEEE. https://doi.org/10.1109/IADCC.2018.8692138
Skopec, M., Fyfe, M., Issa, H., Ippolito, K., Anderson, M., & Harris, M. (2021). Decolonization in a higher education STEM institution—is “epistemic fragility” a barrier? London Review of Education, 19(1), 1–21. https://doi.org/10.14324/LRE.19.1.18
Sloan-Lynch, J., & Morse, R. (2024). Equity-forward learning analytics: Designing a dashboard to support marginalized student success. In Proceedings of the 14th International Conference on Learning Analytics and Knowledge (LAK 2024), 18–22 March 2024, Tokyo, Japan (pp. 1–11). ACM. https://doi.org/10.1145/3636555.3636844
Tajfel, H. (1970). Experiments in intergroup discrimination. Scientific American, 223(5), 96–102. https://doi.org/10.1038/scientificamerican1170-96
Tajfel, H., Billig, M. G., Bundy, R. P., & Flament, C. (1971). Social categorization and intergroup behaviour. European Journal of Social Psychology, 1(2), 149–178. https://doi.org/10.1002/ejsp.2420010202
Tajfel, H., Turner, J. C., Austin, W. G., & Worchel, S. (2000). An integrative theory of intergroup conflict. In M. J. Hatch & M. Schultz (Eds.), Organizational identity: A reader (pp. 56–65). Oxford University Press. https://doi.org/10.1093/oso/9780199269464.003.0005
Tate, T., & Warschauer, M. (2022). Equity in online learning. Educational Psychologist, 57(3), 192–206. https://doi.org/10.1080/00461520.2022.2062597
The Pandas development team. (2020). Pandas-dev/pandas: Pandas. https://github.com/pandas-dev/pandas
Tincher, M. M., Lebois, L. A. M., & Barsalou, L. W. (2016). Mindful attention reduces linguistic intergroup bias. Mindfulness, 7(2), 349–360. https://doi.org/10.1007/s12671-015-0450-3
Yu, R., Lee, H., & Kizilcec, R. F. (2021). Should college dropout prediction models include protected attributes? In Proceedings of the Eighth ACM Conference on Learning at Scale (L@S 2021), 22–25 June 2021, online (pp. 91–100). ACM. https://doi.org/10.1145/3430895.3460139
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution 4.0 International License.
