Mapping the Landscape of Generative Artificial Intelligence in Learning Analytics
A Systematic Literature Review
DOI:
https://doi.org/10.18608/jla.2025.8591Keywords:
generative artificial intelligence, GenAI, learning analytics, educational data mining, systematic review, research paperAbstract
Generative artificial intelligence (GenAI) has opened new possibilities for designing learning analytics (LA) tools, gaining new insights about student learning processes and their environment, and supporting teachers in assessing and monitoring students. This systematic literature review maps the empirical research of 41 papers utilizing GenAI and LA and interprets the results through the lens of the LA/EDM process cycle. Currently, GenAI is mostly implemented to automate discourse coding, scoring, or classification tasks. Few papers used GenAI to generate data or to summarize text. Classroom integrations of GenAI and LA mostly explore facilitating human–GenAI collaboration, rather than implementing automated feedback generation or GenAI-powered learning analytics dashboards. Most papers use Generative Adversarial Network models to generate synthetic data, BERT models for classification or prediction tasks, BERT or GPT models for discourse coding, and GPT models for tool integration. Although most studies evaluate the GenAI output, we found examples of using GenAI without the output validation, especially when its output feeds into an LA pipeline aiming to, for example, develop a dashboard. This review offers a comprehensive overview of the field to aid LA researchers in the design of research studies and a contribution to establishing best practices to integrate GenAI and LA.
References
Bozkurt, A. (2024). Why generative AI literacy, why now and why it matters in the educational landscape? Kings, queens and GenAI dragons. Open Praxis, 16(3), 283–290. https://doi.org/10.55982/openpraxis.16.3.739
Clow, D. (2012). The learning analytics cycle: Closing the loop effectively. In S. Dawson, C. Haythornthwaite, S. Buckingham Shum, D. Gašević, & R. Ferguson (Eds.), LAK ’12: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 134–138). ACM Press. https://doi.org/10.1145/2330601.2330636
Crompton, H., & Burke, D. (2024). The educational affordances and challenges of ChatGPT: State of the field. TechTrends, 68(2), 380–392. https://doi.org/10.1007/s11528-024-00939-0
Dorodchi, M., Al-Hossami, E., Benedict, A., & Demeter, E. (2019). Using synthetic data generators to promote open science in higher education learning analytics. In R. Barga & C. Zaniolo (Eds.), 2019 IEEE International Conference on Big Data (Big Data 2019) (pp. 4672–4675). IEEE. https://doi.org/10.1109/bigdata47090.2019.9006475
Ferguson, R. (2019). Ethical challenges for learning analytics. Journal of Learning Analytics, 6(3), 25–30. https://doi.org/10.18608/jla.2019.63.5
García-Peñalvo, F. J., & Vázquez-Ingelmo, A. (2023). What do we mean by GenAI? A systematic mapping of the evolution, trends, and techniques involved in generative AI. International Journal of Interactive Multimedia and Artificial Intelligence, 8(4), 7–16. https://doi.org/10.9781/ijimai.2023.07.006
Giannakos, M., Azevedo, R., Brusilovsky, P., Cukurova, M., Dimitriadis, Y., Hernandez-Leo, D., Järvelä, S., Mavrikis, M., & Rienties, B. (2024). The promise and challenges of generative AI in education. Behaviour & Information Technology, 1–27. https://doi.org/10.1080/0144929X.2024.2394886
González-Calatayud, V., Prendes-Espinosa, P., & Roig-Vila, R. (2021). Artificial intelligence for student assessment: A systematic review. Applied Sciences, 11(12), Article 5467. https://doi.org/10.3390/app11125467
Guo, S., Zheng, Y., & Zhai, X. (2024). Artificial intelligence in education research during 2013–2023: A review based on bibliometric analysis. Education and Information Technologies, 29(13), 16387–16409. https://doi.org/10.1007/s10639-024-12491-8
Gupta, P., Ding, B., Guan, C., & Ding, D. (2024). Generative AI: A systematic review using topic modelling techniques. Data and Information Management, 8(2), Article 100066. https://doi.org/10.1016/j.dim.2024.100066
Haindl, P., & Weinberger, G. (2024). Does ChatGPT help novice programmers write better code? Results from static code analysis. IEEE Access, 12, 114146–114156. https://doi.org/10.1109/ACCESS.2024.3445432
Hopfenbeck, T. N., Zhang, Z., Sun, S. Z., Robertson, P., & McGrane, J. A. (2023). Challenges and opportunities for classroom-based formative assessment and AI: A perspective article. Frontiers in Education, 8, Article 1270700. https://doi.org/10.3389/feduc.2023.1270700
Johnson, N., & Phillips, M. (2018). Rayyan for systematic reviews. Journal of Electronic Resources Librarianship, 30(1), 46–48. https://doi.org/10.1080/1941126X.2018.1444339
Kaliisa, R., Kluge, A., & Mørch, A. I. (2022). Overcoming challenges to the adoption of learning analytics at the practitioner level: A critical analysis of 18 learning analytics frameworks. Scandinavian Journal of Educational Research, 66(3), 367–381. https://doi.org/10.1080/00313831.2020.1869082
Kumar, S., Musharaf, D., Musharaf, S., & Sagar, A. K. (2023). A comprehensive review of the latest advancements in large generative AI models. In R. N. Shaw, M. Paprzycki, & A. Ghosh (Eds.), Advanced communication and intelligent systems: Second international conference, ICACIS 2023, Warsaw, Poland, June 16–17, 2023, revised selected papers, part I (pp. 90–103). Springer Cham. https://doi.org/10.1007/978-3-031-45121-8_9
Lodge, J. M., de Barba, P., & Broadbent, J. (2023). Learning with generative artificial intelligence within a network of co-regulation. Journal of University Teaching and Learning Practice, 20(7), Article 02. https://doi.org/10.53761/1.20.7.02
Luckin, R., Cukurova, M., Kent, C., & du Boulay, B. (2022). Empowering educators to be AI-ready. Computers and Education: Artificial Intelligence, 3, Article 100076. https://doi.org/10.1016/j.caeai.2022.100076
McCalla, G. (2023). The history of artificial intelligence in education: The first quarter century. In B. du Boulay, A. Mitrovic, & K. Yacef (Eds.), Handbook of artificial intelligence in education (pp. 10–29). Edward Elgar Publishing. https://doi.org/10.4337/9781800375413.00010
Misiejuk, K., López-Pernas, S., Kaliisa, R., & Saqr, M. (2024). Learning together: Modeling the process of student–AI interactions to generate learning resources. ResearchGate. https://www.researchgate.net/publication/387864942_Learning_together_Modeling_the_process_of_student-AI_interactions_when_generating_learning_resources
Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187–192. https://doi.org/10.1126/science.adh2586
Ouyang, F., & Zhang, L. (2024). AI-driven learning analytics applications and tools in computer-supported collaborative learning: A systematic review. Educational Research Review, 44, Article 100616. https://doi.org/10.1016/j.edurev.2024.100616
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W. Mayo-Wilson, E., McDonald, S., ... Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. International Journal of Surgery, 88, Article 105906. https://doi.org/10.1016/j.ijsu.2021.105906
Perrotta, C., & Selwyn, N. (2020). Deep learning goes to school: Toward a relational understanding of AI in education. Learning, Media and Technology, 45(3), 251–269. https://doi.org/10.1080/17439884.2020.1686017
Pressey, S. L. (1926). A simple apparatus which gives tests and scores-and teaches. School and Society, 23, 373–376.
Romero, C., & Ventura, S. (2020). Educational data mining and learning analytics: An updated survey. WIREs Data Mining and Knowledge Discovery, 10(3), Article e1355. https://doi.org/10.1002/widm.1355
Saqr, M. (2018). Using learning analytics to understand and support collaborative learning [Doctoral dissertation, Stockholm University]. DiVA. https://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-159479
Saqr, M., Cheng, R., López-Pernas, S., & Beck, E. D. (2024). Idiographic artificial intelligence to explain students’ self-regulation: Toward precision education. Learning and Individual Differences, 114, Article 102499. https://doi.org/10.1016/j.lindif.2024.102499
Schöbel, S., Schmitt, A., Benner, D., Saqr, M., Janson, A., & Leimeister, J. M. (2024). Charting the evolution and future of conversational agents: A research agenda along five waves and new frontiers. Information Systems Frontiers, 26(2), 729–754. https://doi.org/10.1007/s10796-023-10375-9
Sengar, S. S., Hasan, A. B., Kumar, S., & Carroll, F. (2024). Generative artificial intelligence: A systematic review and applications. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-024-20016-1
Skinner, B. F. (1958). Teaching machines: From the experimental study of learning come devices which arrange optimal conditions for self-instruction. Science, 128(3330), 969–977. https://doi.org/10.1126/science.128.3330.969
Suppes, P. (1966). The uses of computers in education. Scientific American, 215(3), 206–223. https://doi.org/10.1038/scientificamerican0966-206
Xia, Q., Weng, X., Ouyang, F., Lin, T. J., & Chiu, T. K. F. (2024). A scoping review on how generative artificial intelligence transforms assessment in higher education. International Journal of Educational Technology in Higher Education, 21(1), Article 40. https://doi.org/10.1186/s41239-024-00468-z
Yan, L., Martinez-Maldonado, R., & Gašević, D. (2024). Generative artificial intelligence in learning analytics: Contextualising opportunities and challenges through the learning analytics cycle. In B. Flanagan, B. Wasson, & D. Gašević (Eds.), LAK ’24: Proceedings of the 14th Learning Analytics and Knowledge Conference (pp. 101–111). ACM Press. https://doi.org/10.1145/3636555.3636856
Yu, H. (2024). The application and challenges of ChatGPT in educational transformation: New demands for teachers’ roles. Heliyon, 10(2), Article e24289. https://doi.org/10.1016/j.heliyon.2024.e24289
Yusuf, A., Pervin, N., Román‐González, M., & Noor, N. M. (2024). Generative AI in education and research: A systematic mapping review. Review of Education, 12(2), Article e3489. https://doi.org/10.1002/rev3.3489
Zhan, C., Deho, O. B., Zhang, X., Joksimović, S., & de Laat, M. (2023). Synthetic data generator for student data serving learning analytics: A comparative study. Learning Letters, 1, Article 5. https://doi.org/10.59453/KHZW9006
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution 4.0 International License.
