Game-Based Learning Analytics for Supporting Adolescents’ Reflection


  • Elizabeth Cloude University of Central Florida
  • Dan Carpenter North Carolina State University
  • Daryn A. Dever University of Central Florida
  • James Lester North Carolina State University
  • Roger Azevedo University of Central Florida



reflection, game-learning analytics, adolescents, problem solving, knowledge acquisition


Reflection is critical for adolescents’ problem solving and learning in game-based learning environments (GBLEs). Yet challenges exist in the literature because most studies lack a theoretical perspective and clear operational definition to inform how and when reflection should be scaffolded during game-based learning. In this paper, we address these issues by studying the quantity and quality of 120 adolescents’ written reflections and their relation to their learning and problem solving with Crystal Island, a GBLE. Specifically, we (1) define reflection and how it relates to skill and knowledge acquisition; (2) review studies examining reflection and its relation to problem solving and learning with emerging technologies; and (3) provide direction for building reflection prompts into GBLEs that are aligned with the learning goals built into the learning session (e.g., learn about microbiology versus successfully solve a problem) to maximize adolescents’ reflection, learning, and performance. Overall, our findings emphasize how important it is to examine not only the quantity of reflection but also the depth of written reflection as it relates to specific learning goals. We discuss the implications of using game-learning analytics to guide instructional decision making in the classroom.


Ak, O., & Kutlu, B. (2017). Comparing 2D and 3D game-based learning environments in terms of learning gains and student perceptions. British Journal of Educational Technology, 48(1), 129–144.

Alzaid, M., & Hsiao, I.-H. (2018). Effectiveness of reflection on programming problem solving self-assessments. In Proceedings of the ieee frontiers in education conference (pp. 1–5). Piscataway, NJ, USA: IEEE.

Azevedo, R., Feyzi-Behnagh, R., Duffy, M., Harley, J., & Trevors, G. (2012). Metacognition and self-regulated learning in student-centered learning environments. In D. Jonassen & S. Land (Eds.), Theoretical Foundations of Learning Environments (pp. 171–197). New York, NY, USA: Routledge.

Azevedo, R., Taub, M., & Mudrick, N. (2018). Understanding and reasoning about real-time cognitive, affective, and metacognitive processes to foster self-regulation with advanced learning technologies. In D. H. Schunk & J. A. Greene (Eds.), Handbook of Self-Regulation of Learning and Performance (pp. 254–270). New York, NY, USA: Routledge.

Bannert, M., & Reimann, P. (2012). Supporting self-regulated hypermedia learning through prompts. Instructional Science, 40(1), 193–211.

Biswas, G., Segedy, J. R., & Bunchongchit, K. (2016). From design to implementation to practice a learning by teaching system: Betty’s Brain. International Journal of Artificial Intelligence in Education, 26(1), 350–364.

Brown, A., Bransford, J., Ferrara, R., & Campione, J. (1983). Learning, Remembering, and Understanding (Tech. Rep. No. 244). New York, NY, USA: Bolt, Beranek and Newman, Inc. and Center for the Study of Reading. Retrieved from

Carpenter, D., Cloude, E. B., Rowe, J., Azevedo, R., & Lester, J. (2021). Investigating student reflection during game-based learning in middle grades science. In Proceedings of the 11th International Conference on Learning Analytics and Knowledge (LAK 2021), 12–16 April 2021, online (pp. 280–291). New York, NY, USA: ACM.

Carpenter, D., Geden, M., Rowe, J., Azevedo, R., & Lester, J. (2020). Automated analysis of middle school students’ written reflections during game-based learning. In I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, & E. Mill´an (Eds.), Proceedings of the International Conference on Artificial Intelligence in Education (AIED 2020), 6–10 July 2020, Utrecht, Netherlands (pp. 67–78). Cham, Switzerland: Springer. 6

Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86(1), 79–122.

Cloude, E. B., Dever, D. A., Wiedbusch, M. D., & Azevedo, R. (2020). Quantifying scientific thinking using multichannel data with Crystal Island: Implications for individualized game-learning analytics. Frontiers in Education, 5, 572546.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. Retrieved from

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.

Dewey, J. (1923). Democracy and Education: An Introduction to the Philosophy of Education. Macmillan.

Dewey, J. (1933). How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process. Lexington, MA, USA: D. C. Heath and Co.

Fiorella, L., & Mayer, R. E. (2012). Paper-based aids for learning with a computer-based game. Journal of Educational Psychology, 104(4), 1074.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906–911.

Geden, M., Emerson, A., Carpenter, D., Rowe, J., Azevedo, R., & Lester, J. (2020). Predictive student modeling in game-based learning environments with word embedding representations of reflection. International Journal of Artificial Intelligence in Education, 7(3), 1–23.

Grubbs, F. E. (1969). Procedures for detecting outlying observations in samples. Technometrics, 11(1), 1–21.

Izu, C., & Alexander, B. (2018). Using unstructured practice plus reflection to develop programming/problem-solving fluency. In Proceedings of the 20th Australasian Computing Education Conference (ACE 2018), 30 January–2 February 2018, Brisbane, Australia (pp. 25–34). New York, NY, USA: ACM.

Johnson, C. I., & Mayer, R. E. (2010). Applying the self-explanation principle to multimedia learning in a computer-based game-like environment. Computers in Human Behavior, 26(6), 1246–1252.

Kovanović, V., Joksimović, S., Mirriahi, N., Blaine, E., Gašević, D., Siemens, G., & Dawson, S. (2018). Understand students’ self-reflections through learning analytics. In Proceedings of the Eighth International Conference on Learning Analytics and Knowledge (LAK 2018), 7–9 March 2018, Sydney, Australia (pp. 389–398). New York, NY, USA: ACM.

Luo, T., & Baaki, J. (2019). Scaffolding problem-solving and instructional design processes: Engaging students in reflection-in-action and external representations in three online courses. In M. Boboc & S. Koc¸ (Eds.), Student-Centered Virtual Learning Environments in Higher Education (pp. 40–69). Hershey, PA, USA: IGI Global.

Mangaroska, K., Sharma, K., Gasevic, D., & Giannakos, M. (2020). Multimodal learning analytics to inform learning design: Lessons learned from computing education. Journal of Learning Analytics, 7(3), 31–60.

Mayer, R. E. (2014). Computer games for learning: An evidence-based approach. Cambridge, MA, USA: MIT Press. McAlpine, L., Weston, C., Beauchamp, C., Wiseman, C., & Beauchamp, J. (1999). Building a metacognitive model of reflection. Higher Education, 37(2), 105–131.

McQuiggan, S. W., Goth, J., Ha, E., Rowe, J. P., & Lester, J. C. (2008). Student note-taking in narrative-centered learning environments: Individual differences and learning effects. In Proceedings of the International Conference on Intelligent Tutoring Systems (ITS 2008), 23–27 June 2008, Montreal, QC, Canada (pp. 510–519). New York, NY, USA: Springer. Retrieved from

Moreno, R., & Mayer, R. E. (2005). Role of guidance, reflection, and interactivity in an agent-based multimedia game. Journal of Educational Psychology, 97(1), 117–128. Retrieved from

Moshman, D. (2011). Adolescent Rationality and Development: Cognition, Morality, and Identity. New York, NY, USA: Taylor & Francis.

Moshman, D. (2013). Adolescent rationality. In R. M. Lerner & J. B. Benson (Eds.), Embodiment and Epigenesis: Theoretical and Methodological Issues in Understanding the Role of Biology within the Relational Developmental System (Vol. 45, pp. 155–183). Amsterdam, Netherlands: Elsevier. Retrieved from

NASEM. (2018). How People Learn II: Learners, Contexts, and Cultures. Washington, DC, USA: The National Academies Press. Retrieved from

NRC. (2012). Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. Washington, DC, USA: The National Academies Press.

OECD. (2016). PISA 2015 Results (Volume I): Excellence and Equity in Education. Paris, France: OECD Publishing. Retrieved from

Patel, N., Baker, S. G., & Scherer, L. D. (2019). Evaluating the cognitive reflection test as a measure of intuition/reflection, numeracy, and insight problem solving, and the implications for understanding real-world judgments and beliefs. Journal of Experimental Psychology: General, 148(12), 2129–2153.

Perez-Colado, I. J., Perez-Colado, V. M., Freire-Moran, M., Martinez-Ortiz, I., & Fernandez-Manjon, B. (2017). Integrating learning analytics into a game authoring tool. In Proceedings of the International Conference on Web-Based Learning (ICWL 2017), 20–22 September 2017, Cape Town, South Africa (pp. 51–61). New York, NY, USA: Springer.

Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory Into Practice, 41(4), 219–225.

Plass, J. L., Homer, B. D., & Kinzer, C. K. (2015). Foundations of game-based learning. Educational Psychologist, 50(4), 258–283.

Plass, J. L., Mayer, R. E., & Homer, B. D. (2020). Handbook of Game-Based Learning. Cambridge, MA, USA: MIT Press. Qian, M., & Clark, K. R. (2016). Game-based learning and 21st century skills: A review of recent research. Computers in Human Behavior, 63, 50–58.

R Core Team. (2019). R: A Language and Environment for Statistical Computing [Computer software manual]. Vienna, Austria. Retrieved from

Roll, I., & Winne, P. H. (2015). Understanding, evaluating, and supporting self-regulated learning using learning analytics. Journal of Learning Analytics, 2(1), 7–12.

Rosenthal, D. (2000). Introspection and self-interpretation. Philosophical Topics, 28(2), 201–233. Retrieved from

Rowe, J. P., Shores, L. R., Mott, B. W., & Lester, J. C. (2011). Integrating learning, problem solving, and engagement in narrative-centered learning environments. International Journal of Artificial Intelligence in Education, 21(1-2), 115–133.

Scheffel, M., Drachsler, H., Stoyanov, S., & Specht, M. (2014). Quality indicators for learning analytics. Journal of Educational Technology & Society, 17(4), 117–132. Retrieved from

Shavelson, R. J., & Stern, P. (1981). Research on teachers’ pedagogical thoughts, judgments, decisions, and behavior. Review of Educational Research, 51(4), 455–498.

Siadaty, M., Gasevic, D., & Hatala, M. (2016). Trace-based micro-analytic measurement of self-regulated learning processes. Journal of Learning Analytics, 3(1), 183–214. Retrieved from

Tarricone, P. (2011). The Taxonomy of Metacognition. London, England, UK: Psychology Press. Retrieved from

Taub, M., Azevedo, R., Bradbury, A. E., & Mudrick, N. V. (2020). Self-regulation and reflection during game-based learning. In J. L. Plass, R. E. Mayer, & B. D. Homer (Eds.), Handbook of Game-Based Learning (pp. 239–262). Cambridge, MA, USA: MIT Press.

Taub, M., Sawyer, R., Smith, A., Rowe, J., Azevedo, R., & Lester, J. (2020). The agency effect: The impact of student agency on learning, emotions, and problem-solving behaviors in a game-based learning environment. Computers & Education, 147, 103781.

Ullmann, T. D. (2019). Automated analysis of reflection in writing: Validating machine learning approaches. International Journal of Artificial Intelligence in Education, 29(2), 217–257.

Van Rossum, G., & Drake, F. L. (2011). The Python Language Reference Manual [Computer software manual]. Network Theory Ltd.

Venables, W. N., & Ripley, B. D. (2002). Modern Applied Statistics with S (Fourth ed.). New York, NY, USA: Springer. Retrieved from

Viberg, O., Khalil, M., & Baars, M. (2020). Self-regulated learning and learning analytics in online learning environments: A review of empirical research. In Proceedings of the 10th international conference on learning analytics and knowledge (pp. 524–533). New York, NY, USA: ACM. Retrieved from

Vrugte, J. t., Jong, T. d., Wouters, P., Vandercruysse, S., Elen, J., & van Oostendorp, H. (2015). When a game supports prevocational math education but integrated reflection does not. Journal of Computer Assisted Learning, 31(5), 462–480.

Wang, H.-H., Chen, H.-T., Lin, H.-S., & Hong, Z.-R. (2017). The effects of college students’ positive thinking, learning motivation and self-regulation through a self-reflection intervention in Taiwan. Higher Education, Research, & Development, 36(1), 201–216.

Wickham, H. (2007). Reshaping data with the reshape package. Journal of Statistical Software, 21(12), 1–20. Retrieved from

Wickham, H. (2016). ggplot2: Elegant Graphics for Data Analysis. New York, NY, USA: Springer.

Wickham, H., & Bryan, J. (2019). readxl: Read Excel Files [Computer software manual]. (R package version 1.3.1) Retrieved from

Wickham, H., François, R., Henry, L., & Müller, K. (2020). dplyr: A Grammar of Data Manipulation [Computer software manual]. (R package version 0.8.5)

Wiedbusch, M. D., Kite, V., Yang, X., Park, S., Chi, M., Taub, M., & Azevedo, R. (2021). A theoretical and evidence-based conceptual design of MetaDash: An intelligent teacher dashboard to support teachers’ decision making and students’ self-regulated learning. Frontiers in Education, 6, 14.

Winne, P. H. (2017). Learning analytics for self-regulated learning. In C. Lang, G. Siemens, A. Wise, & D. Gašević (Eds.), Handbook of Learning Analytics (pp. 241–249). New York, NY, USA: Society for Learning Analytics Research.

Winne, P. H., Teng, K., Chang, D., Lin, M. P.-C., Marzouk, Z., Nesbit, J. C., . . . Vytasek, J. (2019). nStudy: Software for learning analytics about processes for self-regulated learning. Journal of Learning Analytics, 6(2), 95–106. Retrieved from

Wu, L., & Looi, C.-K. (2012). Agent prompts: Scaffolding for productive reflection in an intelligent learning environment. Journal of Educational Technology & Society, 15(1), 339–353. Retrieved from

Yamashita, T., Yamashita, K., & Kamimura, R. (2007). A stepwise AIC method for variable selection in linear regression. Communications in Statistics—Theory and Methods, 36(13), 2395–2403.

Zimmerman, B. J. (2013). From cognitive modeling to self-regulation: A social cognitive career path. Educational Psychologist, 48(3), 135–147. Retrieved from




How to Cite

Cloude, E., Carpenter, D., Dever, D. A., Lester, J., & Azevedo, R. (2021). Game-Based Learning Analytics for Supporting Adolescents’ Reflection. Journal of Learning Analytics, 8(2), 51-72.



Special Section: Learning Analytics for Primary and Secondary Schools