Associations of Research Questions, Analytical Techniques, and Learning Insight in Temporal Educational Research

A Systematic Mapping Study




learning analytics, systematic mapping, temporal analytics, research paper


Learning has a temporal characteristic in nature, which means that it occurs over the passage of time. The research on the temporal aspects of learning faces several challenges, one of which is utilizing appropriate analytical techniques to exploit the temporal data. There is no coherent guide to selecting certain temporal techniques to lead to results that truthfully uncover underlying phenomena. To fill this gap, this systematic mapping study contributes to understanding the type of questions and approaches in works in the area of temporal educational research. This study aims to analyze different components of published research and explores the current trends in educational studies that explicitly consider the temporal aspect. Using the thematic coding method, we identified trends in three components, including asked research questions, utilized methodological techniques, and inferred insight about learning. The distribution of codes regarding asked research questions showed that the highest number of studies focused on method development or proposing a methodological framework. We discussed that methodological development, with the underlying theory, led to identifying learning indicators that can provide the ability to identify individual students with respect to the learning concepts of interest. In terms of utilized techniques, there was a strong trend in visualization analysis and process mining. This study found that to discover insight into learning, it is important to utilize techniques that are interpretable to characterize temporal patterns.


Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to sequential analysis (2nd ed.). Cambridge University Press.

Basit, T. (2003). Manual or electronic? The role of coding in qualitative data analysis. Educational Research, 45(2), 143–154.

Beheshitha, S. S., Gašević, D., & Hatala, M. (2015). A process mining approach to linking the study of aptitude and event facets of self-regulated learning. Proceedings of the 5th International Conference on Learning Analytics and Knowledge (LAK ʼ15), 16–20 March 2015, Poughkeepsie, NY, USA (pp. 265–269). ACM Press.

Bienkowski, M., Feng, M., & Means, B. (2014). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. U.S. Department of Education.

Bogarín, A., Cerezo, R., & Romero, C. (2018). A survey on educational process mining. WIREs Data Mining and Knowledge Discovery, 8(1).

Shirvani Boroujeni, M., & Dillenbourg, P. (2019). Discovery and temporal analysis of MOOC study patterns. Journal of Learning Analytics, 6(1), 16–33.

Chen, B., Knight, S., & Wise, A. F. (2018). Critical issues in designing and implementing temporal analytics. Journal of Learning Analytics, 5(1), 1–9.

Chen, B., Resendes, M., Chai, C. S., & Hong, H.-Y. (2017). Two tales of time: Uncovering the significance of sequential patterns among contribution types in knowledge-building discourse. Interactive Learning Environments, 25(2), 162–175.

Chen, B., Wise, A. F., Knight, S., & Cheng, B. H. (2016). Putting temporal analytics into practice: The 5th international workshop on temporality in learning data. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 488–489). ACM Press.

de Barba, P. G., Malekian, D., Oliveira, E. A., Bailey, J., Ryan, T., & Kennedy, G. (2020). The importance and meaning of session behaviour in a MOOC. Computers & Education, 146, 103772.

Dickersin, K., Scherer, R., & Lefebvre, C. (1994). Systematic reviews: Identifying relevant studies for systematic reviews. BMJ, 309(6964), 1286–1291.

Du, X., Zhang, M., Shelton, B. E., & Hung, J.-L. (2022). Learning anytime, anywhere: A spatio-temporal analysis for online learning. Interactive Learning Environments, 30(1), 34–48.

Fan, Y., Saint, J., Singh, S., Jovanović, J., & Gašević, D. (2021). A learning analytic approach to unveiling self-regulatory processes in learning tactics. Proceedings of the 11th International Conference on Learning Analytics and Knowledge (LAK ’21), 12–16 April 2021, Irvine, CA, USA (pp. 184–195). ACM Press.

Gabadinho, A., Ritschard, G., Müller, N. S., & Studer, M. (2011). Analyzing and visualizing state sequences in R with TraMineR. Journal of Statistical Software, 40(4).

Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59, 64–71.

Gašević, D., Kovanović, V., & Joksimović, S. (2017). Piecing the learning analytics puzzle: A consolidated model of a field of research and practice. Learning: Research and Practice, 3(1), 63–78.

Google Scholar. (n.d.). Educational Technology – Google scholar metric.

Hatala, M., Nazeri, S., & Salehian Kia, F. (2023). Progression of students’ SRL processes in subsequent programming problem-solving tasks and its association with tasks outcomes. The Internet and Higher Education, 56, 100881.

Huang, L., & Lajoie, S. P. (2021). Process analysis of teachers’ self-regulated learning patterns in technological pedagogical content knowledge development. Computers & Education, 166, 104169.

Jovanović, J., Dawson, S., Joksimović, S., & Siemens, G. (2020). Supporting actionable intelligence: Reframing the analysis of observed study strategies. Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK ’20), 23–27 March 2020, Frankfurt, Germany (pp. 161–170). ACM Press.

Jovanović, J., Gašević, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017). Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education, 33, 74–85.

Kinnebrew, J. S., Segedy, J. R., & Biswas, G. (2014). Analyzing the temporal evolution of students’ behaviors in open-ended learning environments. Metacognition and Learning, 9, 187–215.

Kitchenham, B. A., Budgen, D., & Pearl Brereton, O. (2011). Using mapping studies as the basis for further research: A participant-observer case study. Information and Software Technology, 53(6), 638–651.

Knight, S., Wise, A. F., & Chen., B. (2017). Time for change: Why learning analytics needs temporal analysis. Journal of Learning Analytics, 4(3), 7–17.

Lee, A. V. Y., & Tan, S. C. (2017). Temporal analytics with discourse analysis: Tracing ideas and impact on communal discourse. Proceedings of the 7th International Conference on Learning Analytics and Knowledge (LAK ’17), 13–17 March 2017, Vancouver, BC, Canada (pp. 120–127). ACM Press.

Liu, S., Kang, L., Liu, Z., Fang, J., Yang, Z., Sun, J., Wang, M., & Hu, M. (2021). Computer-supported collaborative concept mapping: The impact of students’ perceptions of collaboration on their knowledge understanding and behavioral patterns. Interactive Learning Environments, 1–20.

Mahzoon, M. J., Maher, M. L., Eltayeby, O., Dou, W., & Grace, K. (2018). A sequence data model for analyzing temporal patterns of student data. Journal of Learning Analytics, 5(1), 55–74.

Matcha, W., Gašević, D., Ahmad Uzir, N., Jovanović, J., Pardo, A., Maldonado-Mahauad, J., & Pérez-Sanagustín, M. (2019). Detection of learning strategies: A comparison of process, sequence and network analytic approaches. Proceedings of the 14th European Conference on Technology Enhanced Learning (EC-TEL 2019), 16–19 September 2019, Delft, The Netherlands (Lecture Notes in Computer Science, vol. 11722, pp. 525–540). Springer.

Mohabbati, B., Asadi, M., Gašević, D., Hatala, M., & Müller, H. A. (2013). Combining service-orientation and software product line engineering: A systematic mapping study. Information and Software Technology, 55(11), 1845–1859.

Molenaar, I. (2014). Advances in temporal analysis in learning and instruction. Frontline Learning Research, 2(4), 15–24.

Nazeri, S., Hatala, M., & Salehian Kia, F. (2023). When to intervene? Utilizing two facets of temporality in students’ SRL processes in a programming course. Proceedings of the 13th International Conference on Learning Analytics and Knowledge (LAK ’22), 13–17 March 2023, Arlington, TX, USA (pp. 293–302). ACM Press.

Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1).

Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008). Systematic mapping studies in software engineering. Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering (EASE 2008), 26–27 June 2008, Bari, Italy (pp. 1–10). BCS Learning and Development.

Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64, 1–18.

Reimann, P. (2009). Time is precious: Variable- and event-centred approaches to process analysis in CSCL research. International Journal of Computer-Supported Collaborative Learning, 4, 239–257.

Reimann, P., Markauskaite, L., & Bannert, M. (2014). e-Research and learning theory: What do sequence and process mining methods contribute? British Journal of Educational Technology, 45(3), 528–540.

Riel, J., Lawless, K. A., & Brown, S. W. (2018). Timing matters: Approaches for measuring and visualizing behaviours of timing and spacing of work in self-paced online teacher professional development courses. Journal of Learning Analytics, 5(1), 25–40.

Saint, J., Fan, Y., Singh, S., Gašević, D., & Pardo, A. (2021). Using process mining to analyse self-regulated learning: A systematic analysis of four algorithms. Proceedings of the 11th International Conference on Learning Analytics and Knowledge (LAK ’21), 12–16 April 2021, Irvine, CA, USA (pp. 333–343). ACM Press.

Sher, N., Kent, C., & Rafaeli, S. (2020). How “networked” are online collaborative concept-maps? Introducing metrics for quantifying and comparing the “networkedness” of collaboratively constructed content. Education Sciences, 10(10), 267.

Wang, M., Guo, W., Le, H., & Qiao, B. (2020). Reply to which post? An analysis of peer reviews in a high school SPOC. Interactive Learning Environments, 28(5), 574–585.

Wang, Q., Saha, K., Gregori, E., Joyner, D., & Goel, A. (2021). Towards mutual theory of mind in human-AI interaction: How language reflects what students perceive about a virtual teaching assistant. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21), 8–13 May 2021, Yokohama, Japan (pp. 1–14). ACM Press.

Wise, A. F., & Shaffer, D. W. (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13.

Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25(1), 3–17.




How to Cite

Nazeri, S., Hatala, M., & Neustaedter, C. (2023). Associations of Research Questions, Analytical Techniques, and Learning Insight in Temporal Educational Research: A Systematic Mapping Study. Journal of Learning Analytics, 10(2), 68-84.



Research Papers