Transmodal Analysis
DOI:
https://doi.org/10.18608/jla.2025.8423Keywords:
multimodal data, transmodal analysis, data fusion, data transfusion, temporal influence functions, horizon functions, learner impact functions, research paperAbstract
Learning is a multimodal process, and learning analytics (LA) researchers can readily access rich learning process data from multiple modalities, including audio-video recordings or transcripts of in-person interactions; logfiles and messages from online activities; and biometric measurements such as eye-tracking, movement, and galvanic skin response. While many techniques are used in LA to model different types of learning process data—most of which are state-dependent (or state-space) approaches that model a learning process at any given time as a function of the preceding events—constructing multimodal models has so far relied on fusion of different data streams, which converts multimodal data into a unimodal format. This creates a number of problems for multimodal modelling, the most important of which is that it treats different data modalities as equivalent. That is, existing state-dependent models of fused data cannot easily account for (a) events that may have different impacts on future events based on what those future events are and the context in which they are occurring; (b) how events may influence some groups of learners differently; and (c) which events are visible (and thus potentially impactful) to which students. In this paper, we propose transmodal analysis (TMA), a mathematical and computational framework designed to address these challenges. TMA is not a data analysis method but rather an approach to modelling that can augment existing state-dependent models of learning processes to account for multimodal data without data fusion. We present a conceptual and methodological description of TMA, and we include an appendix with a detailed worked example as a proof of concept. While this approach is in the early stages of development, it has the potential to significantly improve the ease, efficiency, and fairness of multimodal analyses of learning processes.
References
Ahmad, S., Fatima, R., Mazhar, S. S., Bajpai, S., Yadav, R. R., & Kanaujia, D. S. (2023). Assessing the linkage between vocational education and economic growth using autoregression analysis: Evidence from India. Journal of Namibian Studies: History Politics Culture, 35, 434–449.
Airey, J., & Linder, C. (2009). A disciplinary discourse perspective on university science learning: Achieving fluency in a critical constellation of modes. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 46(1), 27–49. https://doi.org/10.1002/tea.20265
Akçapınar, G., & Hasnine, M. N. (2022). Discovering the effects of learning analytics dashboard on students’ behavioral patterns using differential sequence mining. Procedia Computer Science, 207, 3818–3825. https://doi.org/10.1016/j.procs.2022.09.443
Al-Adwan, A. S., Albelbisi, N. A., Hujran, O., Al-Rahmi, W. M., & Alkhalifah, A. (2021). Developing a holistic success model for sustainable e-learning: A structural equation modeling approach. Sustainability, 13(16), 9453. https://doi.org/10.3390/su13169453
Alibali, M. W., & Nathan, M. J. (2012). Embodiment in mathematics teaching and learning: Evidence from learners’ and teachers’ gestures. Journal of the Learning Sciences, 21(2), 247–286. https://doi.org/10.1080/10508406.2011.611446
Alkabbany, I., Ali, A. M., Foreman, C., Tretter, T., Hindy, N., & Farag, A. (2023). An experimental platform for real-time students engagement measurements from video in STEM classrooms. Sensors, 23(3), 1614. https://doi.org/10.3390/s23031614
Almond, R., Kim, Y. J., Shute, V. J., & Ventura, M. (2013). Debugging the evidence chain. In Proceedings of the 2013 UAI Application Workshops: Big Data Meet Complex Models and Models for Spatial, Temporal and Network Data, 15 July 2013, Bellevue, Washington, USA. Association for Uncertainty in Artificial Intelligence. http://purl.flvc.org/fsu/fd/FSU_libsubv1_scholarship_submission_1472579448
Alonso-Fernández, C., Calvo-Morata, A., Freire, M., Martínez-Ortiz, I., & Fernández-Manjón, B. (2019). Applications of data science to game learning analytics data: A systematic literature review. Computers & Education, 141, 103612. https://doi.org/10.1016/j.compedu.2019.103612
Alwahaby, H., & Cukurova, M. (2022). The ethical implications of using multimodal learning analytics: Towards an ethical research and practice framework. EdArXiv preprint. https://doi.org/10.35542/osf.io/4znby
Anderson, L. A., & Paulus, T. M. (2021). Secondary qualitative analysis in the family sciences. Family and Consumer Sciences Research Journal, 49(4), 362–375. https://doi.org/10.1111/fcsr.12403
Ba, S., Hu, X., Stein, D., & Liu, Q. (2023). Assessing cognitive presence in online inquiry-based discussion through text classification and epistemic network analysis. British Journal of Educational Technology, 54(1), 247–266. https://doi.org/10.1111/bjet.13285
Balogh, Z., & Kuchárik, M. (2019). Predicting student grades based on their usage of LMS Moodle using Petri nets. Applied Sciences, 9(20), 4211. https://doi.org/10.3390/app9204211
Bowman, D., Swiecki, Z., Cai, Z., Wang, Y., Eagan, B., Linderoth, J., & Shaffer, D. W. (2021). The mathematical foundations of epistemic network analysis. In Advances in quantitative ethnography (pp. 91–105). Springer International Publishing. https://doi.org/10.1007/978-3-030-67788-6_7
Buckingham Shum, S., Echeverria, V., & Martinez-Maldonado, R. (2019). The multimodal matrix as a quantitative ethnography methodology. In A. Ruis & S. Lee (Eds.), Advances in quantitative ethnography. ICQE 2021. Communications in computer and information science (pp. 26–40, Vol. 1312). Springer International. https://doi.org/10.1007/978-3-030-33232-7_3
Buigut, S. K., & Valev, N. T. (2005). Is the proposed East African Monetary Union an optimal currency area? A structural vector autoregression analysis. World Development, 33(12), 2119–2133. https://doi.org/10.1016/j.worlddev.2005.06.006
Cagiltay, B., Ho, H. -R., Michaelis, J. E., & Mutlu, B. (2020). Investigating family perceptions and design preferences for an in-home robot. In Proceedings of the Interaction Design and Children Conference (IDC 2020), 21–24 June 2020, London, UK (pp. 229–242). ACM. https://doi.org/10.1145/3392063.3394411
Chan, K. I., Tse, R., & Lei, P. I. (2022). Tracing students’ learning performance on multiple skills using Bayesian methods. In Proceedings of the Sixth International Conference on Education and Multimedia Technology (ICEMT 2022), 13–15 July 2022, Guangzhou, China (pp. 84–89). ACM. https://doi.org/10.1145/3551708.3556202
Chan, M. C. E., Ochoa, X., & Clarke, D. (2019). Multimodal learning analytics in a laboratory classroom. In M. Virvou, E. Alepis, G. Tsihrintzis, & L. Jain (Eds.), Machine learning paradigms (pp. 131–156, Vol. 158). Springer International. https://doi.org/10.1007/978-3-030-13743-4_8
Chango, W., Cerezo, R., & Romero, C. (2021). Multi-source and multimodal data fusion for predicting academic performance in blended learning university courses. Computers & Electrical Engineering, 89, 106908. https://doi.org/10.1016/j.compeleceng.2020.106908
Chatfield, S. L. (2020). Recommendations for secondary analysis of qualitative data. The Qualitative Report, 25(3), 833–842. https://doi.org/10.46743/2160-3715/2020.4092
Chen, F., & Lu, C. (2022). Learning outcome modeling in computer-based assessments for learning. In F. Ouyang, P. Jiao, B. M. McLaren, & A. H. Alavi (Eds.), Artificial intelligence in STEM education (pp. 175–194). CRC Press. https://doi.org/10.1201/9781003181187-15
Chen, K. -Z., & Li, S.- C. (2021). Sequential, typological, and academic dynamics of self-regulated learners: Learning analytics of an undergraduate chemistry online course. Computers and Education: Artificial Intelligence, 2, 100024. https://doi.org/10.1016/j.caeai.2021.100024
Chen, L., Feng, G., Leong, C. W., Joe, J., Kitchen, C., & Lee, C. M. (2016). Designing an automated assessment of public speaking skills using multimodal cues. Journal of Learning Analytics, 3(2), 261–281. https://doi.org/10.18608/jla.2016.32.13
Chesler, N. C., Ruis, A. R., Collier, W., Swiecki, Z., Arastoopour, G., & Williamson Shaffer, D. (2015). A novel paradigm for engineering education: Virtual internships with individualized mentoring and assessment of engineering thinking. Journal of Biomechanical Engineering, 137(2), 024701. https://doi.org/10.1115/1.4029235
Chiu, M. M. (2018). Statistically modelling effects of dynamic processes on outcomes: An example of discourse sequences and group solutions. Journal of Learning Analytics, 5(1), 75–91. https://doi.org/10.18608/jla.2018.51.6
Choi, Y., & Cho, Y. I. (2020). Learning analytics using social network analysis and Bayesian network analysis in sustainable computer-based formative assessment system. Sustainability, 12(19), 7950. https://doi.org/10.3390/su12197950
Chouldechova, A., & Roth, A. (2018). The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810. https://doi.org/10.48550/arXiv.1810.08810
Cloude, E. B., Azevedo, R., Winne, P. H., Biswas, G., & Jang, E. E. (2022). System design for using multimodal trace data in modeling self-regulated learning. Frontiers in Education, 7, 928632. https://doi.org/10.3389/feduc.2022.928632
Conijn, R., Van Waes, L., & van Zaanen, M. (2020). Human-centered design of a dashboard on students’ revisions during writing. In C. Alario-Hoyos, M. Rodríguez-Triana, M. Scheffel, I. Arnedillo-Sánchez, & S. Dennerlein (Eds.), Addressing global challenges and quality education. EC-TEL 2020. Lecture notes in computer science (pp. 30–44, Vol. 12315). Springer International. https://doi.org/10.1007/978-3-030-57717-9_3
Csanadi, A., Eagan, B., Kollar, I., Shaffer, D. W., & Fischer, F. (2018). When coding-and-counting is not enough: using epistemic network analysis (ENA) to analyze verbal data in CSCL research. International Journal of Computer-Supported Collaborative Learning, 13(4), 419–438. https://doi.org/10.1007/s11412-018-9292-z
Cukurova, M., Giannakos, M., & Martinez-Maldonado, R. (2020). The promise and challenges of multimodal learning analytics. British Journal of Educational Technology, 51(5), 1441–1449. https://doi.org/10.1111/bjet.13015
Devine, A., Fawcett, K., Sz˝ucs, D., & Dowker, A. (2012). Gender differences in mathematics anxiety and the relation to mathematics performance while controlling for test anxiety. Behavioral and Brain Functions, 8(1). https://doi.org/10.1186/1744-9081-8-33
Di Mitri, D., Schneider, J., Specht, M., & Drachsler, H. (2018). From signals to knowledge: A conceptual model for multimodal learning analytics. Journal of Computer Assisted Learning, 34(4), 338–349. https://doi.org/10.1111/jcal.12288
Dominguez, F., Ochoa, X., Zambrano, D., Camacho, K., & Castells, J. (2021). Scaling and adopting a multimodal learning analytics application in an institution-wide setting. IEEE Transactions on Learning Technologies, 14(3), 400–414. https://doi.org/10.1109/tlt.2021.3100778
Dorodchi, M., Mahzoon, M. J., Maher, M., & Benedict, A. (2020). A learning analytics approach to assessing student risk in active learning. In J. Keith-Le & M. Morgan (Eds.), Faculty experiences in active learning (pp. 86–100). UNC Press. https://omp.charlotte.edu/library/catalog/download/9/13/243?inline=1
Dowell, N. M. M., Nixon, T. M., & Graesser, A. C. (2018). Group communication analysis: A computational linguistics approach for detecting sociocognitive roles in multiparty interactions. Behavior Research Methods, 51(3), 1007–1041. https://doi.org/10.3758/s13428-018-1102-z
Eagan, B., Rogers, B., Pozen, R., Marquart, C., & Shaffer, D. W. (2019). rhoR: Rho for inter rater reliability. R package version, 1(0.0). https://cran.r-project.org/web/packages/rhoR/rhoR.pdf
Eagan, B., Swiecki, Z., Farrell, C., & Shaffer, D. W. (2019). The binary replicate test: Determining the sensitivity of CSCL models to coding error. In K. Lund, G. P. Niccolai, E. Lavoué, C. Hmelo-Silver, G. Gweon, & M. Baker (Eds.), A wide lens: Combining embodied, enactive, extended, and embedded learning in collaborative settings, 13th International Conference on Computer Supported Collaborative Learning (CSCL) 2019 (pp. 328–335, Vol. 1). International Society of the Learning Sciences (ISLS). https://repository.isls.org//handle/1/4421
Emerson, A., Henderson, N., Rowe, J., Min, W., Lee, S., Minogue, J., & Lester, J. (2020). Investigating visitor engagement in interactive science museum exhibits with multimodal Bayesian hierarchical models. In I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, & E. Millán (Eds.), Artificial intelligence in education. AIED 2020. Lecture notes in computer science (pp. 165–176, Vol. 12163). Springer International. https://doi.org/10.1007/978-3-030-52237-7_14
Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). From data mining to knowledge discovery in databases. AI Magazine, 17(3), 37. https://doi.org/10.1609/aimag.v17i3.1230
Fincham, E., Whitelock-Wainwright, A., Kovanovic, V., Joksimovic, S., van Staalduinen, J. - P., & Gaševic, D. (2019). Counting clicks is not enough: Validating a theorized model of engagement in learning analytics. In Proceedings of the Ninth International Conference on Learning Analytics and Knowledge (LAK 2019), 4–8 March 2019, Tempe, Arizona, USA (pp. 501–510). ACM. https://doi.org/10.1145/3303772.3303775
Gardner, J., Brooks, C., & Baker, R. (2019). Evaluating the fairness of predictive student models through slicing analysis. In Proceedings of the Ninth International Conference on Learning Analytics and Knowledge (LAK 2019), 4–8 March 2019, Tempe, Arizona, USA (pp. 225–234). ACM. https://doi.org/10.1145/3303772.3303791
Gee, J. P. (2015). Discourse, small d, big D. In K. Tracy (Ed.), The international encyclopedia of language and social interaction. Wiley. https://doi.org/10.1002/9781118611463.wbielsi016
González-Brambila, S., Sánchez-Guerrero, L., González-Beltrán, B., & Figueroa-González, J. (2021). Using the trajectories analysis for determining computer engineering students’ risks at an superior educational institution. In L. Gomez Chova, A. Lopez, & I. Candel Torres (Eds.), Proceedings of the 14th Annual International Conference of Education, Research and Innovation (ICERI 2021), 8–9 November 2021, online (pp. 4441–4445). IATED. https://doi.org/10.21125/iceri.2021.1023
Gupta, A., Garg, D., & Kumar, P. (2022). Mining sequential learning trajectories with hidden Markov models for early prediction of at-risk students in e-learning environments. IEEE Transactions on Learning Technologies, 15(6), 783–797. https://doi.org/10.1109/tlt.2022.3197486
Huang, L., Zheng, J., Lajoie, S. P., Chen, Y., Hmelo-Silver, C. E., & Wang, M. (2023). Examining university teachers’ self-regulation in using a learning analytics dashboard for online collaboration. Education and Information Technologies, 29(7), 8523–8547. https://doi.org/10.1007/s10639-023-12131-7
Hung, Y. - W., & Higgins, S. (2015). Learners’ use of communication strategies in text-based and video-based synchronous computer-mediated communication environments: Opportunities for language learning. Computer Assisted Language Learning, 29(5), 901–924. https://doi.org/10.1080/09588221.2015.1074589
Hutchins, E. (1995). Cognition in the wild. MIT press. https://doi.org/10.7551/mitpress/1881.001.0001
Jeng, H. -L., & Chung-Nien, C. (2022). Where can we find the differences between experts and novices with lag sequential analysis of spatial behavioral patterns in digital pentomino games. Journal of Research in Education Sciences, 67(4), 105–142. https://doi.org/10.6209/JORIES.202212_67(4).0004
Jewitt, C., Kress, G., Ogborn, J., & Tsatsarelis, C. (2001). Exploring learning through visual, actional and linguistic communication: The multimodal environment of a science classroom. Educational Review, 53(1), 5–18. https://doi.org/10.1080/00131910123753
Jiang, S., Huang, X., Sung, S. H., & Xie, C. (2023). Learning analytics for assessing hands-on laboratory skills in science classrooms using Bayesian network analysis. Research in Science Education, 53(2), 425–444. https://doi.org/10.1007/s11165-022-10061-x
Knight, S., Wise, A. F., & Chen, B. (2017). Time for change: Why learning analytics needs temporal analysis. Journal of Learning Analytics, 4(3), 7–17. https://doi.org/10.18608/jla.2017.43.2
Kokoç, M., Akçapınar, G., & Hasnine, M. N. (2021). Unfolding students’ online assignment submission behavioral patterns using temporal learning analytics. Educational Technology & Society, 24(1), 223–235. https://www.jstor.org/stable/26977869
Lahat, D., Adali, T., & Jutten, C. (2015). Multimodal data fusion: An overview of methods, challenges, and prospects. Proceedings of the IEEE, 103(9), 1449–1477. https://doi.org/10.1109/jproc.2015.2460697
Lämsä, J., Hämäläinen, R., Koskinen, P., Viiri, J., & Mannonen, J. (2020). The potential of temporal analysis: Combining log data and lag sequential analysis to investigate temporal differences between scaffolded and non-scaffolded group inquiry-based learning processes. Computers & Education, 143, 103674. https://doi.org/10.1016/j.compedu.2019.103674
Larmuseau, C., Cornelis, J., Lancieri, L., Desmet, P., & Depaepe, F. (2020). Multimodal learning analytics to investigate cognitive load during online problem solving. British Journal of Educational Technology, 51(5), 1548–1562. https://doi.org/10.1111/bjet.12958
Lee, M. P., Croteau, E., Gurung, A., Botelho, A. F., & Heffernan, N. T. (2023). Knowledge tracing over time: A longitudinal analysis. In M. Feng, T. Käser, & P. Talukdar (Eds.), Proceedings of the 16th International Conference on Educational Data Mining (EDM 2023), 11–14 July 2023, Bengaluru, India. International Educational Data Mining Society. https://doi.org/10.5281/zenodo.8115788
Lemke, J. (1998). Multiplying meaning: Visual and verbal semiotics in scientific text. In J. R. Martin & R. Veel (Eds.), Reading science: Critical and functional perspectives on discourses of science (pp. 87–113). Routledge.
Lewandowski, L. J., Codding, R. S., Kleinmann, A. E., & Tucker, K. L. (2003). Assessment of reading rate in postsecondary students. Journal of Psychoeducational Assessment, 21(2), 134–144. https://doi.org/10.1177/073428290302100202
Liu, F., Fan, Z., Huang, F., Li, Y., He, Y., & Hu, W. (2022). Modeling learner behavior analysis based on educational big data and dynamic Bayesian network. In 2022 IEEE Eighth International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing (HPSC) and IEEE International Conference on Intelligent Data and Security (IDS), 6–8 May 2022, Jinan, China (pp. 48–53). IEEE. https://doi.org/10.1109/bigdatasecurityhpscids54978.2022.00019
Liu, R., Stamper, J., Davenport, J., Crossley, S., McNamara, D., Nzinga, K., & Sherin, B. (2019). Learning linkages: Integrating data streams of multiple modalities and timescales. Journal of Computer Assisted Learning, 35(1), 99–109. https://doi.org/10.1111/jcal.12315
Liu, X. (2022). Primary science curriculum student acceptance of blended learning: Structural equation modeling and visual analytics. Journal of Computers in Education, 9(3), 351–377. https://doi.org/10.1007/s40692-021-00206-8
Ma, Y., Celepkolu, M., & Boyer, K. E. (2022). Detecting impasse during collaborative problem solving with multimodal learning analytics. In Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK 2022), 21–25 March 2022, online (pp. 45–55). ACM. https://doi.org/10.1145/3506860.3506865
Marquart, C., Swiecki, Z., Eagan, B., & Shaffer, D. W. (2019). ncodeR: Techniques for automated classifiers. University of Wisconsin–Madison. https://cran.r-project.org/web/packages/ncodeR/index.html
Martinez-Maldonado, R., Echeverria, V., Fernandez Nieto, G., & Buckingham Shum, S. (2020). From data to insights: A layered storytelling approach for multimodal learning analytics. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI 2020), 25–30 April 2020, Honolulu, Hawaii, USA (pp. 1–15). ACM. https://doi.org/10.1145/3313831.3376148
Mayfield, E., Madaio, M., Prabhumoye, S., Gerritsen, D., McLaughlin, B., Dixon-Román, E., & Black, A. W. (2019). Equity beyond bias in language technologies for education. In H. Yannakoudakis, E. Kochmar, C. Leacock, N. Madnani, I. Pilán, & T. Zesch (Eds.), Proceedings of the 14th Workshop on Innovative Use of NLP for Building Educational Applications, 2 August 2019, Florence, Italy (pp. 444–460). Association for Computational Linguistics. https://doi.org/10.18653/v1/w19-4446
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6), 1–35. https://doi.org/10.1145/3457607
Michaelis, J. E., & Mutlu, B. (2018). Reading socially: Transforming the in-home reading experience with a learning companion robot. Science Robotics, 3(21). https://doi.org/10.1126/scirobotics.aat5999
Michaelis, J. E., & Mutlu, B. (2019). Supporting interest in science learning with a social robot. In Proceedings of the 18th ACM International Conference on Interaction Design and Children (IDC 2019), 12–15 June 2019, Boise, Idaho, USA (pp. 71–82). ACM. https://doi.org/10.1145/3311927.3323154
Nourbakhsh, N., Wang, Y., Chen, F., & Calvo, R. A. (2012). Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks. In V. Farrell, G. Farrell, C. Chua, W. Huang, R. Vasa, & C. Woodward (Eds.), Proceedings of the 24th Australian Computer-Human Interaction Conference (OzCHI 2012), 26–30 November 2012, Melbourne, Australia (pp. 420–423). ACM. https://doi.org/10.1145/2414536.2414602
Ochoa, X. (2022). Multimodal learning analytics—Rationale, process, examples, and direction. In C. Lang, G. Siemens, A. F. Wise, D. Gaševic, & A. Merceron (Eds.), The handbook of learning analytics (pp. 54–65). SoLAR. https://doi.org/10.18608/hla22.006
Ochoa, X., Chiluiza, K., Granda, R., Falcones, G., Castells, J., & Guamán, B. (2018). Multimodal transcript of face-to-face group-work activity around interactive tabletops. In Companion Proceedings of the Eighth International Conference on Learning Analytics and Knowledge (CrossMMLA@ LAK 2018), 7–9 March 2018, Sydney, New South Wales, Australia. ACM. https://ceur-ws.org/Vol-2163/paper4.pdf
Ochoa, X., Chiluiza, K., Méndez, G., Luzardo, G., Guamán, B., & Castells, J. (2013). Expertise estimation based on simple multimodal features. In Proceedings of the 15th ACM on International Conference on Multimodal Interaction (ICMI 2013), 9–13 December 2013, Sydney, Australia (pp. 583–590). ACM. https://doi.org/10.1145/2522848.2533789
Ochoa, X., & Worsley, M. (2016). Augmenting learning analytics with multimodal sensory data. Journal of Learning Analytics, 3(2), 213–219. https://doi.org/10.18608/jla.2016.32.10
Poitras, E. G., Behnagh, R. F., & Bouchet, F. (2020). A dimensionality reduction method for time series analysis of student behavior to predict dropout in massive open online courses. In D. Ifenthaler & D. Gibson (Eds.), Adoption of data analytics in higher education learning and teaching (pp. 391–406). Springer. https://doi.org/10.1007/978-3-030-47392-1_20
Porter, C., Donegan, S., Eagan, B., Geröly, A., Jeney, A., Jiao, S., Peters, G.-J., & Zörg˝o, S. (2021). A systematic review of quantitative ethnography methods. In B. Wasson & S. Zörg˝o (Eds.), Second International Conference on Quantitative Ethnography: Conference Proceedings Supplement, 6–11 November 2021, online (pp. 35–38). International Society for Quantitative Ethnography. https://doi.org/10.31235/osf.io/erbt5
Priestley, M. (1980). State-dependent models: A general approach to non-linear time series analysis. Journal of Time Series Analysis, 1(1), 47–71. https://doi.org/10.1111/j.1467-9892.1980.tb00300.x
Raca, M., & Dillenbourg, P. (2013). System for assessing classroom attention. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (LAK 2013), 8–13 April 2013, Leuven, Belgium (pp. 265–269). ACM. https://doi.org/10.1145/2460296.2460351
Rahmani, D., & Fay, D. (2022). A state-dependent linear recurrent formula with application to time series with structural breaks. Journal of Forecasting, 41(1), 43–63. https://doi.org/10.1002/for.2778
Reilly, J. M., & Dede, C. (2019). Differences in student trajectories via filtered time series analysis in an immersive virtual world. In Proceedings of the Ninth International Conference on Learning Analytics and Knowledge (LAK 2019), 4–8 March 2019, Tempe, Arizona, USA (pp. 130–134). ACM. https://doi.org/10.1145/3303772.3303832
Reimann, P. (2009). Time is precious: Variable- and event-centred approaches to process analysis in CSCL research. International Journal of Computer-Supported Collaborative Learning, 4, 239–257. https://doi.org/10.1007/s11412-009-9070-z
Reimann, P., Markauskaite, L., & Bannert, M. (2014). E-research and learning theory: What do sequence and process mining methods contribute? British Journal of Educational Technology, 45(3), 528–540. https://doi.org/10.1111/bjet.12146
Reimann, P., Yacef, K., & Kay, J. (2011). Analyzing collaborative interactions with data mining methods for the benefit of learning. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL: Methods, approaches and issues. Computer-supported collaborative learning series (pp. 161–185, Vol. 12). Springer. https://doi.org/10.1007/978-1-4419-7710-6_8
Rivers, K., Harpstead, E., & Koedinger, K. R. (2016). Learning curve analysis for programming: Which concepts do students struggle with? In Proceedings of the 2016 ACM Conference on International Computing Education Research (ICER 2016), 8–12 September 2016, Melbourne, Australia (pp. 143–151, Vol. 16). ACM. https://doi.org/10.1145/2960310.2960333
Rubin, D. C., & Wenzel, A. E. (1996). One hundred years of forgetting: A quantitative description of retention. Psychological Review, 103(4), 734. https://doi.org/10.1037//0033-295x.103.4.734
Ruis, A., Siebert-Evenstone, A., Pozen, R., Eagan, B., & Shaffer, D. W. (2019). Finding common ground: A method for measuring recent temporal context in analyses of complex, collaborative thinking. In K. Lund, G. P. Niccolai, E. Lavoué, C. Hmelo-Silver, G. Gweon, & M. Baker (Eds.), A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning in Collaborative Settings, 13th International Conference on Computer Supported Collaborative Learning (CSCL 2019), 17–21 June 2019, Lyon, France (Vol. 1). International Society of the Learning Sciences. https://repository.isls.org//handle/1/4395
Runyan, M. K. (1991). The effect of extra time on reading comprehension scores for university students with and without learning disabilities. Journal of Learning Disabilities, 24(2), 104–108. https://doi.org/10.1177/002221949102400207
Schneider, B., & Blikstein, P. (2015). Unraveling students’ interaction around a tangible interface using multimodal learning analytics. Journal of Educational Data Mining, 7(3), 89–116. https://doi.org/10.5281/zenodo.3554729
Shaffer, D. W. (2017). Quantitative ethnography. Lulu.com. https://books.google.com/books/about/Quantitative_Ethnography.html?id=H-iMDwAAQBAJ
Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45. https://doi.org/10.18608/jla.2016.33.3
Shaffer, D. W., & Ruis, A. (2017). Epistemic network analysis: A worked example of theory-based learning analytics. In C. Lang, G. Siemens, A. Wise, & D. Gaševi´c (Eds.), Handbook of learning analytics. SoLAR. https://doi.org/10.18608/hla17.015
Shaffer, D. W., & Ruis, A. R. (2021). How we code. In A. Ruis & S. Lee (Eds.), Advances in quantitative ethnography. ICQE 2021. Communications in computer and information science (pp. 62–77, Vol. 1312). Springer. https://doi.org/10.1007/978-3-030-67788-6_5
Sharma, K., & Giannakos, M. (2020). Multimodal data capabilities for learning: What can multimodal data tell us about learning? British Journal of Educational Technology, 51(5), 1450–1484. https://doi.org/10.1111/bjet.12993
Sharma, K., Papamitsiou, Z., & Giannakos, M. N. (2019). Modelling learners’ behaviour: A novel approach using GARCH with multimodal data. In M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, & J. Schneider (Eds.), Transforming learning with meaningful technologies. EC-TEL 2019. Lecture notes in computer science (pp. 450–465, Vol. 11722). Springer. https://doi.org/10.1007/978-3-030-29736-7_34
Siebert-Evenstone, A. L., Irgens, G. A., Collier, W., Swiecki, Z., Ruis, A. R., & Shaffer, D. W. (2017). In search of conversational grain size: Modeling semantic structure using moving stanza windows. Journal of Learning Analytics, 4(3), 123–139. https://doi.org/10.18608/jla.2017.43.7
Spätgens, T., & Schoonen, R. (2019). Individual differences in reading comprehension in monolingual and bilingual children: The influence of semantic priming during sentence reading. Learning and Individual Differences, 76, 101777. https://doi.org/10.1016/j.lindif.2019.101777
Sümer, Ö., Goldberg, P., D’Mello, S., Gerjets, P., Trautwein, U., & Kasneci, E. (2021). Multimodal engagement analysis from facial videos in the classroom. IEEE Transactions on Affective Computing, 14(2), 1012–1027. https://doi.org/10.1109/taffc.2021.3127692
Sung, H., Cao, S., Ruis, A. R., & Shaffer, D. W. (2019). Reading for breadth, reading for depth: Understanding the relationship between reading and complex thinking using epistemic network analysis. In K. Lund, G. P. Niccolai, E. Lavoué, C. Hmelo-Silver, G. Gweon, & M. Baker (Eds.), A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning in Collaborative Settings, 13th International Conference on Computer Supported Collaborative Learning (CSCL 2019), 17–21 June 2019, Lyon, France. International Society of the Learning Sciences. https://repository.isls.org//handle/1/4428
Sung, H., Swart, M. I., & Nathan, M. J. (2022). Methods for analyzing temporally entangled multimodal data. In A. Weinberger, W. Chen, D. Hernández-Leo, & B. Chen (Eds.), Proceedings of the 15th International Conference on Computer-Supported Collaborative Learning (CSCL 2022), 6–10 June 2022, Hiroshima, Japan, and online (pp. 242–249). International Society of the Learning Sciences. https://repository.isls.org//handle/1/8282
Swiecki, Z., Lian, Z., Ruis, A., & Shaffer, D. W. (2019). Does order matter? Investigating sequential and cotemporal models of collaboration. In K. Lund, G. P. Niccolai, E. Lavoué, C. Hmelo-Silver, G. Gweon, & M. Baker (Eds.), A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning in Collaborative Settings, 13th International Conference on Computer Supported Collaborative Learning (CSCL 2019), 17–21 June 2019, Lyon, France (Vol. 1). International Society of the Learning Sciences. https://repository.isls.org//handle/1/1556
Tang, H., Dai, M., Yang, S., Du, X., Hung, J.- L., & Li, H. (2022). Using multimodal analytics to systemically investigate online collaborative problem-solving. Distance Education, 43(2), 290–317. https://doi.org/10.1080/01587919.2022.2064824
Tang, K.- s., Delgado, C., & Moje, E. B. (2014). An integrative framework for the analysis of multiple and multimodal representations for meaning-making in science education. Science Education, 98(2), 305–326. https://doi.org/10.1002/sce.21099
Teasley, S. D., Popov, V., Bae, J. - S., & Elkins, S. (2023). Self-regulated learning theory and epistemic network analysis: Understanding university students’ use of a learning analytics dashboard. In T. Urdan & E. N. Gonida (Eds.), Remembering the life, work, and influence of Stuart A. Karabenick: A legacy of research on self-regulation, help seeking, teacher motivation, and more (pp. 215–240). Emerald Publishing Limited. https://doi.org/10.1108/s0749-742320230000022015
Thiyagarajan, G., & Prasanna, S. (2022). Process mining-based behavioral modeling of learners in self-paced learning environment. In K. Ray, A. Dixit, D. Adhikari, & R. Mathew (Eds.), Proceedings of the 2nd international conference on signal and data processing. CSDP 2022. Lecture notes in electrical engineering (pp. 121–132, Vol. 1026). Springer. https://doi.org/10.1007/978-981-99-1410-4_11
Thorne, S. (2013). Secondary qualitative data analysis. In C. Tatano Beck (Ed.), Routledge international handbook of qualitative nursing research (pp. 393–404). Routledge. https://doi.org/10.4324/9780203409527
Wang, Y., Swiecki, Z., Ruis, A. R., & Shaffer, D. W. (2021). Simplification of epistemic networks using parsimonious removal with interpretive alignment. In A. Ruis & S. Lee (Eds.), Advances in quantitative ethnography. ICQE 2021. Communications in computer and information science (pp. 137–151, Vol. 1312). Springer. https://doi.org/10.1007/978-3-030-67788-6_10
Wise, A. F., & Shaffer, D. W. (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13. https://doi.org/10.18608/jla.2015.22.2
Wixted, J. T., & Ebbesen, E. B. (1997). Genuine power curves in forgetting: A quantitative analysis of individual subject forgetting functions. Memory & Cognition, 25, 731–739. https://doi.org/10.3758/bf03211316
Wong, T.- L., Zou, D., Cheng, G., Tang, J. K. T., Cai, Y., & Wang, F. L. (2021). Enhancing skill prediction through generalising Bayesian knowledge tracing. International Journal of Mobile Learning and Organisation, 15(4), 358–373. https://doi.org/10.1504/ijmlo.2021.10040632
Worsley, M. (2014). Multimodal learning analytics as a tool for bridging learning theory and complex learning behaviors. In Proceedings of the 2014 ACM Workshop on Multimodal Learning Analytics Workshop and Grand Challenge (MLA 2014), 12 November 2014, Istanbul, Türkiye (pp. 1–4). ACM. https://doi.org/10.1145/2666633.2666634
Worsley, M. (2022). Framing the future of multimodal learning analytics. In M. Giannakos, D. Spikol, D. Di Mitri, K. Sharma, X. Ochoa, & R. Hammad (Eds.), The multimodal learning analytics handbook (pp. 359–369). Springer. https://doi.org/10.1007/978-3-031-08076-0_14
Worsley, M., & Blikstein, P. (2011). What’s an expert? Using learning analytics to identify emergent markers of expertise through automated speech, sentiment and sketch analysis. In M. Pechenizkiy, T. Calders, C. Conati, S. Ventura, C. Romero, & J. Stamper (Eds.), Proceedings of the Fourth International Conference on Educational Data Mining (EDM 2011), 6–8 July 2011, Eindhoven, Netherlands (pp. 235–240). International Educational Data Mining Society. https://educationaldatamining.org/EDM2011/wp-content/uploads/proc/edm2011_paper18_short_Worsley.pdf
Worsley, M., & Blikstein, P. (2013). Towards the development of multimodal action based assessment. In Proceedings of the Third International Conference on Learning Analytics and Knowledge (LAK 2013), 8–13 April 2013, Leuven, Belgium (pp. 94–101). ACM. https://doi.org/10.1145/2460296.2460315
Worsley, M., & Blikstein, P. (2014). Deciphering the practices and affordances of different reasoning strategies through multimodal learning analytics. In Proceedings of the 2014 ACM Workshop on Multimodal Learning Analytics Workshop and Grand Challenge (MLA 2014), 12 November 2014, Istanbul, Türkiye (pp. 21–27). ACM. https://doi.org/10.1145/2666633.2666637
Yan, L., Zhao, L., Gasevic, D., & Martinez-Maldonado, R. (2022). Scalability, sustainability, and ethicality of multimodal learning analytics. In Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK 2022), 21–25 March 2022, online (pp. 13–23). ACM. https://doi.org/10.1145/3506860.3506862
Yusuf, A., Noor, N. M., & Bello, S. (2023). Using multimodal learning analytics to model students’ learning behavior in animated programming classroom. Education and Information Technologies, 1–44. https://doi.org/10.1007/s10639-023-12079-8
Zhang, S., Gao, Q., Sun, M., Cai, Z., Li, H., Tang, Y., & Liu, Q. (2022). Understanding student teachers’ collaborative problem solving: Insights from an epistemic network analysis (ENA). Computers & Education, 183, 104485. https://doi.org/10.1016/j.compedu.2022.104485
Zhou, G., Moulder, R. G., Sun, C., & D’Mello, S. K. (2022). Investigating temporal dynamics underlying successful collaborative problem solving behaviors with multilevel vector autoregression. In Proceedings of the 15th International Conference on Educational Data Mining (EDM 2022), 24–27 July 2022, Durham, UK. International Educational Data Mining Society. https://educationaldatamining.org/edm2022/proceedings/2022.EDM-long-papers.25/index.html
Zhou, Y., & Kang, J. (2023). Enriching multimodal data: A temporal approach to contextualize joint attention in collaborative problem-solving. Journal of Learning Analytics, 10(3), 87–101. https://doi.org/10.18608/jla.2023.7989
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution 4.0 International License.
