Combining Checkpoint and Process Learning Analytics to Support Learning Design Decisions in Blended Learning Environments




Learning analytics, Learning design, Blended learning, Teachers, Canvas, Social network analysis, Text network analysis


Learning analytics (LA) constitutes a key opportunity to support learning design (LD) in blended learning environments. However, details as to how LA supports LD in practice and information on teacher experiences with LA are limited. This study explores the potential of LA to inform LD based on a one-semester undergraduate blended learning course at a Norwegian university. Our findings indicate that creating valuable connections between LA and LD requires a detailed analysis of student checkpoints (e.g., online logins) and process analytics (e.g., online content and interaction dynamics) to find meaningful learning behaviour patterns that can be forwarded to teachers in retrospect to support the redesign of courses. Moreover, the teachers in our study found the LA visualizations to be valuable for understanding student online learning processes, but they also requested the timely sharing of aggregated LA visualizations in a simple, easy-to-interpret format, yet detailed enough to be informative and actionable. We conclude the paper by arguing that the potential of LA to support LD is improved when multiple levels of LA are considered.

Author Biographies

Rogers Kaliisa, University of Oslo

Doctoral Research Fellow, The University of Oslo, Norway

Anders I. Mørch, University of Oslo

Professor, Department of Education, University of Oslo, Norway


Agostinho, S. (2011). The use of a visual learning design representation to support the design process of teaching in higher education. Australasian Journal of Educational Technology, 27(6). https://

Alhadad, S. S., & Thompson, K. (2017). Understanding the mediating role of teacher inquiry when connecting learning analytics with design for learning. Interaction Design and Architecture(s) Journal, 33, 54–74. Retrieved from

Andersen, R., & Mørch, A. I. (2016). Mutual development in mass collaboration: Identifying interaction patterns in customer-initiated software product development. Computers in Human Behavior, 65, 77–91.

Bakharia, A., Corrin, L., de Barba, P., Kennedy, G., Gašević, D., Mulder, R., Lockyer, L. (2016). A conceptual framework linking learning design with learning analytics. Proceedings of the 6th International Conference on Learning Analytics and Knowledge (LAK ʼ16), 25–29 April 2016, Edinburgh, UK (pp. 329–338). New York, NY: ACM.

Bennett, S., Agostinho, S., & Lockyer, L. (2015). Technology tools to support learning design: Implications derived from an investigation of university teachers’ design practices. Computers & Education, 81, 211–220.

Benesty, J., Chen, J., Huang, Y., & Cohen, I. (2009). Pearson correlation coefficient. In I. Cohen, Y. Huang, J. Chen, & J. Benesty (Eds.), Noise reduction in speech processing (pp. 1–4). Berlin, Germany: Springer.

Buckingham Shum, S., Ferguson, R., & Martinez-Maldonado, R. (2019). Human-centred learning analytics. Journal of Learning Analytics, 6(2), 1–9.

Bonk, C. J., & Graham, C. R. (2012). The handbook of blended learning: Global perspectives, local designs. Hoboken, NJ: John Wiley & Sons. Retrieved from

Conole, G. (2012). Designing for learning in an open world (Vol. 4). Leicester, UK: Springer Science & Business Media. Retrieved from

Dobozy, E., & Cameron, L. (2018). Special issue on learning design research: Mapping the terrain. Australasian Journal of Educational Technology, 34(2).

Eradze, M., Rodriguez Triana, M. J., & Laanpere, M. (2017). Semantically annotated lesson observation data in learning analytics datasets: A reference model. Interaction Design and Architecture(s) Journal, 33, 75–91. Retrieved from

Fritz, J. (2016). LMS course design as learning analytics variable. In J. Greer, M. Molinaro, X. Ochoa, & T. McKay (Eds.), Proceedings of the 1st Learning Analytics for Curriculum and Program Quality Improvement Workshop (PCLA 2016), 25 April 2016, Edinburgh, UK (pp. 15–19). Retrieved from

Gašević, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84.

Griffiths, D. (2017). The use of models in learning design and learning analytics. Interaction Design and Architecture(s) Journal, 33, 113–133. Retrieved from

Gunn, C., McDonald, J., Donald, C., Nichols, M., Milne, J., & Blumenstein, M. (2017). Building an evidence base for teaching and learning design using learning analytics. Wellington, Australia: Ako Aotearoa – The National Centre for Tertiary Teaching Excellence. Retrieved from

Haya, P. A., Daems, O., Malzahn, N., Castellanos, J., & Hoppe, H. U. (2015). Analysing content and patterns of interaction for improving the learning design of networked learning environments. British Journal of Educational Technology, 46(2), 300–316.

Hernández‐Leo, D., Martinez‐Maldonado, R., Pardo, A., Muñoz‐Cristóbal, J. A., & Rodríguez‐Triana, M. J. (2019). Analytics for learning design: A layered framework and tools. British Journal of Educational Technology, 50(1), 139–152.

Harel, D., & Koren, Y. (2001). A fast multi-scale method for drawing large graphs. In J. Marks (Ed.) Graph drawing. GD 2000. Notes in computer science (Vol. 1984). Berlin, Germany: Springer.

Ifenthaler, D., Gibson, D., & Dobozy, E. (2018). Informing learning design through analytics: Applying network graph analysis. Australasian Journal of Educational Technology, 34(2), 117–132.

Kaliisa, R., Mørch, A. I., & Kluge, A. (2019). Exploring social learning analytics to support teaching and learning decisions in online learning environments. In M. Scheffel, J. Broisin, V. Pammer-Schindler, A. Ioannou, & J. Schneider (Eds.), Transforming learning with meaningful technologies. Proceedings of the 14th European Conference on Technology Enhanced Learning (EC-TEL 2019), 16–19 September 2019, Delft, The Netherlands. Lecture Notes in Computer Science (Vol. 11722) (pp. 187–198). Cham, Switzerland: Springer.

Kaliisa, R., Kluge, A., & Mørch, A. I. (in press a). Overcoming the challenges to learning analytics adoption at the practitioner level: A critical analysis of 18 learning analytics frameworks. Scandinavian Journal of Educational Research.

Kaliisa, R., Kluge, A., & Mørch, A. I.. (in press b). “My point of departure for analytics is extreme skepticism”: Implications derived from an investigation of university teachers’ learning analytics perspectives and design practices. Technology, Knowledge & Learning.

Knight, S., Shum, S. B., & Littleton, K. (2014). Epistemology, assessment, pedagogy: Where learning meets analytics in the middle space. Journal of Learning Analytics, 1(2), 23–47.

Law, N., Li, L., Herrera, L. F., Chan, A., & Pong, T.-C. (2017). A pattern language based learning design studio for an analytics informed inter-professional design community. Interaction Design and Architecture(s) Journal, 33, 92–112. Retrieved from

Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439–1459.

Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system” for educators: A proof of concept. Computers & Education, 54(2), 588–599.

Macfadyen, L. P., & Dawson, S. (2012). Numbers are not enough: Why e-learning analytics failed to inform an institutional strategic plan. Journal of Educational Technology & Society, 15(3), 149–163. Retrieved from

Mangaroska, K., & Giannakos, M. N. (2018). Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies, 12(4), 516–534.

Martinez-Maldonado, R., Kay, J., Yacef, K., & Schwendimann, B. (2012). An interactive teacher’s dashboard for monitoring groups in a multi-tabletop learning environment. In S. A. Cerri, W. J. Clancey, G. Papadourakis, & K. Panourgia (Eds.), Proceedings of the 11th International Conference on Intelligent Tutoring Systems (ITS 2012), 14–18 June 2012, Chania, Greece,. Lecture notes in computer science (Vol. 7315) (pp. 482–492). Berlin, Germany: Springer.

McKenney, S., & Mor, Y. (2015). Supporting teachers in data‐informed educational design. British Journal of Educational Technology, 46(2), 265–279.

Michos, K., Hernández‐Leo, D., & Albó, L. (2018). Teacher‐led inquiry in technology‐supported school communities. British Journal of Educational Technology, 49(6), 1077–1095.

Mor, Y., Ferguson, R., & Wasson, B. (2015). Learning design, teacher inquiry into student learning and learning analytics: A call for action. British Journal of Educational Technology, 46(2), 221–229.

Melero, J., Hernández‐Leo, D., Sun, J., Santos, P., & Blat, J. (2015). How was the activity? A visualization support for a case of location‐based learning design. British Journal of Educational Technology, 46(2), 317–329.

Nguyen, Q., Huptych, M., & Rienties, B. (2018). Linking students’ timing of engagement to learning design and academic performance. Proceedings of the 8th International Conference on Learning Analytics and Knowledge (LAK ’18), 5–9 March 2018, Sydney, NSW, Australia (pp. 141–150). New York, NY: ACM.

Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., & Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior, 76, 703–714.

Paranyushkin, D. (2019). InfraNodus: Generating insight using text network analysis. Proceedings of the 28th International Conference on World Wide Web (WWW ’19), 13–17 May 2019, San Francisco, CA, USA (pp. 3584–3589). New York, NY: ACM.

Persico, D., & Pozzi, F. (2015). Informing learning design with learning analytics to improve teacher inquiry. British Journal of Educational Technology, 46(2), 230–248.

Rienties, B., Cross, S., & Zdrahal, Z. (2017). Implementing a learning analytics intervention and evaluation framework: What works? In D. Kei Daniel (Ed.) Big data and learning analytics in higher education (pp. 147–166). Cham, Switzerland: Springer.

Rienties, B., & Toetenel, L. (2016). The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules. Computers in Human Behavior, 60, 333–341.

Rodríguez‐Triana, M. J., Martínez‐Monés, A., Asensio‐Pérez, J. I., & Dimitriadis, Y. (2015). Scripting and monitoring meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations. British Journal of Educational Technology, 46(2), 330–343.

Samuelsen, J., Chen, W., & Wasson, B. (2019). Integrating multiple data sources for learning analytics: Review of literature. Research and Practice in Technology Enhanced Learning, 14(11), 1-20.

Shibani, A., Knight, S., & Shum, S. B. (2019). Contextualizable learning analytics design: A generic model and writing analytics evaluations. Proceedings of the 9th International Conference on Learning Analytics and Knowledge (LAK ’19), 4–8 March 2019, Tempe, AZ, USA (pp. 210–219). New York, NY: ACM.

Smith, M. A., Shneiderman, B., Milic-Frayling, N., Mendes Rodrigues, E., Barash, V., Dunne, C., & Gleave, E. (2009). Analyzing (social media) networks with NodeXL. Proceedings of the 4th International Conference on Communities and Technologies (C&T ’09), 25–27 June 2009, University Park, PA, USA (pp. 255–264). New York, NY: ACM.

Wise, A. F., & Jung, Y. (2019). Teaching with analytics: Towards a situated model of instructional decision-making. Journal of Learning Analytics, 6(2), 53–69.

Wise, A. F., & Shaffer, D. W. (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13.




How to Cite

Kaliisa, R., Kluge, A., & Mørch, A. I. (2020). Combining Checkpoint and Process Learning Analytics to Support Learning Design Decisions in Blended Learning Environments. Journal of Learning Analytics, 7(3), 33-47.



Special Section: Learning Design and Learning Analytics