Investigating the Effect of Visualization Literacy and Guidance on Teachers' Dashboard Interpretation
DOI:
https://doi.org/10.18608/jla.2024.8471Keywords:
learning analytics, dashboards, human-centred design, data literacy, visualization guidance, data storytelling, research paperAbstract
Recent research on learning analytics dashboards has focused on designing user interfaces that offer various forms of visualization guidance (often referring to notions such as data storytelling or narrative visualization) to teachers (e.g., emphasizing data points or trends with colour and adding annotations), aiding them in interpreting visual elements to gain a comprehensive understanding of students’ learning processes. Yet, while some studies have explored how teachers interpret students’ data through these dashboards, many have overlooked the diverse technical capabilities of teachers, which can significantly impact their use of LA dashboards. In particular, visualization literacy (VL) skills can greatly influence how effectively teachers interpret dashboards. To the best of our knowledge, no comprehensive account exists that details how teachers with varying VL skills interpret visual representations of students’ data. In this paper, we address this gap by investigating how teachers interpret LA dashboards, both with and without visualization guidance, taking into account their VL. We illustrate this by analyzing teachers’ think-aloud sessions as they engage with dashboards in the context of monitoring synchronous online learning tasks undertaken by student groups using Zoom and Google Docs. Using epistemic network analysis, we examine the differences in interpretations between teachers with varying VL levels. Our findings revealed that teachers with low VL exhibited shallower dashboard interpretations than those with high VL. However, the association of VL with successful task completion rate was not significant. Also, visualization guidance did not enable teachers to deepen their interpretations. While some visualization guidance helped teachers to complete tasks correctly, excessive visualization guidance can also be detrimental.
References
Ahn, J., Campos, F., Hays, M., & DiGiacombo, D. (2019). Designing in context: Reaching beyond usability in learning analytics dashboard design. Journal of Learning Analytics, 6(2), 70–85. https://doi.org/10.18608/jla.2019.62.5
Alfredo, R., Echeverria, V., Jin, Y., Yan, L., Swiecki, Z., Gaševi´c, D., & Martinez-Maldonado, R. (2024). Human-centred learning analytics and AI in education: A systematic literature review. Computers and Education: Artificial Intelligence, 6, 100215. https://doi.org/10.1016/j.caeai.2024.100215
Auerbach, A. J., Higgins, M., Brickman, P., & Andrews, T. C. (2018). Teacher knowledge for active-learning instruction: Expert–novice comparison reveals differences. CBE—Life Sciences Education, 17. https://doi.org/10.1187/cbe.17-07-0149
Bao, H., Li, Y., Su, Y., Xing, S., Chen, N. - S., & Rosé, C. P. (2021). The effects of a learning analytics dashboard on teachers’ diagnosis and intervention in computer-supported collaborative learning. Technology, Pedagogy and Education, 30(2), 287–303. https://doi.org/10.1080/1475939x.2021.1902383
Barral, O., Lalle, S., Iranpour, A., & Conati, C. (2021). Effect of adaptive guidance and visualization literacy on gaze attentive behaviors and sequential patterns on magazine-style narrative visualizations. ACM Transactions on Interactive Intelligent Systems, 11(3-4), 1–46. https://doi.org/10.1145/3447992
Barral, O., Lallé, S., Guz, G., Iranpour, A., & Conati, C. (2020). Eye-tracking to predict user cognitive abilities and performance for user-adaptive narrative visualizations. In Proceedings of the 2020 International Conference on Multimodal Interaction (ICMI 2020), 25–29 October 2020, online (pp. 163–173). ACM. https://doi.org/10.1145/3382507.3418884
Bartindale, T., Chen, P., Marshall, H., Pozdniakov, S., & Richardson, D. (2021). ZoomSense: A scalable infrastructure for augmenting Zoom. In Proceedings of the 29th ACM International Conference on Multimedia (MM 2021), 20–24 October 2021, online (pp. 3771–3774). ACM. https://doi.org/10.1145/3474085.3478332
Börner, K., Bueckle, A., & Ginda, M. (2019). Data visualization literacy: Definitions, conceptual frameworks, exercises, and assessments. Proceedings of the National Academy of Sciences, 116(6), 1857–1864. https://doi.org/10.1073/pnas.1807180116
Bowles, M. A. (2018). Introspective verbal reports: Think-alouds and stimulated recall. In A. Phakiti, P. De Costa, L. Plonsky, & S. Starfield (Eds.), The Palgrave handbook of applied linguistics research methodology (pp. 339–357). Palgrave Macmillan. https://doi.org/10.1057/978-1-137-59900-1_16
Bressler, D. M., Annetta, L. A., Dunekack, A., Lamb, R. L., & Vallett, D. B. (2022). How STEM game design participants discuss their project goals and their success differently. In B. Wasson & S. Zörg˝o (Eds.), Advances in quantitative ethnography. ICQE 2021. Communications in computer and information science (pp. 176–190, Vol. 1522). Springer International. https://doi.org/10.1007/978-3-030-93859-8_12
Buckingham Shum, S., Ferguson, R., & Martinez-Maldonado, R. (2019). Human-centred learning analytics. Journal of Learning Analytics, 6(2), 1–9. https://doi.org/10.18608/jla.2019.62.1
Burns, A., Xiong, C., Franconeri, S., Cairo, A., & Mahyar, N. (2020). How to evaluate data visualizations across different levels of understanding. In A. Bezerianos, K. Hall, S. Huron, M. Kay, M. Meyer, & M. Sedlmair (Eds.), 2020 IEEE Workshop on Evaluation and Beyond—Methodological Approaches to Visualization (BELIV), 25–30 October 2020, Salt Lake City, Utah, USA (pp. 19–28). IEEE. https://doi.org/10.1109/beliv51497.2020.00010
Campbell, J. L., Quincy, C., Osserman, J., & Pedersen, O. K. (2013). Coding in-depth semistructured interviews: Problems of unitization and intercoder reliability and agreement. Sociological Methods & Research, 42(3). https://doi.org/10.1177/0049124113500475
Campos, F., Ahn, J., DiGiacomo, D. K., Nguyen, H., & Hays, M. (2021). Making sense of sensemaking: Understanding how K–12 teachers and coaches react to visual analytics. Journal of Learning Analytics, 8(3), 1–21. https://doi.org/10.18608/jla.2021.7113
Ceneda, D., Gschwandtner, T., May, T., Miksch, S., Schulz, H. -J., Streit, M., & Tominski, C. (2017). Characterizing guidance in visual analytics. IEEE Transactions on Visualization and Computer Graphics, 23(1), 111–120. https://doi.org/10.1109/TVCG.2016.2598468
Chavan, P., & Mitra, R. (2022). Tcherly: A teacher-facing dashboard for online video lectures. Journal of Learning Analytics, 9(3), 125–151. https://doi.org/10.18608/jla.2022.7555
Chen, Q., Li, Z., Pong, T.- C., & Qu, H. (2019, April). Designing narrative slideshows for learning analytics. In Proceedings of the 2019 IEEE Pacific Visualization Symposium (PacificVis 2019), 23–26 April 2019, Bangkok, Thailand (pp. 237– 246). IEEE. https://doi.org/10.1109/pacificvis.2019.00036
Choudhry, A., Sharma, M., Chundury, P., Kapler, T., Gray, D. W. S., Ramakrishnan, N., & Elmqvist, N. (2021). Once upon a time in visualization: Understanding the use of textual narratives for causality. IEEE Transactions on Visualization and Computer Graphics, 27(2), 1332–1342. https://doi.org/10.1109/tvcg.2020.3030358
Deakin Crick, R. E., Knight, S., & Barr, S. (2017). Towards analytics for wholistic school improvement: Hierarchical process modelling and evidence visualization. Journal of Learning Analytics, 4(2), 160–188. https://doi.org/10.18608/jla.2017.42.13
Dickler, R., Gobert, J., & Sao Pedro, M. (2021). Using innovative methods to explore the potential of an alerting dashboard for science inquiry. Journal of Learning Analytics, 8(2), 105–122. https://doi.org/10.18608/jla.2021.7153
Donohoe, D., & Costello, E. (2020). Data visualisation literacy in higher education: An exploratory study of understanding of a learning dashboard tool. International Journal of Emerging Technologies in Learning (iJET), 15(17), 115–126. https://doi.org/10.3991/ijet.v15i17.15041
Dourado, R. A., Rodrigues, R. L., Ferreira, N., Mello, R. F., Gomes, A. S., & Verbert, K. (2021). A teacher-facing learning analytics dashboard for process-oriented feedback in online learning. In Proceedings of the 11th International Conference on Learning Analytics and Knowledge (LAK 2021), 12–16 April 2021, Irvine, California, USA (pp. 482–489). ACM. https://doi.org/10.1145/3448139.3448187
Echeverria, V., Martinez-Maldonado, R., Buckingham Shum, S., Chiluiza, K., Granda, R., & Conati, C. (2018). Exploratory versus explanatory visual learning analytics: Driving teachers’ attention through educational data storytelling. Journal of Learning Analytics, 5(3), 73–97. https://doi.org/10.18608/jla.2018.53.6
Echeverria, V., Martinez-Maldonado, R., Granda, R., Chiluiza, K., Conati, C., & Buckingham Shum, S. (2018). Driving data storytelling from learning design. In Proceedings of the Eighth International Conference on Learning Analytics and Knowledge (LAK 2018), 7–9 March 2018, Sydney, New South Wales, Australia (pp. 131–140). ACM. https://doi.org/10.1145/3170358.3170380
Echeverria, V., Yan, L., Zhao, L., Abel, S., Alfredo, R., Dix, S., Jaggard, H., Wotherspoon, R., Osborne, A., Buckingham Shum, S., Gasevic, D., & Martinez-Maldonado, R. (2024). TeamSlides: A multimodal teamwork analytics dashboard for teacher-guided reflection in a physical learning space. In Proceedings of the 14th International Conference on Learning Analytics and Knowledge (LAK 2024), 18–22 March 2024, Kyoto, Japan (pp. 112–122). ACM. https://doi.org/10.1145/3636555.3636857
Elmoazen, R., Saqr, M., Tedre, M., & Hirsto, L. (2022). A systematic literature review of empirical research on epistemic network analysis in education. IEEE Access, 10, 17330–17348. https://doi.org/10.1109/access.2022.3149812
Fernandez Nieto, G. M., Kitto, K., Buckingham Shum, S., & Martinez-Maldonado, R. (2022). Beyond the learning analytics dashboard: Alternative ways to communicate student data insights combining visualisation, narrative and storytelling. In Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK 2022), 21–25 March 2022, online (pp. 219–229). ACM. https://doi.org/10.1145/3506860.3506895
Fernandez-Nieto, G., An, P., Zhao, J., Buckingham Shum, S., & Martinez-Maldonado, R. (2022). Classroom dandelions: Visualising participant position, trajectory and body orientation augments teachers’ sensemaking. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI 2022), 29 April–5 May 2022, New Orleans, Louisiana, USA (pp. 1–17). ACM. https://doi.org/10.1145/3491102.3517736
Fernandez-Nieto, G. M., Martinez-Maldonado, R., Echeverria, V., Kitto, K., Gaševic, D., & Buckingham Shum, S. (2024). Data storytelling editor: A teacher-centred tool for customising learning analytics dashboard narratives. In Proceedings of the 14th International Conference on Learning Analytics and Knowledge (LAK 2024), 18–22 March 2024, Kyoto, Japan (pp. 678–689). ACM. https://doi.org/10.1145/3636555.3636930
Gamoran Sherin, M., & van Es, E. A. (2009). Effects of video club participation on teachers’ professional vision. Journal of Teacher Education, 60(1), 20–37. https://doi.org/10.1177/0022487108328155
Germanakos, P., Kasinidou, M., Constantinides, M., & Samaras, G. (2019). A metacognitive perspective of InfoVis in education. In Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization (UMAP’19 Adjunct), 9–12 June 2019, Larnaca, Cyprus (pp. 331–336). ACM. https://doi.org/10.1145/3314183.3323674
Hanckel, B., Petticrew, M., Thomas, J., & Green, J. (2021). The use of qualitative comparative analysis (QCA) to address causality in complex systems: A systematic review of research on public health interventions. BMC Public Health, 21(1). https://doi.org/10.1186/s12889-021-10926-2
Hernández-Leo, D., Martinez-Maldonado, R., Pardo, A., Muñoz-Cristóbal, J. A., & Rodríguez-Triana, M. J. (2018). Analytics for learning design: A layered framework and tools. British Journal of Educational Technology, 50(1), 139–152. https://doi.org/10.1111/bjet.12645
Herodotou, C., Rienties, B., Boroowa, A., Zdrahal, Z., & Hlosta, M. (2019). A large-scale implementation of predictive learning analytics in higher education: The teachers’ role and perspective. Educational Technology Research and Development, 67(5), 1273–1306. https://doi.org/10.1007/s11423-019-09685-0
Jivet, I., Scheffel, M., Specht, M., & Drachsler, H. (2018). License to evaluate: Preparing learning analytics dashboards for educational practice. In Proceedings of the Eighth International Conference on Learning Analytics and Knowledge (LAK 2018), 7–9 March 2018, Sydney, New South Wales, Australia (pp. 31–40). ACM. https://doi.org/10.1145/3170358.3170421
Jørnø, R. L., & Gynther, K. (2018). What constitutes an “actionable insight” in learning analytics? Journal of Learning Analytics, 5(3), 198–221. https://doi.org/10.18608/jla.2018.53.13
Kaliisa, R., Jivet, I., & Prinsloo, P. (2023). A checklist to guide the planning, designing, implementation, and evaluation of learning analytics dashboards. International Journal of Educational Technology in Higher Education, 20(1). https://doi.org/10.1186/s41239-023-00394-6
Kaliisa, R., Mørch, A. I., & Kluge, A. (2022). “My point of departure for analytics is extreme skepticism”: Implications derived from an investigation of university teachers’ learning analytics perspectives and design practices. Technology, Knowledge and Learning, 27(2), 505–527. https://doi.org/10.1007/s10758-020-09488-w
Kasepalu, R., Chejara, P., Prieto, L. P., & Ley, T. (2023). Studying teacher withitness in the wild: Comparing a mirroring and an alerting and guiding dashboard for collaborative learning. International Journal of Computer-Supported Collaborative Learning, 18(4), 575–606. https://doi.org/10.1007/s11412-023-09414-z
Khosravi, H., Shabaninejad, S., Bakharia, A., Sadiq, S., Indulska, M., & Gaševi´c, D. (2021). Intelligent learning analytics dashboards: Automated drill-down recommendations to support teacher data exploration. Journal of Learning Analytics, 8(3), 133–154. https://doi.org/10.18608/jla.2021.7279
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y. - S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gaševic, D. (2022). Explainable artificial intelligence in education. Computers and Education: Artificial Intelligence, 3, 100074. https://doi.org/10.1016/j.caeai.2022.100074
Kirsh, D. (2014). Metacognition, distributed cognition, and visual design. In P. Gardenfors & P. Johansson (Eds.), Cognition, education, and communication technology (pp. 147–179). Routledge.
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017). lmerTest package: Tests in linear mixed effects models. Journal of Statistical Software, 82(13), 1–26. https://doi.org/10.18637/jss.v082.i13
Lalle, S., Toker, D., & Conati, C. (2021). Gaze-driven adaptive interventions for magazine-style narrative visualizations. IEEE Transactions on Visualization and Computer Graphics, 27(6), 2941–2952. https://doi.org/10.1109/TVCG.2019.2958540
Lee, S., Kim, S. - H., & Kwon, B. C. (2017). VLAT: Development of a visualization literacy assessment test. IEEE Transactions on Visualization and Computer Graphics, 23(1), 551–560. https://doi.org/10.1109/tvcg.2016.2598920
Li, Q., Jung, Y., & Friend Wise, A. (2021). Beyond first encounters with analytics: Questions, techniques and challenges in instructors’ sensemaking. In Proceedings of the 11th International Conference on Learning Analytics and Knowledge (LAK 2021), 12–16 April 2021, Irvine, California, USA (pp. 344–353). ACM. https://doi.org/10.1145/3448139.3448172
Liu, Y., Pozdniakov, S., & Martinez-Maldonado, R. (2024). The effects of visualisation literacy and data storytelling dashboards on teachers’ cognitive load. Australasian Journal of Educational Technology, 40(1), 78–93. https://doi.org/10.14742/ajet.8988
Marquart, C. L., Swiecki, Z., Collier, W., Eagan, B., Woodward, R., & Shaffer, D. W. (2017). rENA: Epistemic network analysis. https://doi.org/10.32614/cran.package.rena
Martinez-Maldonado, R., Echeverria, V., Fernandez Nieto, G., & Buckingham Shum, S. (2020). From data to insights: A layered storytelling approach for multimodal learning analytics. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI 2020), 25–30 April 2020, Honolulu, Hawaii, USA (pp. 1–15). ACM. https://doi.org/10.1145/3313831.3376148
Martinez-Maldonado, R., Echeverria, V., Fernandez-Nieto, G., Yan, L., Zhao, L., Alfredo, R., Li, X., Dix, S., Jaggard, H., Wotherspoon, R., Osborne, A., Buckingham Shum, S., & Gaševic, D. (2023). Lessons learnt from a multimodal learning analytics deployment in-the-wild. ACM Transactions on Computer-Human Interaction, 31(1), 1–41. https://doi.org/10.1145/3622784
McDonald, N., Schoenebeck, S., & Forte, A. (2019). Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1–23. https://doi.org/10.1145/3359174
McDonald, S., Zhao, T., & Edwards, H. M. (2013). Dual verbal elicitation: The complementary use of concurrent and retrospective reporting within a usability test. International Journal of Human-Computer Interaction, 29(10), 647–660. https://doi.org/10.1080/10447318.2012.758529
McIntyre, N. A., Draycott, B., & Wolff, C. E. (2022). Keeping track of expert teachers: Comparing the affordances of think- aloud elicited by two different video perspectives. Learning and Instruction, 80, 101563. https://doi.org/10.1016/j.learninstruc.2021.101563
Milesi, M. E., & Martinez-Maldonado, R. (2024). Data storytelling in learning analytics? A qualitative investigation into educators’ perceptions of benefits and risks. In Proceedings of the 14th International Conference on Learning Analytics and Knowledge (LAK 2024), 18–22 March 2024, Kyoto, Japan (pp. 167–177). ACM. https://doi.org/10.1145/3636555.3636865
Molenaar, I., & Knoop-van Campen, C. A. N. (2019). How teachers make dashboard information actionable. IEEE Transactions on Learning Technologies, 12(3), 347–355. https://doi.org/10.1109/TLT.2018.2851585
Molenaar, I., Horvers, A., Dijkstra, R., & Baker, R. S. (2020). Personalized visualizations to promote young learners’ SRL: The learning path app. In Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK 2020), 23–27 March 2020, Frankfurt, Germany. ACM. https://doi.org/10.1145/3375462.3375465
Nazaretsky, T., Ariely, M., Cukurova, M., & Alexandron, G. (2022). Teachers’ trust in AI-powered educational technology and a professional development program to improve it. British Journal of Educational Technology, 53(4), 914–931.
https://doi.org/10.1111/bjet.13232
Nazaretsky, T., Cukurova, M., & Alexandron, G. (2022). An instrument for measuring teachers’ trust in AI-based educational technology. In Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK 2022), 21–25 March 2022, online (pp. 56–66). ACM. https://doi.org/10.1145/3506860.3506866
Ndukwe, I. G., & Daniel, B. K. (2020). Teaching analytics, value and tools for teacher data literacy: A systematic and tripartite approach. International Journal of Educational Technology in Higher Education, 17(1). https://doi.org/10.1186/s41239-020-00201-6
Pandey, S., & Ottley, A. (2023). Mini-VLAT: A short and effective measure of visualization literacy. Computer Graphics Forum, 42(3), 1–11. https://doi.org/10.1111/cgf.14809
Poquet, O. (2024). A shared lens around sensemaking in learning analytics: What activity theory, definition of a situation and affordances can offer. British Journal of Educational Technology, 55(4), 1811–1831. https://doi.org/10.1111/bjet.13435
Pozdniakov, S., Martinez-Maldonado, R., Singh, S., Khosravi, H., & Gaševi´c, D. (2023). Using learning analytics to support teachers. In B. du Boulay, A. Mitrovic, & K. Yacef (Eds.), Handbook of artificial intelligence in education (pp. 322–349). Edward Elgar Publishing. https://doi.org/10.4337/9781800375413.00026
Pozdniakov, S., Martinez-Maldonado, R., Tsai, Y.- S., Cukurova, M., Bartindale, T., Chen, P., Marshall, H., Richardson, D., & Gasevic, D. (2022). The question-driven dashboard: How can we design analytics interfaces aligned to teachers’ inquiry? In Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK 2022), 21–25 March 2022, online (pp. 175–185). ACM. https://doi.org/10.1145/3506860.3506885
Pozdniakov, S., Martinez-Maldonado, R., Tsai, Y.-S., Echeverria, V., Srivastava, N., & Gasevic, D. (2023). How do teachers use dashboards enhanced with data storytelling elements according to their data visualisation literacy skills? In Proceedings of the 13th International Conference on Learning Analytics and Knowledge (LAK 2023), 13–17 March 2023, Arlington, Texas, USA (pp. 89–99). ACM. https://doi.org/10.1145/3576050.3576063
Pozdniakov, S., Martinez-Maldonado, R., Tsai, Y.-S., Srivastava, N., Liu, Y., & Gasevic, D. (2023). Single or multi-page learning analytics dashboards? Relationships between teachers’ cognitive load and visualisation literacy. In O. Viberg, I. Jivet, P. Muñoz-Merino, M. Perifanou, & T. Papathoma (Eds.), Responsive and sustainable educational futures. EC-TEL 2023. Lecture notes in computer science (pp. 339–355, Vol. 14200). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-42682-7_23
Rets, I., Herodotou, C., & Gillespie, A. (2023). Six practical recommendations enabling ethical use of predictive learning analytics in distance education. Journal of Learning Analytics, 10(1), 149–167. https://doi.org/10.18608/jla.2023.7743
Sahin, M., & Ifenthaler, D. (2021). Visualizations and dashboards for learning analytics: A systematic literature review. In M. Sahin & D. Ifenthaler (Eds.), Visualizations and Dashboards for Learning Analytics (pp. 3–22). Springer International Publishing. https://doi.org/10.1007/978-3-030-81222-5_1
Salas-Pilco, S. Z., Xiao, K., & Hu, X. (2022). Artificial intelligence and learning analytics in teacher education: A systematic review. Education Sciences, 12(8), 569. https://doi.org/10.3390/educsci12080569
Saldaña, J. (2015). The coding manual for qualitative researchers. SAGE.
Sampson, D. (2017). Teaching and learning analytics to support teacher inquiry. In Proceedings of the 2017 IEEE Global Engineering Education Conference (EDUCON 2017), 25–28 April 2017, Athens, Greece (pp. 1881–1882). IEEE. https://doi.org/10.1109/EDUCON.2017.7943109
Sarmiento, J. P., & Wise, A. F. (2022). Participatory and co-design of learning analytics: An initial review of the literature. In Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK 2022), 21–25 March 2022, online (pp. 535–541). ACM. https://doi.org/10.1145/3506860.3506910
Schwendimann, B. A., Rodriguez-Triana, M. J., Vozniuk, A., Prieto, L. P., Boroujeni, M. S., Holzer, A., Gillet, D., & Dillenbourg, P. (2017). Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies, 10(1), 30–41. https://doi.org/10.1109/TLT.2016.2599522
Shaffer, D. W., Hatfield, D., Svarovsky, G. N., Nash, P., Nulty, A., Bagley, E., Frank, K., Rupp, A. A., & Mislevy, R. (2009). Epistemic network analysis: A prototype for 21st-century assessment of learning. International Journal of Learning and Media, 1(2), 33–53. https://doi.org/10.1162/ijlm.2009.0013
Sloan-Lynch, J., & Morse, R. (2024). Equity-forward learning analytics: Designing a dashboard to support marginalized student success. In Proceedings of the 14th International Conference on Learning Analytics and Knowledge (LAK 2024),
–22 March 2024, Kyoto, Japan (pp. 1–11). ACM. https://doi.org/10.1145/3636555.3636844
Sousa, E. B. d., Alexandre, B., Ferreira Mello, R., Pontual Falcão, T., Vesin, B., & Gaševi´c, D. (2021). Applications of learning analytics in high schools: A systematic literature review. Frontiers in Artificial Intelligence, 4, 737891. https://doi.org/10.3389/frai.2021.737891
Tsai, Y.-S., Kovanovi´c, V., & Gaševi´c, D. (2021). Connecting the dots: An exploratory study on learning analytics adoption factors, experience, and priorities. The Internet and Higher Education, 50, 100794. https://doi.org/10.1016/j.iheduc.2021.100794
Van Es, E. A., & Sherin, M. G. (2002). Learning to notice: Scaffolding new teachers’ interpretations of classroom interactions. Journal of Technology and Teacher Education, 10(4), 571–596. https://www.learntechlib.org/primary/p/9171/
van Leeuwen, A., van Wermeskerken, M., Erkens, G., & Rummel, N. (2017). Measuring teacher sense making strategies of learning analytics: A case study. Learning: Research and Practice, 3(1), 42–58. https://doi.org/10.1080/23735082.2017.1284252
van Leeuwen, A. (2019). Teachers’ perceptions of the usability of learning analytics reports in a flipped university course: When and how does information become actionable knowledge? Educational Technology Research and Development, 67(5), 1043–1064. https://doi.org/10.1007/s11423-018-09639-y
van Leeuwen, A., Knoop-van Campen, C. A. N., Molenaar, I., & Rummel, N. (2021). How teacher characteristics relate to how teachers use dashboards: Results from two case studies in K-12. Journal of Learning Analytics, 8(2), 6–21. https://doi.org/10.18608/jla.2021.7325
van Leeuwen, A., & Rummel, N. (2022). The function of teacher dashboards depends on the amount of time pressure in the classroom situation: Results from teacher interviews and an experimental study. Unterrichtswissenschaft, 50(4), 561–588. https://doi.org/10.1007/s42010-022-00156-9
van Leeuwen, A., Rummel, N., & van Gog, T. (2019). What information should CSCL teacher dashboards provide to help teachers interpret CSCL situations? International Journal of Computer-Supported Collaborative Learning, 14(3), 261–289. https://doi.org/10.1007/s11412-019-09299-x
Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1500–1509. https://doi.org/10.1177/0002764213479363
Verbert, K., Ochoa, X., De Croon, R., Dourado, R. A., & De Laet, T. (2020). Learning analytics dashboards: The past, the present and the future. In Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK 2020), 23–27 March 2020, Frankfurt, Germany (pp. 35–40). ACM. https://doi.org/10.1145/3375462.3375504
Willett, W., Heer, J., & Agrawala, M. (2007). Scented widgets: Improving navigation cues with embedded visualizations. IEEE Transactions on Visualization and Computer Graphics, 13(6), 1129–1136. https://doi.org/10.1109/TVCG.2007.70589
Wise, A. F., & Jung, Y. (2019). Teaching with analytics: Towards a situated model of instructional decision-making. Journal of Learning Analytics, 6(2), 53–69. https://doi.org/10.18608/jla.2019.62.4
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution 4.0 International License.
