Using Innovative Methods to Explore the Potential of an Alerting Dashboard for Science Inquiry




teacher dashboard, epistemic network analyses, science inquiry


Educational technologies, such as teacher dashboards, are being developed to support teachers’ instruction and students’ learning. Specifically, dashboards support teachers in providing the just-in-time instruction needed by students in complex contexts such as science inquiry. In this study, we used the Inq-Blotter teacher-alerting dashboard to investigate whether teacher support elicited by the technology influenced students’ inquiry performance in a science intelligent tutoring system, Inq-ITS. Results indicated that students’ inquiry improved after receiving teachers’ help, elicited by the Inq-Blotter alerts. This inquiry improvement was significantly greater than for matched students who did not receive help from the teacher in response to alerts. Epistemic network analyses were then used to investigate the patterns in the discursive supports provided to students by teachers. These analyses revealed significant differences in the types of support that fostered (versus did not foster) student improvement; differences across teachers were also found. Overall, this study used innovative tools and analyses to understand how teachers use this technological genre of alerting dashboards to dynamically support students in science inquiry.


Acosta, A., & Slotta, J. D. (2018). CKBiology: An active learning curriculum design for secondary biology. Frontiers in Education, 3, 1–19.

Adair, A., Dickler, R., & Gobert, J. (2020). Supporting teachers supporting students: Evidence-based TIPS in a dashboard to guide inquiry scaffolding. In Proceedings of the 14th International Conference of the Learning Sciences (ICLS 2020), 19–23 June 2020, Nashville, TN, USA (pp. 1769–1770). ISLS. Retrieved from

Aleven, V., Roll, I., McLaren, B. M., & Koedinger, K. R. (2016). Help helps, but only so much: Research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 26(1), 205–223.

Charleer, S., Klerkx, J., & Duval, E. (2014). Learning dashboards. Journal of Learning Analytics, 1(3), 199–202.

Dickler, R., Gobert, J., & Sao Pedro, M. (2019a). Using epistemic network analysis to characterize teacher discourse in response to an alerting dashboard. Paper presented at the 29th Annual Meeting of the Society for Text and Discourse (ST&D 2019), 9–11 July 2019, New York.

Dickler, R., Li, H., & Gobert, J. (2019b). Teacher scaffolds mediated by a science inquiry dashboard. Paper presented at the 14th Conference of the European Science Education Research Association (ESERA 2019), 26–30 August 2019, Bologna, Italy.

Dillenbourg, P. (2013). Design for classroom orchestration. Computers & Education, 69, 485–492.

Gee, J. P. (2004). An Introduction to discourse analysis: Theory and method. London, UK: Routledge.

Gobert, J. D., Baker, R. S., & Sao Pedro, M. A. (2014). Inquiry skills tutoring system. U.S. Patent No. 9,373,082. Washington, DC: U.S. Patent and Trademark Office.

Gobert, J., Moussavi, R., Li, H., Sao Pedro, M., & Dickler, R. (2018). Real-time scaffolding of students’ online data interpretation during inquiry with Inq-ITS using educational data mining. In A. K. M. Azad, M. Auer, A. Edwards, & T. de Jong (Eds.), Cyber-physical laboratories in engineering and science education (pp. 191–217). Cham: Springer.

Gobert, J. D., & Sao Pedro, M. A. (2017). Digital assessment environments for scientific inquiry practices. In A. A. Rupp & J. P. Leighton (Eds.), The Wiley handbook of cognition and assessment: Frameworks, methodologies, and applications (pp. 508–534), West Sussex, UK: Wiley.

Gobert, J., Sao Pedro, M., Betts, C., & Baker, R. S. (2016). Inquiry skills tutoring system (child patent for alerting system). U.S. Patent No. 9,564,057. Washington, DC: U.S. Patent and Trademark Office.

Gobert, J. D., Sao Pedro, M., Raziuddin, J., & Baker, R. S. (2013). From log files to assessment metrics: Measuring students’ science inquiry skills using educational data mining. Journal of the Learning Sciences, 22(4), 521–563.

Holstein, K., McLaren, B. M., & Aleven, V. (2018a). Student learning benefits of a mixed-reality teacher awareness tool in AI-enhanced classrooms. In Proceedings of the 19th International Conference on Artificial Intelligence in Education (AIED 2018), 27–30 June 2018, London, UK (pp. 154–168). Cham: Springer.

Holstein, K., McLaren, B. M., & Aleven, V. (2018b). Informing the design of teacher awareness tools through causal alignment analysis. In Proceedings of the 13th International Conference of the Learning Sciences (ICLS 2018), 23–27 June 2018, London, UK (pp. 104–111).

Holstein, K., McLaren, B. M., & Aleven, V. (2019). Co-designing a real-time classroom orchestration tool to support teacher–AI complementarity. Journal of Learning Analytics, 6(2), 27–52.

Howe, C., Hennessy, S., Mercer, N., Vrikki, M., & Wheatley, L. (2019). Teacher–student dialogue during classroom teaching: Does it really impact on student outcomes? Journal of the Learning Sciences, 28(4-5), 462–512.

Jurow, A. S., & Creighton, L. (2005). Improvisational science discourse: Teaching science in two K-1 classrooms. Linguistics and Education, 16(3), 275–297.

Laurillard, D. (2008). The teacher as action researcher: Using technology to capture pedagogic form. Studies in Higher Education, 33(2), 139–154.

Manz, E., & Renga, I. P. (2017). Understanding how teachers guide evidence construction conversations. Science Education, 101(4), 584–615.

Marquart, C. L., Hinojosa, C., Swiecki, Z., Eagan, B., & Shaffer, D. W. (2018). Epistemic network analysis (version 1.5.2). Retrieved from

Martinez-Maldonado, R., Clayphan, A., Yacef, K., & Kay, J. (2015). MTFeedback: Providing notifications to enhance teacher awareness of small group work in the classroom. IEEE Transactions on Learning Technologies, 8(2), 187–200.

Martinez-Maldonado, R., Kay, J., Yacef, K., Edbauer, M. T., & Dimitriadis, Y. (2013). MTClassroom and MTDashboard: Supporting analysis of teacher attention in an orchestrated multi-tabletop classroom. In Proceedings of the 10th International Conference on Computer Supported Collaborative Learning (CSCL 2013), 15–19 June 2013, Madison, WI, USA. (pp. 320–327).

Martinez-Maldonado, R., Kay, J., Yacef, K., & Schwendimann, B. (2012). An interactive teacher’s dashboard for monitoring groups in a multi-tabletop learning environment. In Proceedings of the International Conference on Intelligent Tutoring Systems (ITS 2012), 14–18 June 2012, Chania, Greece (pp. 482–492). Springer.

Matuk, C., Gerard, L., Lim-Breitbart, J., & Linn, M. (2016). Gathering requirements for teacher tools: Strategies for empowering teachers through co-design. Journal of Science Teacher Education, 27(1), 79–110.

Mislevy, R. J., Behrens, J. T., Dicerbo, K. E., & Levy, R. (2012). Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. Journal of Educational Data Mining, 4(1), 11–48.

Molenaar, I., & Knoop-van Campen, C. (2017). Teacher dashboards in practice: Usage and impact. In Proceedings of the 12th European Conference on Technology Enhanced Learning (EC-TEL 2017), 12–15 September 2017, Tallinn, Estonia (pp. 125–138). Springer, Cham.

Molenaar, I., &. Knoop-van Campen, C. (2018). Experience matters: The impact of dashboards on teachers’ feedback. In Proceedings of the 13th International Conference of the Learning Sciences (ICLS ’18), 23–27 June 2018, London, UK (pp. 1231–1232).

Next Generation Science Standards Lead States. (2013). Next generation science standards: For states, by states. The National Academies Press.

Pruitt, S. L. (2014). The next generation science standards: The features and challenges. Journal of Science Teacher Education, 25(2), 145–156.

Roschelle, J., Martin, W., Ahn, J., & Schank, P. (Eds.) (2017). Cyberlearning community report: The state of cyberlearning and the future of learning with technology (technical report). SRI International. Retrieved from

Rupp, A. A., Choi, Y., Gushta, M., Mislevy, R., Thies, M.C., Bagley, E., … Shaffer, D. (2009). Modeling learning progressions in epistemic games with epistemic network analysis: Principles for data analysis and generation. In Proceedings of the 2009 Learning Progressions in Science Conference (LeaPS 2009), 24–26 June 2009, Iowa City, IA, USA (pp. 24–26). Retrieved from

Sao Pedro, M. A., Gobert, J., & Dickler, R. (2019). Can an alerting teacher dashboard improve how teachers help their students learn science inquiry practices? Paper presented at the 2019 American Educational Research Association Annual Meeting: Learning and Instruction (AERA 2019), 5–9 April 2019, Toronto, ON, Canada.

Schwarz, B. B., Prusak, N., Swidan, O., Livny, A., Gal, K., & Segal, A. (2018). Orchestrating the emergence of conceptual learning: A case study in a geometry class. International Journal of Computer-Supported Collaborative Learning, 13(2), 189–211.

Segal, A., Hindi, S., Prusak, N., Swidan, O., Livni, A., Palatnic, A., & Schwarz, B. (2017). Keeping the teacher in the loop: Technologies for monitoring group learning in real-time. In Proceedings of the 18th International Conference on Artificial Intelligence in Education (AIED 2017), 28 June–1 July 2017, Wuhan, China (pp. 64–76). Cham: Springer.

Shaffer, D. W., Collier, W., & Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. Journal of Learning Analytics, 3(3), 9–45.

Sherin, M., Jacobs, V., & Philipp, R. (Eds.). (2011). Mathematics teacher noticing: Seeing through teachers’ eyes. London, UK: Routledge.

Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 7, 1–22.

Svarovsky, G. N. (2011). Exploring complex engineering learning over time with epistemic network analysis. Journal of Pre-College Engineering Education, 1(2), Article 4.

Tissenbaum, M., & Slotta, J. (2019). Supporting classroom orchestration with real-time feedback: A role for teacher dashboards and real-time agents. International Journal of Computer-Supported Collaborative Learning, 14(3), 325–351.

Tabak, I., & Kyza, E. A. (2018). Research on scaffolding in the learning sciences: A methodological perspective. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, & Peter Reimann (Eds.), International handbook of the learning sciences (pp. 191–200). Routledge.

van Leeuwen, A. & Rummel, N. (2018). Considering teachers’ informational needs and the role division between teacher and orchestration tool in the context of collaborative learning. In Proceedings of the 13th International Conference of the Learning Sciences (ICLS 2018), 23–27 June 2018, London, UK (pp. 1230–1231).

van Leeuwen, A., van Wermeskerken, M., Erkens, G., & Rummel, N. (2017). Measuring teacher sense making strategies of learning analytics: A case study. Learning: Research and Practice, 3(1), 42–58.

Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Van Assche, F., Parra, G., & Klerkx, J. (2014). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499–1514.

West, D. M. (2012). Big data for education: Data mining, data analytics, and web dashboards. Governance Studies at Brookings. Retrieved from




How to Cite

Dickler, R., Gobert, J., & Sao Pedro, M. (2021). Using Innovative Methods to Explore the Potential of an Alerting Dashboard for Science Inquiry. Journal of Learning Analytics, 8(2), 105-122.