Beyond Time on Task
A Novel Analytical Framework for Assessing Student Workload and Its Relationship with Learning
DOI:
https://doi.org/10.18608/jla.2026.8725Keywords:
student workload, workload peaks, workload distribution, time-on-task, learning management systems, research paperAbstract
Student workload analysis has the potential to play a crucial role in providing both actionable insights to inform course design and curricular adjustments that promote student learning and well-being. While numerous studies have emphasized the need for analyzing workload beyond single-value metrics, such as credit hours, the interpretation and practical application of these metrics for educational interventions remains unclear. In this study, we explore the interplay between time-on-task measurements with student-perceived learning and difficulty.We move beyond average indicators of time-on-task by proposing and examining various metrics related to the dynamics of workload over time. Across 14 engineering courses taught at Pontificia Universidad Católica de Chile, we analyze three different sources of data: (1) self-reported time-on-task and perceived difficulty obtained through a weekly timesheet
survey, (2) interactions with the learning management system (LMS), and (3) perceived learning attainment obtained from the course evaluation survey. Our results show that LMS-based and self-reported time-on-task were highly correlated. Also, workload dynamics metrics, such as the presence of workload peaks, were highly correlated with perceived learning and perceived difficulty. As such, this study provides evidence in support of considering workload dynamics, rather than average measures of time-on-task, to predict variables related to student learning. The metrics proposed by this framework could be used to implement practical tools for educators and administrators willing to optimize course design and improve learning attainment.
References
Abazi-Bexheti, L., Kadriu, A., Jajaga, E., Apostolova-Trpkovska, M., & Abazi-Alili, H. (2018). LMS solution: Evidence of google classroom usage in higher education. Business Systems Research: International journal of the Society for Advancing Innovation and Research in Economy, 9(1), 31–43. https://doi.org/10.2478/bsrj-2018-0003
Armstrong, J. (1996). Workload in engineering courses and how to reduce it. Proceedings of the Eighth Annual Conference of the Australasian Association for Engineering Education (AAEE 1996), 15–18 December 1996, Sydney, Australia.
Bachman, L., & Bachman, C. (2006). Student perceptions of academic workload in architectural education. Journal of Architectural and Planning Research, 271–304. https://www.jstor.org/stable/43030781
Baturay, M. H. (2008). Characteristics of basic instructional design models. Ekev Academic Review, 12(34), 471–482. https://www.academia.edu/42158434/
Beer, N. (2019). Estimating student workload during the learning design of online courses: Creating a student workload calculator. In R. Ørngreen, M. Buhl, & B. Mayer (Eds.), Proceedings of the European Conference on E-Learning (ECEL 2019), 7–8 November 2019, Aalborg, Denmark (pp. 629–638). ACPI. https://research.lancaster-university.uk/en/publications/estimating-student-workload-during-the-learning-design-of-online-/
Bennett, A., & Burke, P. J. (2018). Re/conceptualising time and temporality: An exploration of time in higher education. Discourse: Studies in the Cultural Politics of Education, 39(6), 913–925. https://doi.org/10.1080/01596306.2017.1312285
Bentley, P. J., & Kyvik, S. (2012). Academic work from a comparative perspective: A survey of faculty working time across 13 countries. Higher Education, 63, 529–547. https://doi.org/10.1007/s10734-011-9457-4
Bladek, M. (2021). Student well-being matters: Academic library support for the whole student. The Journal of Academic Librarianship, 47(3), 102349. https://doi.org/10.1016/j.acalib.2021.102349
Bono, R., Blanca, M. J., Arnau, J., & Gómez-Benito, J. (2017). Non-normal distributions commonly used in health, education, and social sciences: A systematic review. Frontiers in Psychology, 8, 1602. https://doi.org/10.3389/fpsyg.2017.01602
Bowling, N. A., Alarcon, G. M., Bragg, C. B., & Hartman, M. J. (2015). A meta-analytic examination of the potential correlates and consequences of workload. Work & Stress, 29(2), 95–113. https://doi.org/10.1080/02678373.2015.1033037
Bowyer, K. (2012). A model of student workload. Journal of Higher Education Policy and Management, 34(3), 239–258. https://doi.org/10.1080/1360080x.2012.678729
Carroll, J. B. (1963). A model of school learning. Teachers College Record, 64(8), 1–9. https://doi.org/10.1177/016146816306400801
Cerezo, R., Sánchez-Santillán, M., Paule-Ruiz, M. P., & Núñez, J. C. (2016). Students’ LMS interaction patterns and their relationship with achievement: A case study in higher education. Computers & Education, 96, 42–54. https://doi.org/10.1016/j.compedu.2016.02.006
Chadha, D., Kogelbauer, A., Campbell, J., Hellgardt, K., Maraj, M., Shah, U., Brechtelsbauer, C., & Hale, C. (2021). Are the kids alright? Exploring students’ experiences of support mechanisms to enhance wellbeing on an engineering programme in the UK. European Journal of Engineering Education, 46(5), 662–677. https://doi.org/10.1080/03043797.2020.1835828
Cho, M. - H., & Yoo, J. S. (2017). Exploring online students’ self-regulated learning with self-reported surveys and log files: A data mining approach. Interactive Learning Environments, 25(8), 970–982. https://doi.org/10.1080/10494820.2016.1232278
Chuderski, A. (2016). Time pressure prevents relational learning. Learning and Individual Differences, 49, 361–365. https://doi.org/10.1016/j.lindif.2016.07.006
Conijn, R., Snijders, C., Kleingeld, A., & Matzat, U. (2016). Predicting student performance from LMS data: A comparison of 17 blended courses using Moodle LMS. IEEE Transactions on Learning Technologies, 10(1), 17–29. https://doi.org/10.1109/tlt.2016.2616312
De Beer, L. T., Pienaar, J., & Rothmann Jr, S. (2016). Work overload, burnout, and psychological ill-health symptoms: A three-wave mediation model of the employee health impairment process. Anxiety, Stress, & Coping, 29(4), 87–399. https://doi.org/10.1080/10615806.2015.1061123
De Winter, J. C., Gosling, S. D., & Potter, J. (2016). Comparing the Pearson and Spearman correlation coefficients across distributions and sample sizes: A tutorial using simulations and empirical data. Psychological Methods, 21(3), 273. https://doi.org/10.1037/met0000079
Edu-Valsania, S., Laguía, A., & Moriano, J. A. (2022). Burnout: A review of theory and measurement. International Journal of Environmental Research and Public Health, 19(3), 1780. https://doi.org/10.3390/ijerph19031780
Egea, G., Rodríguez-Lizana, A., Pérez-Urrestarazu, L., Pérez-Ruiz, M., Rallo, P., & Suárez, M. P. (2022). Assessment of actual workload and student performance in the agricultural engineering final degree project in a Spanish higher education context. Education Sciences, 12(6), 418. https://doi.org/10.3390/educsci12060418
Gasevic, D., Dawson, S., Rogers, T., & Gasevic, D. (2016). Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education, 28, 68–84. https://doi.org/10.1016/j.iheduc.2015.10.002
Gleeson, J., Lynch, R., & McCormack, O. (2021). The European Credit Transfer System (ECTS) from the perspective of Irish = teacher educators. European Educational Research Journal, 20(3), 365–389. https://doi.org/10.1177/1474904120987101
Heffernan, J. M. (1973). The credibility of the credit hour: The history, use, and shortcomings of the credit system. The Journal of Higher Education, 44(1), 61–72. https://doi.org/10.1080/00221546.1973.11776844
Henrie, C. R., Bodily, R., Larsen, R., & Graham, C. R. (2018). Exploring the potential of LMS log data as a proxy measure of student engagement. Journal of Computing in Higher Education, 30, 344–362. https://doi.org/10.1007/s12528-017-9161-1
Hicks, T. G., & Wierwille, W. W. (1979). Comparison of five mental workload assessment procedures in a moving-base driving simulator. Human Factors, 21(2), 129–143. https://doi.org/10.1177/001872087902100201
Hilliger, I., Astudillo, G., & Baier, J. (2023). Lacking time: A case study of student and faculty perceptions of academic workload in the COVID-19 pandemic. Journal of Engineering Education, 112(3), 796–815. https://doi.org/10.1002/jee.20525
Hilliger, I., Miranda, C., Schuit, G., Duarte, F., Anselmo, M., & Parra, D. (2021). Evaluating a learning analytics dashboard to visualize student self-reports of time-on-task: A case study in a Latin American university. In Proceedings of the 11th International Conference on Learning Analytics and Knowledge (LAK 2021), 12–16 April 2021, Irvine, California, USA (pp. 592–598). ACM. https://doi.org/10.1145/3448139.3448203
Jovanović, J., Gasević, D., Dawson, S., Pardo, A., & Mirriahi, N. (2017). Learning analytics to unveil learning strategies in a flipped classroom. The Internet and Higher Education, 33, 74–85. https://doi.org/10.1016/j.iheduc.2017.02.001
Jovanović, J., Saqr, M., Joksimović, S., & Gasević, D. (2021). Students matter the most in learning analytics: The effects of internal and instructional conditions in predicting academic success. Computers & Education, 172, 104251. https://doi.org/10.1016/j.compedu.2021.104251
Karjalainen, A., Alha, K., & Jutila, S. (2006). Give me time to think: Determining student workload in higher education, five years, two degrees (tech. rep.). Ministry of Education, 2004–2006, Finland. Oulu University Press. https://oula.finna.fi/Record/oy.9910448063906252
Karweit, N. (1984). Time-on-task reconsidered: Synthesis of research on time and learning. Educational leadership, 41(8), 32–35. https://files.ascd.org/staticfiles/ascd/pdf/journals/edlead/el_198405_karweit.pdf
Kember, D. (2004). Interpreting student workload and the factors which shape students’ perceptions of their workload. Studies in Higher Education, 29(2), 165–184. https://doi.org/10.1080/0307507042000190778
Kember, D., & Leung, D. Y. (1998). The dimensionality of approaches to learning: An investigation with confirmatory factor analysis on the structure of the SPQ and LPQ. British Journal of Educational Psychology, 68(3), 395–407. https://doi.org/10.1111/j.2044-8279.1998.tb01300.x
Khan, I., & Pardo, A. (2016). Data2U: Scalable real-time student feedback in active learning environments. In Proceedings of the Sixth International Conference on Learning Analytics and Knowledge (LAK 2016), 25–29 April 2016, Edinburgh, Scotland, UK (pp. 249–253). ACM. https://doi.org/10.1145/2883851.2883911
Kovanović, V., Gasević, D., Dawson, S., Joksimović, S., & Baker, R. (2015). Does time-on-task estimation matter? Implications on validity of learning analytics findings. Journal of Learning Analytics, 2(3), 81–110. https://doi.org/10.18608/jla.2015.23.6
Kyndt, E., Berghmans, I., Dochy, F., & Bulckens, L. (2014). “Time is not enough.” Workload in higher education: a student perspective. Higher Education Research & Development, 33(4), 684–698. https://doi.org/10.1080/07294360.2013.863839
Leinonen, J., Castro, F. E. V., & Hellas, A. (2022). Time-on-task metrics for predicting performance. ACM Inroads, 13(2), 42-49. https://doi.org/10.1145/3534564
Leitner, P., Khalil, M., & Ebner, M. (2017). Learning analytics in higher education—A literature review. In A. Pena-Ayala (Ed.), Learning analytics: Fundamentals, applications, and trends: A view of the current state of the art to enhance e-learning (pp. 1–23). Springer. https://doi.org/10.1007/978-3-319-52977-6_1
Liu, Q., & Evans, G. (2020). Supporting information for student workload quick guides for instructors and students (tech. rep.). Institute for Studies in Transdisciplinary Engineering Education and Practice, University of Toronto. https://istep.utoronto.ca/wp-content/uploads/sites/35/2020/08/Supporting-Information-for-Student-Workload-Quick-Guides-for-Instructors-and-Students-Aug10-2020.pdf
Marshall, S. J. (2018). Student time choices and success. Higher Education Research & Development, 37(6), 1216–1230. https://doi.org/10.1080/07294360.2018.1462304
Maslach, C., & Leiter, M. P. (2016). Understanding the burnout experience: Recent research and its implications for psychiatry.
World Psychiatry, 15(2), 103–111. https://doi.org/10.1002/wps.20311
Maslennikova, A., Rotelli, D., & Monreale, A. (2023). Session-based time-window identification in virtual learning environments. Journal of Learning Analytics, 10(3), 7–27. https://doi.org/10.18608/jla.2023.7911
Matcha, W., Gasević, D., Jovanović, J., Uzir, N. A., Oliver, C. W., Murray, A., & Gasevic, D. (2020). Analytics of learning strategies: The association with the personality traits. In Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK 2020), 23–27 March 2020, Frankfurt, Germany (pp. 151–160). ACM. https://doi.org/10.1145/3375462.3375534
Miller, S. (2001). Workload measures (tech. rep.). National Advanced Driving Simulator, University of Iowa. https://www.nads-sc.uiowa.edu/publicationstorage/200501251347060.n01-006.pdf
Nassar, A. K., Waheed, A., & Tuma, F. (2019). Academic clinicians’ workload challenges and burnout analysis. Cureus, 11(11). https://doi.org/10.7759/cureus.6108
Nguyen, Q. (2020). Rethinking time-on-task estimation with outlier detection accounting for individual, time, and task differences. In Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK 2020), 23–27 March 2020, Frankfurt, Germany (pp. 376–381). ACM. https://doi.org/10.1145/3375462.3375538
Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., & Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior, 76, 703–714. https://doi.org/10.1016/j.chb.2017.03.028
Pardos, Z. A., Borchers, C., & Yu, R. (2023). Credit hours is not enough: Explaining undergraduate perceptions of course workload using LMS records. The Internet and Higher Education, 56, 100882. https://doi.org/10.1016/j.iheduc.2022.100882
Pastores, S. M., Kvetan, V., Coopersmith, C. M., Farmer, J. C., Sessler, C., Christman, J. W., D’Agostino, R., Diaz-Gomez, J., Gregg, R. A., Sara R.and Khan, Kapu, A. N., Masur, H., Mehta, G., Moore, J., Oropello, J. M., & Price, K. (2019). Workforce, workload, and burnout among intensivists and advanced practice providers: A narrative review. Critical Care Medicine, 47(4), 550–557. https://doi.org/10.1097/ccm.0000000000003637
Riestra-González, M., del Puerto Paule-Ruiz, M., & Ortin, F. (2021). Massive LMS log data analysis for the early prediction of course-agnostic student performance. Computers & Education, 163, 104108. https://doi.org/10.1016/j.compedu.2020.104108
Rotelli, D., & Monreale, A. (2022). Time-on-task estimation by data-driven outlier detection based on learning activities. In Proceedings of the 12th International Conference on Learning Analytics and Knowledge (LAK 2022), 21–25 March 2022, online (pp. 336–346). ACM. https://doi.org/10.1145/3506860.3506913
Ruiz-Gallardo, J.-R., Castaño, S., Gómez-Alday, J. J., & Valdés, A. (2011). Assessing student workload in problem based learning: Relationships among teaching method, student workload and achievement. A case study in natural sciences. Teaching and Teacher Education, 27(3), 619–627. https://doi.org/10.1016/j.tate.2010.11.001
Schaufeli, W. B., Enzmann, D., & Girault, N. (2017). Measurement of burnout: A review. In W. B. Schaufeli, C. Maslach, & T. Marek (Eds.), Professional burnout (pp. 199–215). Routledge. https://doi.org/10.4324/9781315227979-16
Schnotz, W., & Kürchner, C. (2007). A reconsideration of cognitive load theory. Educational Psychology Review, 19, 469–508. https://doi.org/10.1007/s10648-007-9053-4
Silva, E., White, T., & Toch, T. (2015). The Carnegie unit: A century-old standard in a changing education landscape. Carnegie Foundation for the Advancement of Teaching. https://www.luminafoundation.org/files/resources/carnegie-unit-report.pdf
Smith, A. P. (2019). Student workload, wellbeing and academic attainment. In L. Longo & M. Leva (Eds.), Human mental workload: Models and applications. H-WORKLOAD 2019. Communications in computer and information science (pp. 35–47, Vol. 1107). Springer. https://doi.org/10.1007/978-3-030-32423-0_3
Souto-Iglesias, A., & Baeza Romero, M. T. (2018). A probabilistic approach to student workload: Empirical distributions and ECTS. Higher Education, 76(6), 1007–1025. https://doi.org/10.1007/s10734-018-0270-1
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1207/s15516709cog1202_4
Sweller, J., Chandler, P., Tierney, P., & Cooper, M. (1990). Cognitive load as a factor in the structuring of technical material. Journal of Experimental Psychology: General, 119(2), 176. https://doi.org/10.1037//0096-3445.119.2.176
Sweller, J., Van Merrienboer, J. J., & Paas, F. G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251–296. https://doi.org/10.1023/a:1022193728205
Tukey, J. W. (1977). Exploratory data analysis. Addison-Wesley.
Van Merrienboer, J. J., & Sweller, J. (2005). Cognitive load theory and complex learning: Recent developments and future directions. Educational Psychology Review, 17, 147–177. https://doi.org/10.1007/s10648-005-3951-0
Wang, Q., & Mousavi, A. (2023). Which log variables significantly predict academic achievement? A systematic review and meta-analysis. British Journal of Educational Technology, 54(1), 142–191. https://doi.org/10.1111/bjet.13282
Wentling, S., & Variawa, C. (2020). Investigating differences between instructor expectation and student workload in undergraduate engineering [Conference scheduled 18–21 June 2020, Montréal, Quebec, Canada (cancelled due to COVID-19)]. In Proceedings of the Canadian Engineering Education Association (CEEA). CEEA. https://doi.org/10.24908/pceea.vi0.14189
Yang, C., Chen, A., & Chen, Y. (2021). College students’ stress and health in the COVID-19 pandemic: The role of academic workload, separation from school, and fears of contagion. PloS One, 16(2), e0246676. https://doi.org/10.1371/journal.pone.0246676
Yangdon, K., Sherab, K., Choezom, P., Passang, S., & Deki, S. (2021). Well-being and academic workload: Perceptions of science and technology students. Educational Research and Reviews, 16(11), 418–427. https://doi.org/10.5897/ERR2021.4197
Zar, J. H. (2005). Spearman rank correlation. In P. Armitage & T. Colton (Eds.), Encyclopedia of biostatistics (Vol. 7). Wiley Online Library. https://doi.org/10.1002/0470011815.b2a15150
Zijlstra, W. P., van der Ark, L. A., & Sijtsma, K. (2011). Outliers in questionnaire data: Can they be detected and should they be removed? Journal of Educational and Behavioral Statistics, 36(2), 186–212. https://doi.org/10.3102/1076998610366263
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution 4.0 International License.
