Constraint-Referenced Analytics of Algebra Learning


  • Scot McRobert Sutherland UC Davis School of Education University of California, Davis
  • Tobin F White UC Davis School of Education University of California, Davis



learning analytics, measurement, psychometrics, assessment, algebra learning, procedural knowledge, conceptual knowledge, collaborative networks, classroom networks


The development of the constraint-referenced analytics tool for monitoring algebra learning activities presented here came from the desire to firstly, take a more quantitative look at student responses in collaborative algebra activities, and secondly, to situate those activities in a more traditional introductory algebra setting focusing on procedural understanding. Procedural skill was analyzed by modeling the complexity of attempts to make equivalent transformations of algebraic expressions. The constraint-referenced analytics system uses log files of student inputs on a classroom network of handheld devices to measure success rate as students make attempts to replace one algebraic expression with another equivalent expression.  The analytics engine produced psychometrically verifiable results. Moving averages of student performance revealed that when students experienced a period of struggle and persisted in attempting similar transformations, an apparent conceptual shift led to subsequent success. Students also responded to periods of struggle by switching to familiar tasks or choosing non-participation.


Ambrose, R, Baek, J., & Carpenter, T. (2013). Children’s Invention Of Multidigit Multiplication And Division Algorithms. In A. J. Baroody & A. Dowker (Eds.), The Development of Arithmetic Concepts and Skills: Constructive Adaptive Expertise (p. 307).

Baghaei, N., Mitrović, A., & Irwin, W. (2007). Supporting collaborative learning and problem-solving in a constraint-based CSCL environment for UML class diagrams. International Journal of Computer-Supported Collaborative Learning, 2(2), 159-190.

Ball, L., Pierce, R., & Stacey, K. (2003, International Group for the Psychology of Mathematics Education. 35 Aandwind Street; Kirstenhof, Cape Town, 7945, South Africa). Recognising Equivalent Algebraic Expressions: An Important Component of Algebraic Expectation for Working with CAS. Proceedings from Psychology of Mathematics Education Annual Conference Proceedings, Honolulu, Hawaii.

Booth, L. R. (1988). Children’s Difficulties in Beginning Algebra. In A. F. Coxford & A. P. Shulte (Eds.), The ideas of algebra, K-12. 1988 Yearbook (pp. 299-306). Reston, VA: National Council of Teachers of Mathematics.

Chase, W. G., & Simon, H. A. (1973). Perception in chess. Cognitive Psychology, 4(1), 55-81.

Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. [Pittsburgh, Pa.]: Learning Research and Development Center, University of Pittsburgh.

Choppin, J. M., Clancy, C. B., & Koch, S. J. (2012). Developing Formal Procedures through Sense Making. [ArticleType: research-article / Full publication date: May 2012 / Copyright © 2012 National Council of Teachers of Mathematics]. Mathematics Teaching in the Middle School, 17(9), 552-557. Retrieved from

Crooks, N. M., & Alibali, M. W. (2014). Defining and measuring conceptual knowledge in mathematics. Developmental Review, 34(4), 344-377. Retrieved from

Denmark, T., & et al. (1976). Final Report: A Teaching Experiment on Equality. PMDC Technical Report No. 6., 158.

Filloy, E., & Rojano, T. (1989). Solving equations: The transition from arithmetic to algebra. For the Learning of Mathematics, 9(2), 19-25.

Gattegno, C. (1974). The common sense of teaching mathematics. [New York: Educational Solutions.

Henningsen, M., & Stein, M. K. (1997). Mathematical Tasks and Student Cognition: Classroom-Based Factors That Support and Inhibit High-Level Mathematical Thinking and Reasoning. Journal for Research in Mathematics Education, 28(5), 524-549.

Hiebert, J. (2013). Conceptual and procedural knowledge: The case of mathematics. Routledge.

Jones, I., Inglis, M., Gilmore, C., & Dowens, M. (2012). Substitution and sameness: Two components of a relational conception of the equals sign. Journal of experimental child psychology, 113(1), 166-176. Retrieved from

Jones, I., Inglis, M., Gilmore, C., & Evans, R. (2013). Teaching the substitutive conception of the equals sign. Research in Mathematics Education, 15(1), 34-49. Retrieved from

Kapur, M. (2011). A further study of productive failure in mathematical problem solving: Unpacking the design components. Instructional Science, 39(4), 561-579. Retrieved from

Kapur, M. (2016). Examining Productive Failure, Productive Success, Unproductive Failure, and Unproductive Success in Learning. Educational Psychologist, 51(2), 289-299. Retrieved from

Kieran, C. (2004). The core of algebra: Reflections on its main activities. In The Future of the Teaching and Learning of Algebra The 12 th ICMI Study (pp. 21-33).

Kieran, C. (1981). Concepts Associated with the Equality Symbol. Educational Studies in Mathematics, 12(3), 317-326. Retrieved from

Kirshner, D. (1993). The Structural Algebra Option: A Discussion Paper., 19.

Lai, K., & White, T. (2012). Exploring quadrilaterals in a small group computing environment. Computers & Education, 59(3), 963-973. Retrieved from

Lai, K., & White, T. (2014). How groups cooperate in a networked geometry learning environment. Instructional Science, 42(4), 615-637. Retrieved from

Linchevski, L. (1995). Algebra with numbers and arithmetic with letters: A definition of pre-algebra. The Journal of Mathematical Behavior, 14(1), 113-120.

Linchevski, L., & Herscovics, N. (1996). Crossing the Cognitive Gap between Arithmetic and Algebra: Operating on the Unknown in the Context of Equations. Educational Studies in Mathematics, 30(1), 39-65.

Linchevski, L., & Livneh, D. (1999). Structure Sense: The Relationship between Algebraic and Numerical Contexts. Educational Studies in Mathematics, 40(2), 173-196.

McNeil, N. M., & Alibali, M. W. (2005). Why Won’t You Change Your Mind? Knowledge of Operational Patterns Hinders Learning and Performance on Equations. Child Development, 76(4), 883-899. doi:10.2307/3696735

Mitrović, A. (2012a). Fifteen years of Constraint-Based Tutors: What we have achieved and where we are going. User modeling and user-adapted interaction, 1-34.

Mitrović, A. (2012b). Fifteen years of Constraint-Based Tutors: What we have achieved and where we are going. User modeling and user-adapted interaction, 1-34.

Mitrović, A., Koedinger, K., & Martin, B. (2003). A comparative analysis of cognitive tutoring and constraint-based modeling. User Modeling 2003, 147-147.

Mitrović, A., Mayo, M., Suraweera, P., & Martin, B. (2001). Constraint-based tutors: a success story. Engineering of Intelligent Systems, 931-940.

Mitrović, A., & Weerasinghe, A. (2009). Revisiting ill-definedness and the consequences for ITSs. In V. Dimitrova, R. Mizoguchi, B. du Boulay, & A. Graesser (Eds.), Artificial Intelligence in Education: Building Learning Systems that Care: From Knowledge Representation to Affective Modelling (pp. 375-382). Amsterdam, Netherlands: IOS Press BV.

Renninger, A., Hidi, S., & Krapp, A. (2014). The role of interest in learning and development. Psychology Press.

Rittle-Johnson, B., Siegler, R. S., & Alibali, M. W. (2001). Developing conceptual understanding and procedural skill in mathematics: An iterative process. Journal of educational psychology, 93(2), 346. Retrieved from

Rossi, P. S. (2008). An Uncommon Approach to a Common Algebraic Error. Primus, 18(6), 554-558. Retrieved from

Saldanha, L., & Kieran, C. (2005). A slippery slope betwen equivalence and equality: Exploring students’ reasoning in the context of algebra instruction involving a computer algebra system. Proceedings from 27th Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education, Roanoke, VA.

Sfard, A. (1991). On the Dual Nature of Mathematical Conceptions: Reflections on Processes and Objects as Different Sides of the Same Coin. Educational Studies in Mathematics, 22(1), 1-36.

Sfard, A., & Linchevski, L. (1994). The Gains and the Pitfalls of Reification--The Case of Algebra. Educational Studies in Mathematics, 26(2-3), 191-228.

Wagner, S., & Parker, S. (1993). Advancing algebra. Research ideas for the classroom: High school mathematics, 119-139.

Warren, E. (2003). The role of arithmetic structure in the transition from arithmetic to algebra. Mathematics Education Research Journal, 15(2), 122-137.

White, T. (2006). Code Talk: Student Discourse and Participation with Networked Handhelds. International Journal of Computer-Supported Collaborative Learning, 1(3), 359-382. Retrieved from

White, T. (2009). Encrypted objects and decryption processes: problem-solving with functions in a learning environment based on cryptography. Educational Studies in Mathematics, 72(1), 17-37. Retrieved from

White, T. (2013). Networked technologies for fostering novel forms of student interaction in high school mathematics classrooms. In Emerging technologies for the classroom (pp. 81-92). Springer. Retrieved from[Tobin White, 2013] Networked Technologies for Fostering Novel Forms of Student Interaction in High School Mathematics Classrooms.pdf

White, T., & Lai, K. (2008). Collaborative Problem Solving with Linked Representational Tools. Retrieved from

White, T., & Pea, R. (2011). Distributed by design: On the promises and pitfalls of collaborative learning with multiple representations. Journal of the Learning Sciences, 20(3), 489-547. Retrieved from

White, T., Sutherland, S. M., & Lai, K. (2010). Constructing Collective Algebraic Objects in a Classroom Network. In P. Brosnan, D. B. Erchick, & L. Flevares (Eds.), Proceedings of the 32nd annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (pp. 1523-1530). Columbus, OH: The Ohio State University.

White, T., Wallace, M., & Lai, K. (2012). Graphing in groups: Learning about lines in a collaborative classroom network environment. Mathematical Thinking and Learning, 14(2), 149-172. Retrieved from

Wilensky, U. (1999). NetLogo. Retrieved from

Wilensky, U., & Stroup, W. (1999). HubNet. Center for Connected Learning and Computer-Based Modeling, Northwestern University: Evanston, IL USA,< http://ccl. northwestern. edu/ps.




How to Cite

Sutherland, S. M., & White, T. F. (2016). Constraint-Referenced Analytics of Algebra Learning. Journal of Learning Analytics, 3(3), 143-169.