Temporal Network Analysis of Collaborative Prototyping from a Knowledge Creation Perspective
DOI:
https://doi.org/10.18608/jla.2026.8959Keywords:
knowledge creation, prototyping, temporal socio-semantic network analysis, ordered network analysis, collaborative problem-solving, research paperAbstract
This study analyzed data from prototyping sessions to support learners’ engagement in collaborative problem-solving through design-mode thinking. Knowledge creation (KC) is essential for addressing 21st-century complex problems, yet learners struggle to share incomplete ideas due to unfamiliarity with design-mode thinking. Although prototyping is known to be effective for sharing and improving incomplete ideas in the business sector, few studies analyze design activities during prototyping from a KC perspective across different design disciplines. This study examined three teams (engineering, product design, service design) during 30-minute prototyping sessions. Using a novel combination of techniques—temporal socio-semantic network analysis (tSSNA) and ordered network analysis (ONA)—we analyzed teams’ shared epistemic agency, segmenting activities into three phases based on idea improvement patterns. Engineers focused on generative collaborative actions, product designers emphasized creating shared understanding, and service designers concentrated on alleviating lack of knowledge. Statistical tests revealed significant differences between teams and phases. The findings suggest three design principles for KC practice: attending to disciplinary differences in interdisciplinary teams, providing timely educator support for concept creation, and using short task durations to encourage sharing incomplete ideas. Furthermore, this work demonstrates the potential of combining tSSNA and ONA for analyzing collaborative KC processes.
References
Aviña, G. E., Schunn, C. D., Silva, A. R., Bauer, T. L., Crabtree, G. W., Johnson, C. M., Odumosu, T., Picraux, S. T., Sawyer, R. K., Schneider, R. P., Sun, R., Feist, G. J., Narayanamurti, V., & Tsao, J. Y. (2018). The art of research: A divergent/convergent thinking framework and opportunities for science-based approaches. In E. Subrahmanian, T. Odumosu, & J. Y. Tsao (Eds.), Engineering a better future: Interplay between engineering, social sciences, and innovation (pp. 167–186). Springer. https://doi.org/10.1007/978-3-319-91134-2_14
Barany, A., Philips, M., Kawakubo, A. J. T., & Oshima, J. (2022). Choosing units of analysis in temporal discourse. In B. Wasson & S. Zörg˝o (Eds.), Advances in quantitative ethnography. ICQE 2021. Communications in computer and information science (pp. 80–94, Vol. 1522). Springer. https://doi.org/10.1007/978-3-030-93859-8_6
Bereiter, C., & Scardamalia, M. (2003). Learning to work creatively with knowledge. In E. De Corte, L. Verschaffel, N. Entwistle, & J. van Merriënboer (Eds.), Powerful learning environments: Unravelling basic components and dimensions (pp. 55–68). European Association for Research on Learning and Instruction.
Bowman, D., Swiecki, Z., Cai, Z., Wang, Y., Eagan, B., Linderoth, J., & Shaffer, D. W. (2021). The mathematical foundations of epistemic network analysis. In A. R. Ruis & S. B. Lee (Eds.), Advances in quantitative ethnography. ICQE 2021. Communications in computer and information science (pp. 91–105, Vol. 1312). Springer. https://doi.org/10.1007/978-3-030-67788-6_7
Camburn, B., Viswanathan, V., Linsey, J., Anderson, D., Jensen, D., Crawford, R., Otto, K., & Wood, K. (2017). Design prototyping methods: State of the art in strategies, techniques, and guidelines. Design Science, 3, e13. https://doi.org/10.1017/dsj.2017.10
Chan, C. K., & van Aalst, J. (2018). Knowledge building: Theory, design, and analysis. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, & P. Reimann (Eds.), International handbook of the learning sciences (pp. 295–307). Routledge. https://www.taylorfrancis.com/chapters/edit/10.4324/9781315617572-29/knowledge-building-carol-chan-jan-van-aalst
Chen, B., Scardamalia, M., & Bereiter, C. (2015). Advancing knowledge-building discourse through judgments of promising ideas. International Journal of Computer-Supported Collaborative Learning, 10, 345–366. https://doi.org/10.1007/s11412-015-9225-z
Chen, B., & Zhang, J. (2016). Analytics for knowledge creation: Towards epistemic agency and design-mode thinking. Journal of Learning Analytics, 3(2), 139–163. https://doi.org/10.18608/jla.2016.32.7
Christie, E., Jensen, D., Buckley, R., Menefee, D., Ziegler, K., Wood, K., & Crawford, R. (2012). Prototyping strategies: Literature review and identification of critical variables. In Proceedings of the 2012 ASEE Annual Conference and Exposition (ASEE 2012), 10–13 June 2012, San Antonio, Texas, USA (pp. 25.1091.1–25.1091.22). ASEE PEER. https://doi.org/10.18260/1-2--21848
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Routledge. https://doi.org/10.4324/9780203771587
Csanadi, A., Eagan, B., Kollar, I., Shaffer, D., & Fischer, F. (2018). When coding-and-counting is not enough: Using epistemic network analysis (ENA) to analyze verbal data in CSCL research. International Journal of Computer-Supported Collaborative Learning, 13, 419–438. https://doi.org/10.1007/s11412-018-9292-z
Damsa, C. I., Kirschner, P. A., Andriessen, J. E., Erkens, G., & Sins, P. H. (2010). Shared epistemic agency: An empirical study of an emergent construct. The Journal of the Learning Sciences, 19(2), 143–186. https://doi.org/10.1080/10508401003708381
Frambach, J. M., Driessen, E. W., Beh, P., & Van der Vleuten, C. P. (2014). Quiet or questioning? Students’ discussion behaviors in student-centered education across cultures. Studies in Higher Education, 39(6), 1001–1021. https://doi.org/10.1080/03075079.2012.754865
Gerber, E., & Carroll, M. (2012). The psychological experience of prototyping. Design Studies, 33(1), 64–84. https://doi.org/10.1016/j.destud.2011.06.005
Goldberg, J. R., & Malassigné, P. (2017). Lessons learned from a 10-year collaboration between biomedical engineering and industrial design students in capstone design projects. The International Journal of Engineering Education, 33(5),1513–1520. https://www.ijee.ie/contents/c330517.html
Goldschmidt, G. (2016). Linkographic evidence for concurrent divergent and convergent thinking in creative design. Creativity Research Journal, 28(2), 115–122. https://doi.org/10.1080/10400419.2016.1162497
Gordon, V., & Bieman, J. (1995). Rapid prototyping: Lessons learned. IEEE Software, 12(1), 85–95. https://doi.org/10.1109/52.363162
Herrenkohl, L. R., & Cornelius, L. (2013). Investigating elementary students’ scientific and historical argumentation. Journal of the Learning Sciences, 22(3), 413–461. https://doi.org/10.1080/10508406.2013.799475
Holme, P., & Saramäki, J. (2012). Temporal networks. Physics Reports, 519(3), 97–125. https://doi.org/10.1016/j.physrep.2012.03.001
IDEO-U. (2016). Prototyping Elmo’s Monster Maker app. https://www.ideou.com/blogs/inspiration/prototyping-elmo-s-monster-maker-app
Janssen, J., Erkens, G., Kirschner, P. A., & Kanselaar, G. (2011). Multilevel analysis in CSCL research. In S. Puntambekar, G. Erkens, & C. Hmelo-Silver (Eds.), Analyzing interactions in CSCL. Computer-supported collaborative learning series (pp. 187–205, Vol. 12). Springer. https://doi.org/10.1007/978-1-4419-7710-6_9
Koh, J. H. L., Chai, C. S., Wong, B., & Hong, H. - Y. (2015). Design thinking and education. In Design thinking for education: Conceptions and applications in teaching and learning. Springer. https://doi.org/10.1007/978-981-287-444-3_1
Lenth, R. V., Bolker, B., Buerkner, P., Giné-Vázquez, I., Herve, M., Jung, M., Love, J., Miguez, F., Piaskowski, J., Riebl, H., & Singmann, H. (2023). emmeans. https://doi.org/10.32614/CRAN.package.emmeans
Lim, Y.-K., Stolterman, E., & Tenenberg, J. (2008). The anatomy of prototypes: Prototypes as filters, prototypes as manifestations of design ideas. ACM Transactions on Computer-Human Interaction, 15(2). https://doi.org/10.1145/1375761.1375762
Mager, B., Sistig, M., Chen, Y., Ruiz, K., & Corona, C. (2020). The future of service design. https://www.academia.edu/44459133/The%5Cstring_Future%5Cstring_of%5Cstring_Service%5Cstring_Design
Matsuoka, Y. (2005). Two types of design. Journal of Japan Society of Mechanical Engineers, 108(1034), 14–17. https://doi.org/10.1299/jsmemag.108.1034_14
McNeish, D., & Kelley, K. (2019). Fixed effects models versus mixed effects models for clustered data: Reviewing the approaches, disentangling the differences, and making recommendations. Psychological Methods, 24(1), 20–35. https://doi.org/10.1037/met0000182
Nonaka, I., & Takeuchi, H. (1995). The knowledge-creating company: How Japanese companies create the dynamics of innovation. Oxford University Press. https://doi.org/10.1093/oso/9780195092691.001.0001
OECD. (2019). An OECD learning framework 2030. In G. Bast, E. G. Carayannis, & D. F. J. Campbell (Eds.), The future of education and labor (pp. 23–35). Springer. https://doi.org/10.1007/978-3-030-26068-2_3
Ohsaki, A., & Oshima, J. (2019). A socio-semantic network analysis of discourse using the network lifetime and the moving stanza window method. In B. Eagan, M. Misfeldt, & A. Siebert-Evenstone (Eds.), Advances in quantitative ethnography. ICQE 2019. Communications in computer and information science (pp. 326–333, Vol. 1112). Springer. https://doi.org/10.1007/978-3-030-33232-7_29
Ohsaki, A., & Oshima, J. (2021). Socio-semantic network analysis of knowledge-creation discourse on a real-time scale. In A. R. Ruis & S. B. Lee (Eds.), Advances in quantitative ethnography. ICQE 2021. Communications in computer and information science (pp. 170–184, Vol. 1312). Springer. https://doi.org/10.1007/978-3-030-67788-6_12
Ohsaki, A., & Oshima, J. (2023). Analyzing the co-design process by engineers and product designers from perspectives of knowledge building. In C. Damsa & A. Barany (Eds.), Advances in quantitative ethnography. ICQE 2022. Communications in computer and information science (pp. 388–401, Vol. 1785). Springer. https://doi.org/10.1007/978-3-031-31726-2_27
Ohtomi, K. (2009). Future direction of design engineering: from SEKKEI to DESIGN. Transactions of the Japan Society of Mechanical Engineers, Series C, 75(751), 516–523. https://doi.org/10.1299/kikaic.75.516
Oshima, J., Oshima, R., & Matsuzawa, Y. (2012). Knowledge building discourse explorer: A social network analysis application for knowledge building discourse. Educational Technology Research and Development, 60, 903–921. https://doi.org/10.1007/s11423-012-9265-2
Oshima, J., Oshima, R., & Saruwatari, S. (2020). Analysis of students’ ideas and conceptual artifacts in knowledge-building discourse. British Journal of Educational Technology, 51(4), 1308–1321. https://doi.org/10.1111/bjet.12961
Razzouk, R., & Shute, V. (2012). What is design thinking and why is it important? Review of Educational Research, 82(3), 330–348. https://doi.org/10.3102/0034654312457429
Ruis, A., Siebert-Evenstone, A., Pozen, R., Eagan, B., & Shaffer, D. (2019). Finding common ground: A method for measuring recent temporal context in analyses of complex, collaborative thinking. In K. Lund, G. P. Niccolai, E. Lavoué, C. Hmelo-Silver, G. Gweon, & M. Baker (Eds.), A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning in Collaborative Settings, Proceedings of the 13th International Conference on Computer Supported Collaborative Learning (CSCL 2019), 17–21 June 2019, Lyon, France (pp. 136–143, Vol. 1). International Society of the Learning Sciences. https://repository.isls.org//handle/1/4395
Scardamalia, M. (2002). Collective cognitive responsibility for the advancement of knowledge. In B. Smith (Ed.), Liberal education in a knowledge society (pp. 67–98). Open Court.
Scardamalia, M., & Bereiter, C. (2014). Knowledge building and knowledge creation: Theory, pedagogy, and technology. In K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd ed., pp. 397–413). Cambridge University Press. https://doi.org/10.1017/CBO9781139519526.025
Scardamalia, M., & Bereiter, C. (2021). Knowledge building: Advancing the state of community knowledge. In K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (3rd ed., pp. 261–279). Cambridge University Press. https://doi.org/10.1007/978-3-030-65291-3_14
Schrage, M. (1996). Cultures of prototyping. In T. Winograd (Ed.), Bringing design to software (pp. 191–213). Association for Computing Machinery. https://doi.org/10.1145/229868.230045
Shaffer, D. W. (2017). Quantitative ethnography. Cathcart Press.
Siebert-Evenstone, A. L., Irgens, G. A., Collier, W., Swiecki, Z., Ruis, A. R., & Shaffer, D. W. (2017). In search of conversational grain size: Modeling semantic structure using moving stanza windows. Journal of Learning Analytics, 4(3), 123–139. https://doi.org/10.18608/jla.2017.43.7
Snijders, T. A., & Bosker, R. J. (2011). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). Sage Publishers. https://www.stats.ox.ac.uk/~snijders/mlbook.htm
Stickdorn, M., Hormess, M. E., Lawrence, A., & Schneider, J. (2018). This is service design doing: Applying service design thinking in the real world. O’Reilly Media, Inc. https://www.thisisservicedesigndoing.com/
Tan, Y., Ruis, A. R., Marquart, C., Cai, Z., Knowles, M. A., & Shaffer, D. W. (2022). Ordered network analysis. In C. Damsa & A. Barany (Eds.), Advances in quantitative ethnography. ICQE 2022. Communications in computer and information science (pp. 101–116, Vol. 1785). https://doi.org/10.1007/978-3-031-31726-2_8
Treffinger, D. J. (1995). Creative problem solving: Overview and educational implications. Educational Psychology Review, 7, 301–312. https://doi.org/10.1007/BF02213375
Viswanathan, V., & Linsey, J. (2011). Understanding physical models in design cognition: A triangulation of qualitative and laboratory studies. In Proceedings of the 2011 Frontiers in Education Conference (FIE 2011), 12–15 October 2011, Rapid City, South Dakota, USA (pp. S3F-1–S3F-6). IEEE. https://doi.org/10.1109/FIE.2011.6142848
Weber, N. F. (2019). The Bauhaus at 100: Science by design. Nature, 579, 174–175. https://doi.org/10.1038/d41586-019-02355-4
Yu, F., Pasinelli, M., & Brem, A. (2018). Prototyping in theory and in practice: A study of the similarities and differences between engineers and designers. Creativity and Innovation Management, 27(2), 121–132. https://doi.org/10.1111/caim.12242
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution 4.0 International License.
