Disentangling the Influences of Creativity and Verbal Fluency on Writing
DOI:
https://doi.org/10.18608/jla.2025.8857Keywords:
writing, creativity, natural language processing, verbal fluency, research paperAbstract
This study investigates how individual differences related to creative and verbal fluencies can be differentiated and modelled through the linguistic features that manifest in student writing. Using computational linguistics, we analyzed samples of intrapersonal writing to extract linguistic features that align to specific dimensions of language. Creative fluency scores were calculated based on the number of unique ideas generated during the Alternate Uses Task (AUT); verbal fluency scores were calculated based on the number of unique semantic categories produced during the animal naming task. We then developed machine learning models to predict creative and verbal fluency scores based on the linguistic features of participant essays. Results indicate that creative fluency is more strongly predicted by linguistic features (particularly descriptive and cohesion indices) than verbal fluency. Key predictors of creative fluency include features that capture lexical diversity and the global connectedness of ideas. Importantly, our results align with theoretical frameworks related to convergent and divergent thinking. Findings highlight the potential for leveraging learning analytics to offer new insights into complex cognitive processes such as creativity. Implications for stealth assessments and personalized feedback within automated writing systems are discussed as paths for future research.
References
Allen, L. K., Likens, A. D., & McNamara, D. S. (2018). A multi-dimensional analysis of writing flexibility in an automated writing evaluation system. In A. Pardo, K. Bartimote-Aufflick, G. Lynch, S. Buckingham Shum, R. Ferguson, A. Merceron, & X. Ochoa (Eds.), LAK ’18: Proceedings of the 8th international conference on learning analytics & knowledge (pp. 380–388). ACM Press. https://doi.org/10.1145/3170358.3170404
Allen, L. K., & McNamara, D. S. (2015). You are your words: Modeling students’ vocabulary knowledge with natural language processing. In O. C. Santos, J. G. Boticario, C. Romero, M. Pechenizkiy, A. Merceron, P. Mitros, J. M. Luna, C. Mihaescu, P. Moreno, A. Hershkovitz, S. Ventura, M. Desmarais (Eds.), Proceedings of the 8th international conference on educational data mining (pp. 258–265). International Educational Data Mining Society. https://www.educationaldatamining.org/EDM2015/proceedings/edm2015_proceedings.pdf
Batey, M. (2012). The measurement of creativity: From definitional consensus to the introduction of a new heuristic framework. Creativity Research Journal, 24(1), 55–65. https://doi.org/10.1080/10400419.2012.649181
Beaty, R. E., Kenett, Y. N., Christensen, A. P., Rosenberg, M. D., Benedek, M., Chen, Q., Fink, A., Qiu, J., Kwapil, T. R., Kane, M. J., & Silvia, P. J. (2018). Robust prediction of individual creative ability from brain functional connectivity. Proceedings of the National Academy of Sciences, 115(5), 1087–1092. https://doi.org/10.1073/pnas.1713532115
Bereiter, C., & Scardamalia, M. (Eds.). (1987). The psychology of written composition. Routledge. https://doi.org/10.4324/9780203812310
Bower, J., Acar, S., & Kursuncu, U. (2023). Measuring creativity in academic writing: An analysis of essays in advanced placement language and composition. Journal of Advanced Academics, 34(3–4), 183–214. https://doi.org/10.1177/1932202X231211633
Butterfuss, R., Roscoe, R. D., Allen, L. K., McCarthy, K. S., & McNamara, D. S. (2022). Strategy uptake in Writing Pal: Adaptive feedback and instruction. Journal of Educational Computing Research, 60(3), 696–721. https://doi.org/10.1177/07356331211045304
Buczak, P., Huang, H., Forthmann, B., & Doebler, P. (2023). The machines take over: A comparison of various supervised learning approaches for automated scoring of divergent thinking tasks. The Journal of Creative Behavior, 57(1), 17–36. https://doi.org/10.1002/jocb.559
Carey, L. J., & Flower, L. (1989). Foundations for creativity in the writing process: Rhetorical representations of ill-defined options. In J. A. Glover, R. R. Ronning, & C. R. Reynolds (Eds.), Handbook of creativity (pp. 283–303). Springer New York. https://doi.org/10.1007/978-1-4757-5356-1_17
Collins, A. M., & Loftus, E. F. (1975). A spreading-activation theory of semantic processing. Psychological Review, 82(6), 407–428. https://doi.org/10.1037/0033-295X.82.6.407
Craik, F. I. M., Govoni, R., Naveh-Benjamin, M., & Anderson, N. D. (1996). The effects of divided attention on encoding and retrieval processes in human memory. Journal of Experimental Psychology: General, 125(2), 159–180. https://doi.org/10.1037/0096-3445.125.2.159
Crossley, S. A., Varner, L. K., Roscoe, R. D., & McNamara, D. S. (2013). Using automated indices of cohesion to evaluate an intelligent tutoring system and an automated writing evaluation system. In H. C. Lane, K. Yacef, J. Mostow, & P. Pavlik (Eds.), Artificial intelligence in education: 16th international conference, AIED 2013, Memphis, TN, USA, July 9–13, 2013, proceedings (pp. 269–278). Springer. https://doi.org/10.1007/978-3-642-39112-5_28
Crossley, S. A., Allen, L. K., & McNamara, D. S. (2014). A multi-dimensional analysis of essay writing: What linguistic features tell us about situational parameters and the effects of language functions on judgments of quality. In T. Berber Sardinha & M. Veirano Pinto (Eds.), Multi-dimensional analysis, 25 years on: A tribute to Douglas Biber (pp. 197–238). John Benjamins Publishing Company. https://doi.org/10.1075/scl.60.07cro
Crossley, S. A., Allen, L. K., Snow, E. L., & McNamara, D. S. (2016). Incorporating learning characteristics into automatic essay scoring models: What individual differences and linguistic features tell us about writing quality. Journal of Educational Data Mining, 8(2), 1–19. https://doi.org/10.5281/ZENODO.3554593
Crossley, S. A., Allen, L. K., & McNamara, D. S. (2016). The writing pal: A writing strategy tutor. In S. A. Crossley & D. S. McNamara (Eds.), Adaptive educational technologies for literacy instruction (pp. 204–224). Routledge. https://doi.org/10.4324/9781315647500
Davis, G. A. (2004). Creativity is forever (5th ed.). Kendall Hunt Publishing.
Deane, P., & Zhang, M. (2020). Automated writing process analysis. In D. Yan, A. A. Rupp, & P. W. Foltz (Eds.), Handbook of automated scoring: Theory into practice. Chapman and Hall/CRC. https://doi.org/10.1201/9781351264808
Dumas, D., Organisciak, P., & Doherty, M. (2021). Measuring divergent thinking originality with human raters and text-mining models: A psychometric comparison of methods. Psychology of Aesthetics, Creativity, and the Arts, 15(4), 645–663. https://doi.org/10.1037/aca0000319
Dumas, D., & Strickland, A. (2018). From book to bludgeon: A closer look at unsolicited malevolent responses on the alternate uses task. Creativity Research Journal, 30(4), 439–450.
Dygert, S. K. C., & Jarosz, A. F. (2020). Individual differences in creative cognition. Journal of Experimental Psychology: General, 149(7), 1249–1274. https://doi.org/10.1037/xge0000713
Ericsson, K. A., & Simon, H. A. (1998). How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking. Mind, Culture, and Activity, 5(3), 178–186. https://doi.org/10.1207/s15327884mca0503_3
Fang, Y., Roscoe, R. D., & McNamara, D. S. (2023). Artificial intelligence-based assessment in education. In B. du Boulay, A. Mitrovic, & K. Yacef (Eds.), Handbook of artificial intelligence in education (pp. 485–504). Edward Elgar Publishing. https://www.elgaronline.com/edcollchap/book/9781800375413/book-part-9781800375413-33.xml
Fredericks, A. D. (2005). The complete idiot’s guide to success as a teacher. Alpha.
Gamer, M., Lemon, J., & Singh, I. F. P. (2019). irr: Various coefficients of interrater reliability and agreement (Version 0.84.1) [Computer software]. https://cran.r-project.org/web/packages/irr/index.html
Gerver, C. R., Griffin, J. W., Dennis, N. A., & Beaty, R. E. (2023). Memory and creativity: A meta-analytic examination of the relationship between memory systems and creative cognition. Psychonomic Bulletin & Review, 30(6), 2116–2154. https://doi.org/10.3758/s13423-023-02303-4
Grey, S., & Morris, P. (2024). Capturing the spark: PISA, twenty-first century skills and the reconstruction of creativity. Globalisation, Societies and Education, 22(2), 156–171. https://doi.org/10.1080/14767724.2022.2100981
Guilford, J. P. (1950). Fundamental statistics in psychology and education (2nd ed.). McGraw-Hill.
Guilford, J. P. (1967). The nature of human intelligence. McGraw-Hill.
Guilford, J. P., Christensen, P. R., Merrifield, P. R., & Wilson, R. C. (1960). Manual: Alternate uses. Sheridan Supply Co. https://www.mindgarden.com/alternate-uses/78-altu-manual.html#horizontalTab1
Harrington, D. M. (1975). Effects of explicit instructions to “be creative” on the psychological meaning of divergent thinking test scores. Journal of Personality, 43(3), 434–454. https://doi.org/10.1111/j.1467-6494.1975.tb00715.x
Hudson, L. (2019). Frames of mind: Ability, perception and self-perception in the arts and sciences. Routledge.
Jauk, E., Benedek, M., & Neubauer, A. C. (2014). The road to creative achievement: A latent variable model of ability and personality predictors. European Journal of Personality, 28(1), 95–105. https://doi.org/10.1002/per.1941
Kounios, J., Osman, A. M., & Meyer, D. E. (1987). Structure and process in semantic memory: New evidence based on speed–accuracy decomposition. Journal of Experimental Psychology: General, 116(1), 3–25. https://doi.org/10.1037/0096-3445.116.1.3
Kuhn, M. (2008). Building predictive models in R using the caret package. Journal of Statistical Software, 28(5), 1–26. https://doi.org/10.18637/jss.v028.i05
Marek, R. J., & Ben-Porath, Y. S. (2017). Using the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF) in behavioral medicine settings. In M. E. Maruish (Ed.), Handbook of psychological assessment in primary care settings (2nd ed., pp. 631–662). Routledge. https://doi.org/10.4324/9781315658407
McCutchen, D., Covill, A., Hoyne, S. H., & Mildes, K. (1994). Individual differences in writing: Implications of translating fluency. Journal of Educational Psychology, 86(2), 256–266. https://doi.org/10.1037/0022-0663.86.2.256
McNamara, D. S., Allen, L. K., & Crossley, S. (2019). WAT: Writing assessment tool. In S. Hsiao, J. Cunningham, K. S. McCarthy, G. Lynch, N. Hoover, C. Brooks, R. Ferguson, & U. Hoppe (Eds.), Companion proceedings of the 9th international conference on learning analytics and knowledge conference (LAK ’19) (p. 256). SoLAR.
Mednick, S. (1962). The associative basis of the creative process. Psychological Review, 69(3), 220–232. https://doi.org/10.1037/h0048850
Metcalfe, J., & Wiebe, D. (1987). Intuition in insight and noninsight problem solving. Memory & Cognition, 15(3), 238–246. https://doi.org/10.3758/BF03197722
Munezero, M., Montero, C. S., Mozgovoy, M., & Sutinen, E. (2013). Exploiting sentiment analysis to track emotions in students’ learning diaries. Proceedings of the 13th Koli Calling International Conference on Computing Education Research, 145–152. https://doi.org/10.1145/2526968.2526984
Newell, A., & Simon, H. A. (1972). Human problem solving. Prentice-Hall.
OECD. (2024). PISA 2022 results: Creative minds, creative schools (Vol. III). OECD Publishing. https://doi.org/10.1787/765ee8c2-en
Öncel, P., Flynn, L. E., Sonia, A. N., Barker, K. E., Lindsay, G. C., McClure, C. M., McNamara, D. S., & Allen, L. K. (2021). Automatic student writing evaluation: Investigating the impact of individual differences on source-based writing. In M. Scheffel, N. Dowell, S. Joksimović, & G. Siemens (Eds.), LAK21: 11th international learning analytics and knowledge conference (pp. 620–625). ACM Press. https://doi.org/10.1145/3448139.3448207
Organisciak, P., Acar, S., Dumas, D., & Berthiaume, K. (2023). Beyond semantic distance: Automated scoring of divergent thinking greatly improves with large language models. Thinking Skills and Creativity, 49, Article 101356. https://doi.org/10.1016/j.tsc.2023.101356
Ovington, L. A., Saliba, A. J., Moran, C. C., Goldring, J., & MacDonald, J. B. (2018). Do people really have insights in the shower? The when, where and who of the aha! moment. The Journal of Creative Behavior, 52(1), 21–34. https://doi.org/10.1002/jocb.126
Pei, Z., Zheng, C., Zhang, M., & Liu, F. (2017). Critical thinking and argumentative writing: Inspecting the association among EFL learners in China. English Language Teaching, 10(10), 31–42. https://doi.org/10.5539/elt.v10n10p31
Peirce, J., Gray, J. R., Simpson, S., MacAskill, M., Höchenberger, R., Sogo, H., Kastman, E., & Lindeløv, J. K. (2019). PsychoPy2: Experiments in behavior made easy. Behavior Research Methods, 51(1), 195–203. https://doi.org/10.3758/s13428-018-01193-y
Plucker, J. A., & Makel, M. C. (2010). Assessment of creativity. In J. C. Kaufman & R. J. Sternberg (Eds.), The Cambridge handbook of creativity (pp. 48–73). Cambridge University Press. https://doi.org/10.1017/CBO9780511763205.005
Puryear, J. S., Kettler, T., & Rinn, A. N. (2017). Relationships of personality to differential conceptions of creativity: A systematic review. Psychology of Aesthetics, Creativity, and the Arts, 11(1), 59–68. https://doi.org/10.1037/aca0000079
Richards, R., Kinney, D. K., Benet, M., & Merzel, A. P. (1988). Assessing everyday creativity: Characteristics of the lifetime creativity scales and validation with three large samples. Journal of Personality and Social Psychology, 54(3), 476–485. https://doi.org/10.1037/0022-3514.54.3.476
Rogaten, J., & Moneta, G. B. (2016). Creativity in higher education: The use of creative cognition in studying. In G. B. Moneta & J. Rogaten (Eds.), Psychology of creativity: Cognitive, emotional, and social processes (pp. 3–20). Nova Science Publishers.
Roscoe, R. D., Allen, L. K., & McNamara, D. S. (2018). Contrasting writing practice formats in a writing strategy tutoring system. Journal of Educational Computing Research, 57(3), 723–754. https://doi.org/10.1177/0735633118763429
Ruesch, J., & Bateson, G. (2008). Communication: The social matrix of psychiatry. Routledge.
Runco, M. A. (1995). Insight for creativity, expression for impact. Creativity Research Journal, 8(4), 377–390. https://doi.org/10.1207/s15326934crj0804_4
Runco, M. A., & Jaeger, G. J. (2012). The standard definition of creativity. Creativity Research Journal, 24(1), 92–96. https://doi.org/10.1080/10400419.2012.650092
Runco, M. A., Millar, G., Acar, S., & Cramond, B. (2010). Torrance tests of creative thinking as predictors of personal and public achievement: A fifty-year follow-up. Creativity Research Journal, 22(4), 361–368. https://doi.org/10.1080/10400419.2010.523393
Runco, M. A., Noble, E. P., Reiter-Palmon, R., Acar, S., Ritchie, T., & Yurkovich, J. M. (2011). The genetic basis of creativity and ideational fluency. Creativity Research Journal, 23(4), 376–380. https://doi.org/10.1080/10400419.2011.621859
Runco, M. A., & Pritzker, S. R. (Eds.). (2011). Encyclopedia of creativity (2nd ed). Academic Press.
Said-Metwaly, S., Van den Noortgate, W., & Kyndt, E. (2017). Approaches to measuring creativity: A systematic literature review. Creativity, 4(2), 238–275. https://doi.org/10.1515/ctra-2017-0013
Sarikaya, I. (2024). Are students with high creativity skills successful writers? A parallel mediation model. Thinking Skills and Creativity, Article 101654. https://doi.org/10.1016/j.tsc.2024.101654
Shao, Z., Janse, E., Visser, K., & Meyer, A. S. (2014). What do verbal fluency tasks measure? Predictors of verbal fluency performance in older adults. Frontiers in Psychology, 5, Article 772. https://doi.org/10.3389/fpsyg.2014.00772
Shermis, M. D., Burstein, J., Higgins, D., & Zechner, K. (2010). Automated essay scoring: Writing assessment and instruction. In P. Peterson, E. Baker, & B. McGraw (Eds.), International encyclopedia of education (3rd ed., 20–26). https://doi.org/10.1016/B978-0-08-044894-7.00233-5
Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 503–524). Information Age Publishers.
Simon, H. A. (1985). Some computer models of human learning. In M. Shafto (Ed.), How we know (pp. 89-120). Harper & Row.
Smith, R. W., & Kounios, J. (1996). Sudden insight: All-or-none processing revealed by speed–accuracy decomposition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22(6), 1443–1462. https://doi.org/10.1037/0278-7393.22.6.1443
Sripada, C., & Taxali, A. (2020). Structure in the stream of consciousness: Evidence from a verbalized thought protocol and automated text analytic methods. Consciousness and Cognition, 85, 103007. https://doi.org/10.1016/j.concog.2020.103007
Stevenson, C., Smal, I., Baas, M., Dahrendorf, M., Grasman, R., Tanis, C., Scheurs, E., Sleiffer, D., & van der Maas, H. (2020). Automated AUT scoring using a big data variant of the consensual assessment technique [Technical report]. Modeling Creativity Project, Universiteit van Amsterdam. https://hdl.handle.net/11245.1/13ad004a-1b61-45a0-8a9a-56d7a165d7ef
Tombaugh, T. N., Kozak, J., & Rees, L. (1999). Normative data stratified by age and education for two measures of verbal fluency: FAS and animal naming. Archives of Clinical Neuropsychology, 14(2), 167–177. https://doi.org/10.1016/S0887-6177(97)00095-4
Torrance, E. P. (1972). Predictive validity of the Torrance tests of creative thinking. The Journal of Creative Behavior, 6(4), 236–262. https://doi.org/10.1002/j.2162-6057.1972.tb00936.x
Treffinger, D. J. (1986). Research on creativity. Gifted Child Quarterly, 30(1), 15–19. https://doi.org/10.1177/001698628603000103
Wati, N. S. (2019). The effect of creativity toward students’ achievement in writing ability. Pedagogy: Journal of English Language Teaching, 6(2), 141–147. https://doi.org/10.32332/pedagogy.v6i2.1330
Zedelius, C. M., Mills, C., & Schooler, J. W. (2019). Beyond subjective judgments: Predicting evaluations of creative writing from computational linguistic features. Behavior Research Methods, 51(2), 879–894. https://doi.org/10.3758/s13428-018-1137-1
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Journal of Learning Analytics
This work is licensed under a Creative Commons Attribution 4.0 International License.
