Generative AI has not (yet?) surpassed humans in creative idea generation: A meta-analysis (2022-2025)
Biskjaer,Michael Mose de Rooij,Alwin
Biskjaer,Michael Mose
de Rooij,Alwin
Abstract
Has AI surpassed humans in creative idea generation? This question has gained traction as generative artificial intelligence (GenAI) tools are rapidly adopted for creativity support. To investigate this, we conducted a meta-analysis of 17 experimental studies published between January 2022 and January 2025, comprising 115 effect sizes comparing human and GenAI-prompted creative idea generation. We found a small but non-significant pooled effect size favoring GenAI. While initial analyses suggested greater originality in GenAI ideas, sensitivity analysis revealed this was driven by a few studies with very large effects. No significant differences emerged between ideas generated by humans and those produced by prompting specific GenAI models (GPT-3, GPT-3.5, GPT-4). Funnel plots, asymmetry tests, and p-curve analysis indicated no publication bias, thus supporting the findings’ validity. The findings raise theoretical and methodological questions about how GenAI generates creative output. Rather than producing genuinely novel, situated ideas, GenAI may retrieve statistically likely exemplars of creativity based on its training data. This underscores the need for sociocultural and sociotechnical perspectives that emphasize the embeddedness, ethical sensitivity, and interpersonal grounding of human creativity—dimensions not captured in current GenAI outputs. We argue that future research should examine GenAI-powered creativity in more ecologically valid settings, especially in co-creative human-AI processes. This includes acknowledging task design, participant diversity, and evolving cultural frameworks as key factors influencing creativity evaluations. Advancing the field will require integrating human and GenAI-powered creativity in ways that reflect their distinct affordances and, hopefully also, inspire new directions for both research and creative practice.
Description
Date
2025-09-03
Journal Title
Journal ISSN
Volume Title
Publisher
Research Projects
Organizational Units
Journal Issue
Keywords
creativity, generative AI, meta-analysis, idea generation
Citation
Biskjaer, M M & de Rooij, A 2025, 'Generative AI has not (yet?) surpassed humans in creative idea generation : A meta-analysis (2022-2025)', 9th MIC Conference 2025, Bolzano, Italy, 3/09/25 - 6/09/25.