A recent study indicates that AI chatbots may excel over the average human in creative thinking tasks, such as suggesting alternative uses for everyday objects. Nevertheless, top-performing humans still surpassed the best chatbot results in these tasks.
A recent study published in the journal Scientific Reports suggests that large language model (LLM) AI chatbots might excel beyond the average human in creative tasks, like brainstorming alternate uses for common items – a reflection of divergent thinking. However, individuals who scored the highest in these tasks still surpassed the top-performing chatbot results.
Divergent thinking is a type of thought process often linked with creativity, emphasizing the generation of many different ideas or solutions for a specific task.
It is commonly assessed with the Alternate Uses Task (AUT), in which participants are asked to come up with as many alternative uses for an everyday object as possible within a short time period. The responses are scored for four different categories: fluency, flexibility, originality, and elaboration.
Mika Koivisto and Simone Grassini compared 256 human participants’ responses with those of three AI chatbots’ (ChatGPT3, ChatGPT4, and Copy.Ai) to AUTs for four objects — a rope, a box, a pencil, and a candle. The authors assessed the originality of the responses by rating them on semantic distance (how closely related the response was to the object’s original use) and creativity.
A computational method was used to quantify semantic distance on a scale from 0 to 2, whilst human evaluators, blind to the originators of the responses, subjectively rated creativity from 1 to 5. On average, the chatbot-generated responses scored significantly higher than the human responses for both semantic distance (0.95 vs. 0.91) and creativity (2.91 vs. 2.47).
The human responses had a far greater range in both measures — the minimum scores were much lower than for the AI responses, but the maximum scores were generally higher. The best human response outperformed each chatbot’s best response in seven out of eight scoring categories.
These findings suggest that AI chatbots can now generate creative ideas at least as well as the average human. However, the authors note that they only considered performance in a single task associated with the assessment of creativity. The authors propose that future research could explore how AI can be integrated into the creative process to improve human performance.
Reference: “Best humans still outperform artificial intelligence in a creative divergent thinking task” by Mika Koivisto and Simone Grassini, 14 September 2023, Scientific Reports.
DOI: 10.1038/s41598-023-40858-3
The study was funded by the University of Bergen.
Be the first to comment on "New Study: AI Chatbots Surpass the Average Human in Creativity"