Exploring ChatGPT’s Capabilities in Creative Writing: Can GPT-4o Conduct Rhetorical Move Analysis in Narrative Short Stories?

Abstract

With the rapid advancement of artificial intelligence (AI) technology, the application of AI in academic research and language education has become increasingly widespread. GPT-4o, a new flagship model that can reason across audio, vision, and text in real time, has been found to offer substantial improvements over its predecessors. However, its capability to analyse rhetorical moves in creative writing remains unclear. This study aims to investigate whether GPT-4o can identify the rhetorical moves in narrative short stories with a focus on language assessment themes, and check the accuracy of the results by human experts. Moreover, the essentiality and frequency of the moves in these stories are reported. Adopting Jiang et al.’s (2024) eight-move model for the annotation of narrative stories, the results indicated that while ChatGPT (GPT-4o) quickly identified rhetorical moves, its segmentation of sentence endpoints differed slightly from human coders, particularly in several moves. However, these differences did not affect the overall range, frequency, or sequence of the moves, suggesting that GPT-4o can still effectively identify moves in narrative short stories. Another finding showed that Moves 1 to 6 were obligatory in all 50 narrative short stories from Tests & Us (Volumes 2 & 3), while Moves 7 and 8 were conventional, with no optional moves identified. As for the occurrence frequency, these stories contain 478 moves, with Move 4, “Complicating the story” being the most frequent, appearing 80 times (16.7%). These findings offer insights into the potential of AI to support textual analysis in creative writing and encourage researchers to explore rhetorical moves in narratives.