Can ChatGPT Analyse Textual Data? The Sub-Themes Reflected by Typical Conceptual Metaphors in Short Stories of Language Assessment

Abstract

Conceptual metaphors constitute a basic facet of human cognition, enabling individuals to comprehend abstract concepts by means of more concrete experiences. Previous research has found that metaphors are frequently employed in discourse and have a substantial impact on how we think. However, less consideration has been given to how conceptual metaphors reflect the views of writers and the themes of stories. The primary objective of this study is to examine the underlying sub-themes that are conveyed and reflected through typical conceptual metaphors found in the 22 short stories from the book Tests and Us – A Collection of Real Stories (Volume 2). Additionally, this study intends to delve into the perspectives of the writers on the principles and associated concepts of language assessment. Following the Conceptual Metaphor Theory (CMT) by Lakoff and Johnson (1980), the structural, orientational, and ontological metaphors were identified by the AI program ChatGPT (GPT-4) and validated by linguistic experts. The findings revealed a cumulative count of 250 conceptual metaphors, comprising 131 structural metaphors, 64 ontological metaphors, and 55 orientational metaphors. Meanwhile, the typical metaphors have served as representations of the underlying sub-themes pertaining to the five principles of language assessment, as well as concepts including fairness and equity, formative assessment, learning environment, ethics, and integrity, among others. Through metaphors, thematic analysis not only becomes a tool for dissecting narratives but also an avenue for exploring the profound connections between language, thought, and emotion. This offers some insights for future research into various literary genres and the application of ChatGPT for the analysis of textual data.