Over one year ago, we got a lot of response here on LinkedIn for our preprint "Large language models do not simulate human psychology". While psychologists seemed to be somewhat puzzled that such… | Benjamin Paaßen | 13 comments
Benjamin Paaßen announces the peer-reviewed publication of a study originally released as the preprint “Large language models do not simulate human psychology.” The paper, now titled “Exploring the Limits of Large Language Models as a Reflection of Human Cognition: An Illustration in the Context of Moral Judgment,” examines whether LLMs can reliably stand in for human participants.
The central finding is that subtle rewordings of moral scenarios produce response patterns in LLMs that differ substantially from those observed in humans. Although some models may respond to wording changes, their reactions do not align with human sensitivity to the same changes. Older and smaller models often ignore rewording, while newer models react more strongly but still in psychologically distinct ways. Across the expanded set of models tested, the correlation between human and LLM responses declines significantly under rephrasing.
Since the preprint, the authors evaluated additional and newer models, substantially revised the theoretical framework using cognitive-architecture research, and added extensive supplementary analyses. The authors emphasize that producing human-like answers on particular tasks does not demonstrate human-like cognition or stable psychological processes.
Their conclusion is that, for research intended to characterize human judgments, experiences, or behavior, LLMs cannot replace human participants. Commenters generally welcomed the publication and highlighted its implications for understanding machine cognition, the distinction between behavioral resemblance and shared underlying processes, and the risks of using LLMs as “virtual panels” or digital twins in consumer and survey research. Some also noted that AI capabilities may evolve as developers specifically target these limitations.
Article timeline
Build this article’s timeline
Extract dated events, key actors, and open questions using the same timeline analysis available in Signal.
Sarah Schroeder, Ulrike Kuhl, Valerie Vaquet, and LinkedIn
🤖 AI prefill
📰What
Benjamin Paaßen announces the peer-reviewed publication of a study originally released as the preprint “Large language models do not simulate human psychology.” The paper, now titled “Exploring the Limits of Large Language Models as a Refl…
🤖 AI prefill
📅WhenWhen did this happen, or what is the timeframe?
📍WhereWhere did this occur?
💡Why
Centers on large language models, human cognition, and human psychology
🤖 AI prefill
Login to answer and earn XP for each gap you fill.
Key Phrases:large language models100%human cognition98%human psychology94%moral judgment93%human participants92%rewording of moral scenarios90%human–LLM response correlations89%simulation of human participants88%cognitive architecture84%sensitivity to input rewording83%
Manipulation Score: LowHow much this article uses emotional framing vs. straight reporting
Loaded Language525%
Absolutist815%
Threat/Panic225%
Us vs Them315%
Engagement Bait1220%
Manipulation score look right?
Factual Claims (10)
mediumscientific
The paper was published by the journal "Technology, Mind, and Behavior" under the title "Exploring the Limits of Large Language Models as a Reflection of Human Cognition."
mediumscientific
Subtle changes to input text can lead to very different reactions from humans and large language models.
mediumscientific
The reactions of LLMs and humans are clearly very different for most moral scenarios tested.
lowscientific
The study evaluated more LLMs, including more recent models.
mediumscientific
Under rewording, the correlation between human responses and LLM responses significantly decreases.
mediumscientific
Older and smaller LLMs tend to ignore rewordings, while newer LLMs react to them differently from humans.
lowscientific
The authors revised the theory section and incorporated arguments from cognitive architecture research.
lowscientific
The authors added supplementary analyses to examine patterns in their data.
highscientific
For research questions concerning human judgments, experiences, or behavior, human participants cannot be replaced.
mediumeconomic
Some market research organizations sell LLM-based virtual panels or digital twins as substitutes for human participants.
Comments
Login to view and post comments.