news LinkedIn

Over one year ago, we got a lot of response here on LinkedIn for our preprint "Large language models do not simulate human psychology". While psychologists seemed to be somewhat puzzled that such… | Benjamin Paaßen | 13 comments

Benjamin Paaßen announces the peer-reviewed publication of a study originally released as the preprint “Large language models do not simulate human psychology.” The paper, now titled “Exploring the Limits of Large Language Models as a Reflection of Human Cognition: An Illustration in the Context of Moral Judgment,” examines whether LLMs can reliably stand in for human participants. The central finding is that subtle rewordings of moral scenarios produce response patterns in LLMs that differ substantially from those observed in humans. Although some models may respond to wording changes, their reactions do not align with human sensitivity to the same changes. Older and smaller models often ignore rewording, while newer models react more strongly but still in psychologically distinct ways. Across the expanded set of models tested, the correlation between human and LLM responses declines significantly under rephrasing. Since the preprint, the authors evaluated additional and newer models, substantially revised the theoretical framework using cognitive-architecture research, and added extensive supplementary analyses. The authors emphasize that producing human-like answers on particular tasks does not demonstrate human-like cognition or stable psychological processes. Their conclusion is that, for research intended to characterize human judgments, experiences, or behavior, LLMs cannot replace human participants. Commenters generally welcomed the publication and highlighted its implications for understanding machine cognition, the distinction between behavioral resemblance and shared underlying processes, and the risks of using LLMs as “virtual panels” or digital twins in consumer and survey research. Some also noted that AI capabilities may evolve as developers specifically target these limitations.

Build this article’s timeline

Extract dated events, key actors, and open questions using the same timeline analysis available in Signal.

Log in to build

Timeline generation is available to signed-in community members.

Five Ws 🤖 3 AI prefills · 2 still unanswered
Who
Sarah Schroeder, Ulrike Kuhl, Valerie Vaquet, and LinkedIn
🤖 AI prefill
What
Benjamin Paaßen announces the peer-reviewed publication of a study originally released as the preprint “Large language models do not simulate human psychology.” The paper, now titled “Exploring the Limits of Large Language Models as a Refl…
🤖 AI prefill
When When did this happen, or what is the timeframe?
Where Where did this occur?
Why
Centers on large language models, human cognition, and human psychology
🤖 AI prefill

Login to answer and earn XP for each gap you fill.

LLM Human Cognition large language models, human cognition, psychology, simulation, human participants Moral Judgment Robustness moral scenarios, rewording, input changes, human responses, LLM responses Model Differences And Development older models, smaller models, newer models, LLM comparison, rewording sensitivity Research And Market Implications peer review, human participants, virtual panels, digital twins, market research
large language models100% human cognition98% human psychology94% moral judgment93% human participants92% rewording of moral scenarios90% human–LLM response correlations89% simulation of human participants88% cognitive architecture84% sensitivity to input rewording83%
8
Manipulation Score: Low How much this article uses emotional framing vs. straight reporting
Loaded Language 525%
Absolutist 815%
Threat/Panic 225%
Us vs Them 315%
Engagement Bait 1220%
Manipulation score look right?

Factual Claims (10)

medium scientific
The paper was published by the journal "Technology, Mind, and Behavior" under the title "Exploring the Limits of Large Language Models as a Reflection of Human Cognition."
medium scientific
Subtle changes to input text can lead to very different reactions from humans and large language models.
medium scientific
The reactions of LLMs and humans are clearly very different for most moral scenarios tested.
low scientific
The study evaluated more LLMs, including more recent models.
medium scientific
Under rewording, the correlation between human responses and LLM responses significantly decreases.
medium scientific
Older and smaller LLMs tend to ignore rewordings, while newer LLMs react to them differently from humans.
low scientific
The authors revised the theory section and incorporated arguments from cognitive architecture research.
low scientific
The authors added supplementary analyses to examine patterns in their data.
high scientific
For research questions concerning human judgments, experiences, or behavior, human participants cannot be replaced.
medium economic
Some market research organizations sell LLM-based virtual panels or digital twins as substitutes for human participants.

Comments

Login to view and post comments.

🏠Portal 📰Links ❓Q&A 📅Events 💼Jobs