news arXiv.org

[2509.05382] User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies

The paper titled "User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies," authored by Jennifer King and colleagues, examines the privacy policies of six leading U.S. AI developers regarding the use of chat data from large language models (LLMs). The study highlights that millions of users interact with chatbots and that developers are eager to gather high-quality training data to enhance their models. Utilizing a qualitative coding framework based on the California Consumer Privacy Act, the analysis reveals that all six companies use user chat data for model training and some retain this data indefinitely. Significantly, the paper indicates that developers may collect sensitive personal information, including biometric and health data, as well as data from uploaded files. Alarmingly, four of the six companies appear to include children's chat data in their training processes. The research underscores that developers' privacy policies frequently lack critical information about their data practices, emphasizing a pressing need for greater transparency and accountability. The authors address several implications of these findings, such as the absence of user consent for data usage, security issues linked to indefinite data retention, and the ethical concerns regarding training models on children’s data. The paper ultimately calls for enhanced privacy measures and clearer communication from developers to protect user privacy in the context of AI advancements.

Build this article’s timeline

Extract dated events, key actors, and open questions using the same timeline analysis available in Signal.

Log in to build

Timeline generation is available to signed-in community members.

Five Ws 🤖 5 AI prefills · all filled
Who
Jennifer King, Kevin Klyman, Emily Capstick, and arXiv
🤖 AI prefill
What
The paper titled "User Privacy and Large Language Models: An Analysis of Frontier Developers' Privacy Policies," authored by Jennifer King and colleagues, examines the privacy policies of six leading U.S.
🤖 AI prefill
When
5 Sep 2025 and 2025
🤖 AI prefill
Where
United States
🤖 AI prefill
Why
Centers on large language models, user privacy, and privacy policies
🤖 AI prefill

Login to answer and earn XP for each gap you fill.

User Privacy Concerns privacy policies, user data, chat data, transparency, consent Data Retention Practices data retention, indefinite storage, sensitive information, biometric data, health data Children's Data Usage children, chat data, model training, data collection, ethical concerns Regulatory Frameworks California Consumer Privacy Act, data protection, legal compliance, privacy rights, accountability
large language models95% user privacy90% privacy policies85% California Consumer Privacy Act80% data retention78% biometric data75% sensitive information73% chat data70% model training68% transparency and accountability65%
30
Manipulation Score: Low How much this article uses emotional framing vs. straight reporting
Loaded Language 1025%
Absolutist 515%
Threat/Panic 2025%
Us vs Them 1015%
Engagement Bait 520%
Manipulation score look right?

Factual Claims (6)

medium social
All six developers appear to employ their users' chat data to train and improve their models.
medium social
Some retain this data indefinitely.
high social
Developers may collect and train on personal information disclosed in chats, including sensitive information.
high social
Four of the six companies examined appear to include children's chat data for model training.
medium social
Developers' privacy policies often lack essential information about their practices.
medium social
There is a need for greater transparency and accountability.

Comments

Login to view and post comments.

🏠Portal 📰Links ❓Q&A 📅Events 💼Jobs