The claim of protecting conversation privacy completely contradicts the exposure of users’ personal text and the display of their sensitive memory summaries before the eyes of external reviewers.
Article index:
- Filing the class-action lawsuit in U.S. courts
- Details of the Project Lily scandal and the 404 Media investigation
- Failure of privacy filters and exposure of user memories
- Conversation evaluation rules and avoiding hypocrisy and legal risks
- Legal dilemmas in Europe and previous Italian fines
- Model training opt-out mechanism and the retroactive dilemma
- Frequently asked questions
Filing the class-action lawsuit in U.S. courts
A group of users of the popular chatbot “ChatGPT” filed a proposed class-action lawsuit against “OpenAI” on Wednesday, accusing the company of intentional misrepresentation and deception regarding the level of confidentiality and privacy of their personal conversations, according to the specialized legal network “Le 360”.
This lawsuit followed a documented investigative journalistic report proving that OpenAI paid hefty sums to hundreds of external contractors and reviewers at a rate exceeding $50 per hour to read, summarize, and evaluate real conversations of the platform’s customers, as part of a secret internal program codenamed “Project Lily”.
Details of the Project Lily scandal and the 404 Media investigation
The lawsuit was based on an investigative report published by journalist Joseph Cox via the “404 Media” platform on September 14, drawing on leaked documents, internal training evidence, and actual user conversations. The report revealed an organized human review pipeline where an employee reads the user’s query, summarizes the exact intent behind it, and then evaluates four alternative responses generated by the bot on a scale from 1 to 7.
The evaluation process does not aim to verify factual accuracy; instead, it instructs reviewers to penalize “artificial AI language”, flag warnings if the bot claims human emotions, and punish sycophantic and hypocritical answers that OpenAI fears could entangle it in lawsuits for psychological damage compensation.
Failure of privacy filters and exposure of user memories
The lawsuit asserted that OpenAI’s defenses regarding the deletion of explicit names and passing texts through an automated privacy filter are flimsy and ineffective, as the company acknowledged in its internal documents that the automated filter overlooks rare identifiers and precise implicit data.
More dangerously, reviewers reported that the dashboards they work on display “user memory summaries”, revealing a person’s profession, approximate geographic location, and family secrets extracted from previous sessions. One contractor stated clearly: “I don’t think any of the 900 million users imagine that a contractor is sitting somewhere analyzing the details of their life.”
Conversation evaluation rules and avoiding hypocrisy and legal risks
This new case joins a heavy and crowded judicial record pursuing OpenAI in the U.S. judiciary, where the company faces multiple lawsuits related to wrongful death and causation resulting from providing misleading psychological advice, alongside major lawsuits for intellectual property infringement and training models on protected content without a license.
The company is also engaged in an independent legal dispute in the state of California accusing it of sharing user data with external parties and marketing companies via embedded digital tracking tools without obtaining explicit and informed legal consent.
Legal dilemmas in Europe and previous Italian fines
These leaks impose escalating regulatory pressure on the company within the European Union, where the Italian Data Protection Authority had already fined OpenAI 15 million euros for processing user data without sufficient legal basis.
Furthermore, a ruling by the Court of Justice of the European Union in the European Data Protection Supervisor case establishes a binding legal principle stipulating that the duty of companies to inform users transparently about how their data is processed and used to train models applies at the exact moment of data collection, rather than through vague subsequent notices.
Model training opt-out mechanism and the retroactive dilemma
OpenAI applies the “Improve the model for everyone” option by default and automatically to free ChatGPT accounts as well as “Plus” and “Pro” tiers, while disabling this option by default only for enterprise and educational institution tiers.
Although users can manually disable the feature via settings and data controls, the lawsuit points out that this complex procedure lacks transparency and carries no retroactive effect, meaning that millions of conversations and sensitive data previously pulled and subjected to human review cannot be retrieved or erased by users from the training databases.
Frequently asked questions
Question: What is the legal basis for the class-action lawsuit filed against OpenAI?
Answer: Misleading users and giving them a false sense of absolute privacy for their conversations while hundreds of external reviewers read them in Project Lily.
Question: What is the nature of the sensitive data accessed by ChatGPT reviewers?
Answer: They accessed full conversation texts and user memory summaries including profession, approximate location, and personal background.
Question: Does disabling the model improvement option protect previous conversations from review?
Answer: No, disabling the option only prevents the use of future conversations and does not delete old chats already pulled for training.
