Hundreds of reviewers look daily into detailed personal details shared by users who asked the bot to keep them confidential, unaware that human eyes are reading their texts.
Article index:
- 404 Media investigation and secret Project Lily
- Contractor duties and hourly response evaluation mechanism
- What reviewers see: memory summaries and sensitive data
- Privacy filter failure and user pleas to keep secrets
- Training rules, avoiding linguistic hypocrisy, and lawsuits
- Industry practices and steps to opt out of data sharing
- Frequently asked questions
404 Media investigation and secret Project Lily
Hundreds of contractors and external employees hired by OpenAI are reading real, highly private conversations and texts exchanged by ChatGPT users as part of a secret internal training initiative codenamed “Project Lily,” according to an extensive investigative report by digital security outlet 404 Media.
This revelation raises embarrassing questions and concerns regarding the violation of privacy for more than 900 million active users, a large segment of whom treat the AI platform as a personal vault or a closed, secure space to record their daily journals, thoughts, and sensitive medical and psychological consultations.
Contractor duties and hourly response evaluation mechanism
These external reviewers are paid more than $50 an hour through intermediary staffing and outsourcing companies. Their primary task consists of reading complete user inquiry texts, summarizing the exact intent and motivations behind a user’s question, and then evaluating four alternative responses generated by ChatGPT on a precise scale from one to seven.
With this process, OpenAI aims to train and modify the robot’s behavior so that its voice becomes less mechanical and dry, less hypocritical and sycophantic, and more natural and conversational like human communication. However, this goal inevitably means that users’ personal data falls under human eyes examining its minute details.
What reviewers see: memory summaries and sensitive data
Internal documents and guideline manuals obtained by the investigative outlet revealed that the contractor dashboard sometimes includes a “user memory summary,” which is a comprehensive overview of what the user has previously discussed with ChatGPT across multiple sessions, revealing to the reviewer their approximate geographic location, nature of profession and work, and complex personal and family circumstances.
OpenAI claims it strips explicit usernames from conversations and passes them through an automated “privacy filter” before they reach reviewers’ screens, but the company admits in internal documents that this filter “may miss and fail to recognize uncommon identifiers or vague personal and private references,” leaving sensitive data exposed.
Privacy filter failure and user pleas to keep secrets
Samples of conversations reviewed by reviewers showed users writing urgent phrases in their queries such as “keep this a secret between us and don’t tell anyone,” without realizing that such programmatic phrases hold no legal or technical authority to bypass the platform’s data collection and model training policies.
A contractor told 404 Media, when asked whether users realize humans are reading their texts: “Absolutely not; I don’t think they imagine at all that there is an employee sitting somewhere analyzing their private texts line by line,” revealing a wide gap between user expectations and company behavior.
Training rules, avoiding linguistic hypocrisy, and lawsuits
Training guidelines instruct reviewers to penalize “artificial AI language,” such as excessive and unjustified use of emojis, to alert the system when the bot claims to experience real human experiences and emotions, and to flag “hypocritical and sycophantic language,” an issue linked to lawsuits accusing ChatGPT of echoing harmful user thoughts and causing them psychological harm.
Reviewers are not asked to verify historical or scientific facts, but they are asked to draw attention to glaring errors and escalate tasks involving highly dangerous personal information or physical safety concerns.
Industry practices and steps to opt out of data sharing
This practice is followed across major tech companies. Anthropic confirmed the use of human reviews to improve the Claude model when users enable the training feature, and Google displays an explicit warning in the Gemini interface stating that “humans may review some saved conversations.”
In free ChatGPT accounts as well as Plus and Pro plans, the model improvement feature is enabled automatically and by default. Users can turn it off by going to settings, selecting data controls, and disabling the “Improve the model for everyone” option, noting that opting out applies only to new chats and does not delete texts already pulled into training and human review pipelines.
Frequently asked questions
Question: What is the codename of the secret project carried out by OpenAI to review conversations?
Answer: It bears the codename “Project Lily” and includes hundreds of contractors who earn more than $50 an hour to evaluate responses.
Question: Do dashboards reveal details about users’ personalities and circumstances to reviewers?
Answer: Yes, they sometimes include the user’s memory summary, which may reveal their job, approximate location, and complex family circumstances.
Question: How can ChatGPT users prevent humans and company algorithms from reading their texts?
Answer: By going to settings, then data controls, and disabling the model improvement option for everyone in their accounts.
