Introduction
In an advanced move reflecting the growing reliance on specialized human expertise to refine artificial intelligence tools, Anthropic is currently running a large-scale operation behind the scenes aimed at improving its smart programming tool. According to a report published by the technology-focused publication Business Insider at the end of last May, the company has enlisted about 1,000 contract software engineers through data supplier Snorkel AI. This exceptional effort aims to evaluate and improve the quality of outputs and code generated by the Claude Code model effectively and professionally, ensuring it is error-free and raising its efficiency in performing complex tasks.
- Project Marlin details and workflow
- Real-world scenarios and benchmark testing
- The shift in model training labor requirements
- Claude Code’s dominance in the startup market
- Frequently asked questions
Project Marlin details and workflow
This intensive technical effort is known internally at Snorkel AI as Project Marlin, an innovative project that provides freelance engineers with a generous pay of up to $280 USD for each completed task. These tasks include creating code prompts and carefully reviewing AI-generated code, with a single task typically taking about an hour of dedicated focus. The core work revolves around blind comparison tests, where contractors compare code generated by two different versions of the language model and then select the better output based on strict criteria including efficiency, clarity, and future maintainability. To ensure maximum objectivity in evaluation, contractors are not informed which version of the model they are testing.
Real-world scenarios and benchmark testing
The tasks assigned to these engineers are derived from actual development scenarios using live code repositories on the popular GitHub platform. Engineers craft complex tasks related to adding new software features or fixing stubborn bugs, then ask Claude Code to generate integrated solutions for subsequent evaluation—work that mimics the daily tasks of professional developers in real-world work environments. Live examples include testing the model’s skills in refactoring system metadata processing or implementing advanced security measures for a machine learning platform, skills considered essential for the success of any technical application in today’s competitive market.
The shift in model training labor requirements
This project illustrates a broader and more complex trend in the modern tech industry, where the demand for labor dedicated to training AI models has shifted from simple tasks like initial data labeling to complex tasks requiring advanced technical and engineering expertise. Data platforms play a vital role in this context; Snorkel AI, founded by Stanford University researchers in 2019, actively supports the development of AI models for major tech companies including Google and Meta. Highly skilled and exceptional engineers working on similar projects can earn more than $3,000 USD weekly, while similar platforms like Scale AI and Mercor operate globally to provide this top-tier talent and meet market needs.
Claude Code’s dominance in the startup market
This massive investment comes as Claude Code strongly solidifies its position in the smart programming tool market. A survey of more than twenty startup founders and venture capitalists showed a growing consensus that the model has become the default and essential programming tool within technological work environments. Last May, Krishna Rao, Anthropic’s Chief Financial Officer, stated that artificial intelligence now writes more than 90 percent of the company’s own code. The tool’s creator also confirmed that the product achieved impressive revenues of $2.5 billion USD in its first year and captured 51 percent of the AI-driven programming market share, confirming that artificial intelligence systems still rely on human expertise to achieve professional performance.
Frequently asked questions
Question: What is the primary goal of Anthropic hiring 1,000 software engineers?
Answer: The goal is to evaluate and improve the quality of code generated by the Claude Code model to ensure its efficiency and clarity before adoption.
Question: What technical evaluation method is used in Project Marlin?
Answer: The project follows a blind comparison test method between two different versions of the model without informing the engineer, to ensure neutrality in evaluation.
Question: What is Claude Code’s market share in the smart programming field?
Answer: The tool captured 51 percent of the AI-driven programming market share with very massive revenues.
