سلامة الذكاء الاصطناعي

AI research leaders call for monitoring model “thoughts” to ensure safety

Written by

Picture of فريقنا

فريقنا

Communications Consultant

In a rare moment of unity, researchers from OpenAI, Google DeepMind, Anthropic, and other major companies are calling for a deeper investigation into monitoring the so-called “thoughts” of AI models, arguing it could be the primary way to maintain control over powerful AI agents.

Introduction: a peek inside the artificial mind

AI researchers from OpenAI, Google DeepMind, Anthropic, and a broad coalition of companies and non-profit groups are calling for a deeper investigation into monitoring techniques for the “thoughts” of AI reasoning models, according to a position paper published on Tuesday. This unified call comes at a time when AI capabilities are scaling dramatically, alongside growing concerns over how to ensure their safety and control.

Chains-of-thought (CoT): the key to understanding

A key feature of AI reasoning models, such as OpenAI’s o3 and DeepSeek’s R1, is their “Chains-of-Thought” (CoTs)—an external process through which AI models solve problems, akin to how humans use scratch paper to work through a difficult math problem. Reasoning models are considered a foundational technology for powering AI agents, and the paper’s authors argue that monitoring the Chain-of-Thought (CoT) could be a crucial method for maintaining control over AI agents as they proliferate and grow more capable.

“CoT monitoring offers a valuable addition to frontier AI safety measures, providing a rare glimpse into how AI agents make decisions,” the researchers wrote in the position paper. “However, there is no guarantee that the current degree of visibility will persist. We encourage the research community and frontier AI developers to make the most of CoT observability and study how to preserve it.”

A call to action and research

This position paper asks leading AI model developers to examine what makes chains-of-thought “observable”—in other words, what factors can increase or decrease transparency into how AI models truly arrive at answers. The paper’s authors say that CoT monitoring could be a key method for understanding AI reasoning models, but note that it may be fragile, warning against any interventions that could reduce its transparency or reliability.

The paper’s authors also call on AI model developers to track CoT observability and study how this method could one day be implemented as a safety measure.

Rare unity in a competing industry

The paper represents a moment of unity among many AI industry leaders in an effort to boost research on AI safety. This comes at a time when tech companies are locked in fierce competition—which has led Meta to poach top researchers from OpenAI, Google DeepMind, and Anthropic with multi-million-dollar offers. Notable signatories on the paper include OpenAI Head of Research Mark Chen, Safe Superintelligence CEO Ilya Sutskever, Nobel laureate Geoffrey Hinton, and Google DeepMind co-founder Shane Legg.

“We are at this critical juncture where we have this new thing called the chain of thought,” said Bowen Baker, an OpenAI researcher who worked on the paper. “It seems very useful, but it could disappear in a few years if people don’t really focus on it. Publishing a position paper like this, for me, is a mechanism to get more research and attention on this topic before that happens.”

Conclusion: the race toward transparency

Although AI labs have excelled at improving AI performance over the past year, this has not necessarily translated to a better understanding of how they arrive at their answers. Companies like Anthropic are industry pioneers in trying to figure out how AI models actually work—a field called interpretability. This paper indicates that the industry as a whole now realizes that building stronger AI must go hand in hand with building more transparent and safer AI.

شارك هذا الموضوع:

شارك هذا الموضوع:

اترك رد

Leave a Reply

الفئات

المنشورات الأخيرة