article contents:
- introduction: the mystery of changing chatbot answers
- a first look at the enigmatic thinking machines lab
- uncovering the root cause of randomness in ai responses
- why does getting consistent responses matter?
- is this the lab’s first product?
- a commitment to open research: will the company keep its promise?
- conclusion: a major challenge at the frontier of ai research
- frequently asked questions
introduction: the mystery of changing chatbot answers
have you ever asked chatgpt the same question multiple times and gotten different answers each time? this phenomenon, known as “nondeterminism,” has long been an accepted feature of current ai models. but thinking machines lab, the new startup founded by mira murati, the former chief technology officer of openai, argues that this is not an immutable truth, but a problem that can be solved. in a first look at its research, the lab reveals its quest to make artificial intelligence responses reproducible, a game-changing step for enterprises and scientists alike.
a first look at the enigmatic thinking machines lab
thinking machines lab has generated significant interest in silicon valley, especially after raising a massive $2 billion seed funding round and adding a team of top former openai researchers. in a recently published research blog post, the lab gave the world its first glimpse into one of its ambitious projects: building ai models with reproducible responses. this step reflects the company’s ambition to tackle fundamental challenges in ai technology rather than simply building larger models.
uncovering the root cause of randomness in ai responses
in the blog post titled “defeating nondeterminism in large language model inference,” researcher horace he attempts to unpack the root cause introducing randomness into ai model responses. he argues that the reason lies not in the model itself, but in how “gpu kernels”—the small programs running inside nvidia chips—are assembled together during the inference process (everything that happens after you press enter in chatgpt). he suggests that by tightly controlling this coordination layer, it is possible to make ai models more deterministic, meaning they will give the exact same answer to the same question every time.
why does getting consistent responses matter?
changing answers may sometimes seem like a creative feature, but they pose a major obstacle in many professional and scientific applications. making ai responses reproducible has massive benefits:
- enterprise reliability: companies using ai in customer service or financial analysis can rely on getting consistent results, reducing errors and increasing trust in the system.
- scientific accuracy: researchers using ai to simulate experiments or analyze data can be confident that their results are reproducible, a cornerstone of the scientific method.
- improved reinforcement learning (rl) training: the researcher notes that making responses consistent can improve reinforcement learning training, the process where models are rewarded for correct answers. if answers vary slightly each time, the data becomes “noisy.” consistent responses make the training process smoother and more efficient.
is this the lab’s first product?
murati stated in july that thinking machines lab’s first product will be unveiled in the coming months, and that it will be “useful for researchers and startups developing custom models.” it remains unclear what this product is, or whether it will use techniques from this research to generate more reproducible responses. nevertheless, this research gives us a strong idea of the core problems the lab aims to solve.
a commitment to open research: will the company keep its promise?
thinking machines lab stated that it plans to frequently publish research blog posts, code, and other information “to benefit the public, but also to improve our own research culture.” this blog post, the first in the company’s new blog series, appears to be part of that effort. interestingly, openai made a similar commitment to open research when it was founded, but the company became more closed off as it grew. it remains to be seen whether murati’s lab will stay true to this promise, especially with its $12 billion valuation.
conclusion: a major challenge at the frontier of ai research
this research blog post offers a rare glimpse inside one of silicon valley’s most secretive ai startups. while it does not reveal the exact direction of the technology, it indicates that thinking machines lab is tackling some of the biggest questions at the frontier of ai research. the real test is whether they can solve these problems and build products around their research to justify their massive valuation. if they succeed, ai answers may no longer be random.
frequently asked questions
q1: what is nondeterminism in artificial intelligence?
a1: it is the tendency of ai models, such as chatbots, to provide different answers to the same input or question when repeated. this happens due to multiple factors in how data is processed at the hardware and software levels.
q2: why are changing answers a problem?
a2: in applications requiring high precision and reliability, such as scientific, financial, or medical uses, inconsistent answers can lead to unreliable or even dangerous results. reliability requires the ability to reproduce results.
q3: who is mira murati?
a3: mira murati was the chief technology officer (cto) at openai and is one of the key figures behind the development of models like gpt-4 and chatgpt. she left to found her own company, thinking machines lab, in 2025.
q4: will making ai predictable affect its creativity?
a4: there will likely be a tradeoff. systems can be designed to be deterministic when needed (for accuracy) and allow a degree of randomness when creative or diverse results are desired. the goal is to control this property, not eliminate it entirely.
