In a major strategic shift, companies like Google, Meta, and NVIDIA are stepping up their investments in “world models,” a technology aimed at understanding and interacting with the physical world through video and robotics data, amid signs that large language models have hit a development ceiling.
Article Contents:
- The race toward artificial superintelligence takes a new path
- Have large language models reached their end?
- What are “world models” and how do they work?
- A $100 trillion market: Massive economic potential
- Tangible progress: Google and Meta unveil their latest innovations
- The big data challenge
- Future vision: Physical AI
- FAQs
The race toward artificial superintelligence takes a new path
The world’s largest artificial intelligence groups are intensifying their focus on so-called “world models” that can better understand human environments, as part of a search for new ways to achieve machine “superintelligence.” Google DeepMind, Meta, and NVIDIA are among the companies trying to make headway in the AI race by developing systems aimed at navigating the physical world by learning from video and robotics data rather than relying solely on language.
Have large language models reached their end?
This trend comes at a time when questions are mounting over whether large language models (LLMs)—the technology powering popular chat-bots like OpenAI’s ChatGPT—have hit a ceiling in their progress. Performance jumps among LLMs released by companies across the sector, such as OpenAI, Google, and Elon Musk’s xAI, have seen a noticeable slowdown despite massive sums invested in their development. This has pushed researchers to look for the “next big thing” that can push the boundaries of AI even further.
What are “world models” and how do they work?
Unlike language models that learn from text, “world models” are trained using data streams from real or simulated environments. These models learn world physics, how objects interact, and causal relationships by watching endless video clips or data from robots interacting with their surroundings. They are seen as a crucial step to drive progress in fields such as self-driving cars, robotics, and so-called “AI agents.” However, they require a massive amount of data and computing power for training and remain an unsolved technical challenge.
A $100 trillion market: Massive economic potential
The potential market for world models could be massive, roughly the size of the global economy, according to Rev Lebaredian, vice president of Omniverse and simulation technology at NVIDIA, as they take technology into the physical realm, such as the manufacturing and healthcare sectors. He said: “What is the foundation world model opportunity? Basically… $100 trillion if we can make intelligence that can understand and work in the physical world.”
Tangible progress: Google and Meta unveil their latest innovations
This focus on an alternative approach to language models has become clear as several AI groups revealed a series of advancements in world models in recent months. Last month, Google DeepMind showcased its Genie 3 model, which generates video frame by frame and takes past interactions into account, unlike previous models that generated video all at once. Meanwhile, Meta is attempting to mimic how children passively learn by observing the world around them, training its V-JEPA models on raw video content. Yann LeCun, Meta’s chief AI scientist, has been a vocal supporter of this new architecture, warning that language models will never achieve the ability to reason and plan like humans.
The big data challenge
To build these models, companies need to gather a vast amount of physical data about the world. San Francisco-based Niantic is mapping 10 million locations, collecting information through games like Pokémon Go, which features 30 million monthly players interacting with a global map. Even after selling the game, its players continue to contribute anonymized data by scanning public landmarks to help build the company’s world model.
Future vision: Physical AI
NVIDIA CEO Jensen Huang emphasized that the company’s next major growth phase will arrive with “physical AI,” where new models will revolutionize robotics. While some, like Meta’s LeCun, have said this vision for a new generation of AI systems powering human-level smart machines could take 10 years to achieve, the potential scale of this cutting-edge technology is vast, capable of unlocking service for all other industries and amplifying the same impact that computers brought to the knowledge world.
FAQs
Q1: What is the fundamental difference between language models and world models?
A1: Language models learn from text to understand and generate language. World models learn from visual and interactive data (video, robotics data) to build an internal understanding of how the physical world works, including physics and causality.
Q2: What are the applications of world models?
A2: Potential applications include robots capable of performing complex tasks in unstructured environments (such as homes and factories), self-driving cars that can better predict the behavior of other drivers and pedestrians, and the creation of realistic, interactive virtual environments for gaming and entertainment.
Q3: Why is the progress of large language models slowing down?
A3: Some experts believe language models are nearing the exhaustion of high-quality text data available on the internet. Furthermore, scaling models larger yields diminishing returns in performance improvements alongside skyrocketing computing costs, making progress much harder and more expensive.
