Meta has introduced a new artificial intelligence model called V-JEPA 2, which can seemingly help AI agents better understand and predict the real world as much like how humans observe, think, and plan before taking any action. According to Meta, this new open-source AI model is a big step towards developing what it calls advanced machine intelligence (AMI). AMI is Meta's vision for the future. It's an AI model that can not only process data but also learn from its surroundings and predict how things will change as just like humans do every day.

Meta calls V-JEPA 2 its most sophisticated world model to date. V-JEPA 2 stands for Video Joint Embedding Predictive Architecture 2. The model is primarily trained on vast amounts of video footage. The company explains that by watching a huge number of video clips as over a million hours as this AI learnt how people interact with objects, how things move, and how different actions affect the world around them. And with this training, AI can further enable robots and AI systems to anticipate how objects behave, how environments respond to motion, and how different elements interact physically.

"As humans, we have the ability to predict how the physical world will evolve in response to our actions or the actions of others," Meta said in its official blog post. "V-JEPA 2 helps AI agents mimic this intelligence, making them smarter about the physical world."

Giving an example Meta explains that just as a person knows a tennis ball will fall back down if thrown into the air, V-JEPA 2 can learn this kind of common-sense behaviour by observing video. This training with video and world understanding further helps AI develop a mental map or understanding of how the physical world works.

What makes Meta's V-JEPA 2 different?

V-JEPA 2 is a 1.2 billion-parametre model that builds on its predecessor V-JEPA, which Meta unveiled last year. This new generation is said to offer significant improvements in understanding, predicting, and planning. The company emphasises that, unlike previous systems, V-JEPA 2 is not just capable of recognising images or responding to commands, but it can actually make predictions. It can look at a situation and estimate what will happen next if a certain action is taken. These capabilities, according to Meta, are essential for AI to function autonomously in real-world settings. For instance, this could allow a robot to navigate unfamiliar terrain or manipulate objects it has never seen before.

Meta reveals that it has also tested this by putting the AI model into robots in its labs. During testing, the company claims these robots were able to complete basic tasks like picking up unfamiliar objects and placing them in new spots as even in environments the robot had never seen before. The robot used the model to plan its next move based on its current view and a goal image. It then chose the best action to take, step by step.

In support of the broader research community, Meta is also releasing three new benchmarks to evaluate how well AI models learn and reason from video. These benchmarks aim to standardise the way researchers test world models, offering a clearer path towards advancing physical reasoning in AI.

"By sharing this work, we aim to give researchers and developers access to the best models and benchmarks to help accelerate research and progress as ultimately leading to better and more capable AI systems that will help enhance people's lives," said Meta.

Meanwhile, while the company is currently focusing on short tasks like picking and placing objects, Meta says it wants to go further as developing models that can plan long-term, break down complex tasks into smaller steps, and even use senses like touch and sound in the future.