Google DeepMind has introduced SIMA 2, the latest iteration of its Scalable Instructable Multiworld Agent, marking a significant advancement in AI systems designed to reason, adapt, and interact with human instructions in complex virtual environments. Building on the original SIMA model launched in March 2024, SIMA 2 leverages Google’s Gemini models to enhance planning, continual learning, and task execution.
Enhanced task reasoning and planning
SIMA 2 can now analyse its actions and determine the steps required to accomplish user-defined objectives. The agent receives a visual feed from a three-dimensional game environment along with instructions such as “build a shelter” or “locate the red house.” It then breaks these goals into smaller, executable actions, utilising inputs similar to a keyboard and mouse. This enables SIMA 2 to translate human instructions into meaningful behaviour based on what it observes on screen.
Improved adaptability in unfamiliar environments
A standout feature of SIMA 2 is its superior performance in new and unfamiliar games. DeepMind tested the system in environments it had never encountered, including Minedojo, a research-focused adaptation of Minecraft, and ASKA, a Viking-themed survival game. SIMA 2 demonstrated better adaptability and higher success rates than the original SIMA, highlighting its ability to learn transferable skills across diverse virtual worlds. The agent can also respond to multimodal prompts, including sketches, emojis, or multiple languages.
Training with human demonstrations and AI-generated annotations
The model is trained using a combination of human demonstrations and automatically generated annotations from Gemini models. Whenever SIMA 2 learns a new skill or movement in an unfamiliar environment, the experience is incorporated back into its training data. This approach reduces reliance on human-labelled datasets and allows the agent to continuously refine its abilities as it explores new scenarios.
Current limitations
Despite its advances, SIMA 2 is not without constraints. DeepMind notes that the system still struggles with long-term memory, complex multi-step reasoning, and highly precise low-level control. These limitations currently prevent the AI from being integrated directly with physical robotics.
Future prospects
DeepMind envisions three-dimensional game environments as testing grounds for AI systems that may eventually control real-world machines. By building agents capable of understanding natural language, planning actions, and executing tasks in complex virtual spaces, the company aims to lay the foundation for general-purpose robots that can operate safely and efficiently in everyday physical settings.
Conclusion
SIMA 2 represents a key step forward in AI research, demonstrating improved reasoning, adaptability, and multimodal interaction in virtual worlds. While challenges remain in memory and fine motor control, the system provides a promising blueprint for the next generation of AI agents that could one day bridge virtual and physical environments.