Home Google DeepMind just introduced an AI agent that plays video games with you

Google DeepMind just introduced an AI agent that plays video games with you


  • Google's DeepMind unveils SIMA, an AI agent designed to learn and play video games like humans.
  • SIMA is trained to follow instructions and mimic human gameplay behaviors for a natural gaming experience.
  • Developed with game developers, SIMA learns basic skills and language cues to tackle more complex tasks.

Imagine playing your favorite RPG, but instead of battling computerized enemies hellbent on defeating you all by yourself, you have a mighty companion – an advanced artificial intelligence (AI) agent that learns and behaves like a human player.

On Wednesday (Mar.13) Google’s AI research subsidiary DeepMind unveiled SIMA, a revolutionary AI agent designed to learn and play video games in a manner akin to human players. Unlike traditional game AI that is programmed to win at all costs, SIMA is trained to run games and follow instructions, mimicking the natural gaming experience of a human participant.

How does Google DeepMind’s SIMA work?

SIMA – which stands for Scalable, Instructable, Multiworld Agent – is currently in the research phase, but it’s a significant step toward developing general, capable AI agents. The agent is being developed to eventually learn how to play any video game, including open-world games and those without a linear path to completion.

“SIMA isn’t trained to win a game; it’s trained to run it and do what it’s told,” explained Tim Harley, a Google DeepMind researcher and co-lead on the SIMA project. “It mixes natural language instruction with understanding 3D worlds and image recognition.”

To train SIMA, Google DeepMind collaborated with eight game developers, including studios like Hello Games, Embracer, Tuxedo Labs, and Coffee Stain. The researchers plugged SIMA into various games, such as No Man’s Sky, Teardown, Valheim, and Goat Simulator 3, to teach the AI agent the basics of gameplay.

The training process involved multiple steps, including building a custom environment in the Unity engine where agents learned object manipulation by creating sculptures.

Then researchers recorded pairs of human players, one controlling the game and the other providing instructions, to capture language cues. This data was then fed to SIMA to help it predict on-screen events based on actions and instructions.

But there’s still a long walk to go before SIMA can be rolled out. It’s learning to walk before it can run.

Currently, SIMA has mastered around 600 basic skills, such as turning left, climbing ladders, and opening in-game menus. However, the team envisions that SIMA could eventually be instructed to perform more complex functions within games, like finding resources and building camps.

Who knows, SIMA may one day become the perfect AI party member, helping gamers to victory (or would that be cheating?) while providing a truly human-like gaming experience.

Featured image: Generated by Ideogram

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Sam Shedden
Executive Editor

Sam Shedden is an experienced journalist and editor with over a decade of experience in online news. A seasoned technology writer and content strategist, he has contributed to many UK regional and national publications including The Scotsman, inews.co.uk, nationalworld.com, Edinburgh Evening News, The Daily Record and more. Sam has written and edited content for audiences whose interests include media, technology, AI, start-ups and innovation. He's also produced and set-up email newsletters in numerous specialist topics in previous roles and his work on newsletters saw him nominated as Newsletter Hero Of The Year at the UK's Publisher Newsletter Awards 2023. He…

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.