AI uses less than two minutes of videogame footage to recreate game engine

September 11, 2017

Game studios and enthusiasts may soon have a new tool at their disposal to speed up game development and experiment with different styles of play. Georgia Institute of Technology researchers have developed a new approach using an artificial intelligence to learn a complete game engine, the basic software of a game that governs everything from character movement to rendering graphics.

Their AI system watches less than two minutes of gameplay video and then builds its own model of how the game operates by studying the frames and making predictions of future events, such as what path a character will choose or how enemies might react.

To get their AI agent to create an accurate predictive model that could account for all the physics of a 2D platform-style game, the team trained the AI on a single "speedrunner" video, where a player heads straight for the goal. This made "the training problem for the AI as difficult as possible."

Their current work uses Super Mario Bros. and they've started replicating the experiments with Mega Man and Sonic the Hedgehog as well. The same team first used AI and Mario Bros. gameplay video to create unique game level designs.

The researchers found that their game engine predicted video frames significantly more similar to those in the original game when compared to the same test on a neural network. This gave them an accurate, general model of a game using only the video footage.

"Our AI creates the predictive model without ever accessing the game's code, and makes significantly more accurate future event predictions than those of convolutional neural networks," says Matthew Guzdial, lead researcher and Ph.D. student in computer science. "A single video won't produce a perfect clone of the game engine, but by training the AI on just a few additional videos you get something that's pretty close."

They next tested how well the cloned engine would perform in actual gameplay. They employed a second AI to play the game level and ensure the game's protagonist wouldn't fall through solid floors or go undamaged if hit by an enemy.

The results: the AI playing with the cloned engine proved indistinguishable compared to an AI playing the original game engine.

"The technique relies on a relatively simple search algorithm that searches through possible sets of rules that can best predict a set of frame transitions," says Mark Riedl, associate professor of Interactive Computing and co-investigator on the project. "To our knowledge this represents the first AI technique to learn a game engine and simulate a game world with gameplay footage.

The current cloning technique works well with games where much of the action happens on-screen. Guzdial says Clash of Clans or other games with action taking place off-screen might prove difficult for their system.

"Intelligent agents need to be able to make predictions about their environment if they are to deliver on the promise of advancing different technology applications," he says. "Our model can be used for a variety of tasks in training or education scenarios, and we think it will scale to many types of games as we move forward."
-end-
The research was presented at the International Joint Conference on Artificial Intelligence, Aug. 19-25, in Melbourne, Australia. The paper, "Game Engine Learning from Video," was authored by Matthew Guzdial, Boyang Li, and Mark Riedl.

Georgia Institute of Technology

Related Artificial Intelligence Articles from Brightsurf:

Physics can assist with key challenges in artificial intelligence
Two challenges in the field of artificial intelligence have been solved by adopting a physical concept introduced a century ago to describe the formation of a magnet during a process of iron bulk cooling.

A survey on artificial intelligence in chest imaging of COVID-19
Announcing a new article publication for BIO Integration journal. In this review article the authors consider the application of artificial intelligence imaging analysis methods for COVID-19 clinical diagnosis.

Using artificial intelligence can improve pregnant women's health
Disorders such as congenital heart birth defects or macrosomia, gestational diabetes and preterm birth can be detected earlier when artificial intelligence is used.

Artificial intelligence (AI)-aided disease prediction
Artificial Intelligence (AI)-aided Disease Prediction https://doi.org/10.15212/bioi-2020-0017 Announcing a new article publication for BIO Integration journal.

Artificial intelligence dives into thousands of WW2 photographs
In a new international cross disciplinary study, researchers from Aarhus University, Denmark and Tampere University, Finland have used artificial intelligence to analyse large amounts of historical photos from WW2.

Applying artificial intelligence to science education
A new review published in the Journal of Research in Science Teaching highlights the potential of machine learning--a subset of artificial intelligence--in science education.

New roles for clinicians in the age of artificial intelligence
New Roles for Clinicians in the Age of Artificial Intelligence https://doi.org/10.15212/bioi-2020-0014 Announcing a new article publication for BIO Integration journal.

Artificial intelligence aids gene activation discovery
Scientists have long known that human genes are activated through instructions delivered by the precise order of our DNA.

Artificial intelligence recognizes deteriorating photoreceptors
A software based on artificial intelligence (AI), which was developed by researchers at the Eye Clinic of the University Hospital Bonn, Stanford University and University of Utah, enables the precise assessment of the progression of geographic atrophy (GA), a disease of the light sensitive retina caused by age-related macular degeneration (AMD).

Classifying galaxies with artificial intelligence
Astronomers have applied artificial intelligence (AI) to ultra-wide field-of-view images of the distant Universe captured by the Subaru Telescope, and have achieved a very high accuracy for finding and classifying spiral galaxies in those images.

Read More: Artificial Intelligence News and Artificial Intelligence Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.