Pitt researcher uses video games to unlock new levels of A.I.

November 05, 2018

PITTSBURGH (November 5, 2018) ... Expectations for artificial intelligences are very real and very high. An analysis in Forbes projects revenues from A.I. will skyrocket from $1.62 billion in 2018 to $31.2 billion in 2025. The report also included a survey revealing 84 percent of enterprises believe investing in A.I. will lead to competitive advantages.

"It is exciting to see the tremendous successes and progress made in recent years," says Daniel Jiang, assistant professor of industrial engineering at the University of Pittsburgh Swanson School of Engineering. "To continue this trend, we are looking to develop more sophisticated methods for algorithms to learn strategies for optimal decision making."

Dr. Jiang designs algorithms that learn decision strategies in complex and uncertain environments. By testing algorithms in simulated environments, they can learn from their mistakes while discovering and reinforcing strategies for success. To perfect this process, Dr. Jiang and many researchers in his field require simulations that mirror the real world.

"As industrial engineers, we typically work on problems with an operational focus. For example, transportation, logistics and supply chains, energy systems and health care are several important areas," he says. "All of those problems are high-stakes operations with real-world consequences. They don't make the best environments for trying out experimental technologies, especially when many of our algorithms can be thought of as clever ways of repeated 'trial and error' over all possible actions."

One strategy for preparing advanced A.I. to take on real-world scenarios and complications is to use historical data. For instance, algorithms could run through decades' worth of data to find which decisions were effective and which led to less than optimal results. However, researchers have found it difficult to test algorithms that are designed to learn adaptive behaviors using only data from the past.

Dr. Jiang explains, "Historical data can be a problem because people's actions fix the consequences and don't present alternative possibilities. In other words, it is difficult for an algorithm to ask the question 'how would things be different if I chose door B instead of door A?' In historical data, all we can see are the consequences of door A."

Video games, as an alternative, offer rich testing environments full of complex decision making without the dangers of putting an immature A.I. fully in charge. Unlike the real world, they provide a safe way for an algorithm to learn from its mistakes.

"Video game designers aren't building games with the goal to test models or simulations," Dr. Jiang says. "They're often designing games with a two-fold mission: to create environments that mimic the real world and to challenge players to make difficult decisions. These goals happen to align with what we are looking for as well. Also, games are much faster. In a few hours of real time, we can evaluate the results of hundreds of thousands of gameplay decisions."

To test his algorithm, Dr. Jiang used a genre of video games called Multiplayer Online Battle Arena or MOBA. Games such as League of Legends or Heroes of the Storm are popular MOBAs in which players control one of several "hero" characters and try to destroy opponents' bases while protecting their own.

A successful algorithm for training a gameplay A.I. must overcome several challenges, such as real-time decision making and long decision horizons--a mathematical term for when the consequences of some decisions are not known until much later.

"We designed the algorithm to evaluate 41 pieces of information and then output one of 22 different actions, including movement, attacks and special moves," says Dr. Jiang. "We compared different training methods against one another. The most successful player used a method called Monte Carlo tree search to generate data, which is then fed into a neural network."

Monte Carlo tree search is a strategy for decision making in which the player moves randomly through a simulation or a video game. The algorithm then analyzes the game results to give more weight to more successful actions. Over time and multiple iterations of the game, the more successful actions persist, and the player becomes better at winning the game.

"Our research also gave some theoretical results to show that Monte Carlo tree search is an effective strategy for training an agent to succeed at making difficult decisions in real-time, even when operating in an uncertain world," Dr. Jiang explains.

Dr. Jiang published his research in a paper co-authored with Emmanuel Ekwedike and Han Liu and presented the results at the 2018 International Conference on Machine Learning in Stockholm, Sweden this past summer.

At the University of Pittsburgh, he continues to work in the area of sequential decision making with Ph.D. students Yijia Wang and Ibrahim El-Shar. The team focuses on problems related to ride-sharing, energy markets, and public health. As industries prepare to put A.I. in charge of critical responsibilities, Dr. Jiang ensures the underlying algorithms stay at the top of their game.

University of Pittsburgh

Related Video Games Articles from Brightsurf:

Video games improve the visual attention of expert players
Long-term experiences of action real-time strategy games leads to improvements in temporal visual selective attention.

Study questions video games' effects on violent behavior
A new Contemporary Economic Policy study finds that there is not enough information to support the claim that violent video games lead to acts of violence.

Do video games drive obesity?
Are children, teenagers and adults who spend a lot of time playing video games really more obese?

DeepMind's new gamer AI goes 'for the win' in multiplayer first-person video games
DeepMind researchers have taught artificially intelligent gamers to play a popular 3D multiplayer first-person video game with human-like skills -- a previously insurmountable task.

How does dark play impact the effectiveness of serious video games?
A new study has shown that allowing ''dark play'' in a serious video game intended to practice skills transferable to a real-life setting does not impact the game's effectiveness.

Study: Collaborative video games could increase office productivity
Move over trust falls and ropes courses, turns out playing video games with coworkers is the real path to better performance at the office.

Pitt researcher uses video games to unlock new levels of A.I.
Dr. Jiang designs algorithms that learn decision strategies in complex and uncertain environments like video games.

For blind gamers, equal access to racing video games
Computer Scientist Brian A. Smith has developed the RAD -- a racing auditory display -- to enable visually impaired gamers play the same types of racing games that sighted players play with the same speed, control, and excitement as sighted players.

Video games to improve mobility after a stroke
A joint research by the Basque research center BCBL and the London Imperial College reveals that, after a cerebral infarction, injuries in areas that control attention also cause motility problems.

No evidence to support link between violent video games and behaviour
Researchers at the University of York have found no evidence to support the theory that video games make players more violent.

Read More: Video Games News and Video Games Current Events
Brightsurf.com is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com.