Google AI program masters classic Atari video games
The Deep Q-network (DQN) developed at London-based AI firm DeepMind, which was acquired by Google last year, can teach itself to play Atari 2600 video games using only the score and the pixel display as input information.
In a study published in the journal Nature, Demis Hassabis and other Google DeepMind collaborators said the program was able to surpass the performance of previous algorithms on most of the 49 games it tested on. It was also able to match the skill of a professional human games tester.
The program did not know the rules beforehand and was armed only with the motivation to maximize its score and the ability to learn from previous gaming sessions. It was able to excel at games including Video Pinball and Breakout, through the use of an algorithm that aims to imitate aspects of human thinking and learning.
The scientists used an approach known as reinforcement learning, which involves offering rewards as motivation for an AI system to learn. They combined that with a kind of artificial neural network, called a deep neural network, which makes use of various computational layers to represent increasingly abstract representations of data.
The team focused on a biologically inspired architecture known as a deep convolutional network, an approach similar to that taken by University of Maryland scientists who have been getting robots to teach themselves to cook by watching videos on YouTube.
The DQN algorithm did well on a variety of games, including side-scrolling shooter games, boxing matches and 3D car racing. It was also able to achieve more than 75 percent of the human score in 29 of the 49 games.
More significantly, however, it was able to learn strategy over many sessions. After 600 sessions playing Breakout, it learned the winning strategy of tunneling behind a wall of bricks that the player must destroy. It repeatedly sent the ball into the tunnel so it bounced around, destroying many bricks.
It's not the first time algorithms have been trained to play video games with minimal input, and DQN fared poorly on games such as Montezuma's Revenge, which requires a long-term planning strategy.
But the researchers said a single architecture has been shown to be able to learn and adapt when faced with various gaming challenges. They view it as another step in building effective, general-purpose AI programs.
"Taken together, our work illustrates the power of harnessing state-of-the-art machine learning techniques with biologically inspired mechanisms to create agents that are capable of learning to master a diverse array of challenging tasks," the authors wrote.
Tim Hornyak covers Japan and emerging technologies for The IDG News Service. Follow Tim on Twitter at @robotopia.