Artificial neural networks are enabling computers to do amazing things such as reading cursive handwriting, powering autonomous cars, recognizing human emotions from facial expressions, and applying color to black and white images.
Here at Omni Resources, we've also been using neural networks to do amazing things -- to beat our childhood video games.
An artificial neural network (ANN) is a computational model inspired by biological neural networks or in other words, our brains. There are three important parts of an ANN: the inputs, the hidden layer (consisting of N hidden layers), and the outputs. The hidden layer is responsible for converting the inputs into something the output layer can use.
In the below picture, the circles represent neurons and the lines represent synapses. Each layer applies a function to the previous layer to produce an output.
For our application, we generated a few random neural networks and had them monitor Ken (player 1, controlled by the neural network) and Ryu (player 2, the computer) on screen by using the memory addresses that relate to their X/Y coordinates. We then mapped those coordinates to an 8x8 grid representation of the video game screen. Those grid coordinates were then used as the inputs for our neural networks. The neural network would process those inputs and then use the corresponding outputs to signal to the game to press buttons.
The neural network would play one round of the game, with each round ending when a player lost all health or when the timer reached zero.
At the end of the round, a fitness function is applied to calculate the score of that neural network. Initially, our fitness function was (max Hit Points – Ryu's current HP) * 2 + Ken's current HP but eventually evolved into a complex calculation that would penalize the neural network if certain buttons were pressed too many times (a.k.a., button mashing).
Once the initial set of neural networks had been tested against the fitness function, we used a genetic algorithm to combine the top scoring networks and create the next generation of neural networks to test.
Unfortunately, mapping the entire screen onto the 8 by 8 grid and using both players’ positions as inputs created a considerable amount of possible inputs for the neural networks. The large amount of inputs led to extremely lengthy generation training times.
In order to reduce the training times, we reduced the grid of inputs to be 5 by 5 and changed Ken's position to be in the center of the grid and Ryu's position to be in relation to Ken's position.
The results of our Street Fighter artificial neural networks:
We have future plans to further minimize training times by creating a scalable solution, incorporating recordings of humans playing as Ryu to use to train the neural networks, and possibly training the neural networks against themselves!
If you are interested in learning more about video games and neural networks, check out Mar I/O, our inspiration for this project, and the Lua scripts we based our code on. And for a more beginner overview of machine learning and artificial intelligence, check out our blog post on AlphaGo.