<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=752538731515435&amp;ev=PageView&amp;noscript=1">

Building Neural Nets in Ryu’s Dojo

Artificial neural networks are enabling computers to do amazing things such as reading cursive handwriting, powering autonomous cars, recognizing human emotions from facial expressions, and applying color to black and white images.

Here at Omni Resources, we've also been using neural networks to do amazing things -- to beat our childhood video games.

An artificial neural network (ANN) is a computational model inspired by biological neural networks or in other words, our brains. There are three important parts of an ANN: the inputs, the hidden layer (consisting of N hidden layers), and the outputs. The hidden layer is responsible for converting the inputs into something the output layer can use.

In the below picture, the circles represent neurons and the lines represent synapses. Each layer applies a function to the previous layer to produce an output.
Neural_Networks_1.png

 

 

 

 

 

 

 

For our application, we generated a few random neural networks and had them monitor Ken (player 1, controlled by the neural network) and Ryu (player 2, the computer) on screen by using the memory addresses that relate to their X/Y coordinates. We then mapped those coordinates to an 8x8 grid representation of the video game screen. Those grid coordinates were then used as the inputs for our neural networks. The neural network would process those inputs and then use the corresponding outputs to signal to the game to press buttons.

Neural_Networks_2.png

 

 

 

 




The neural network would play one round of the game, with each round ending when a player lost all health or when the timer reached zero.

At the end of the round, a fitness function is applied to calculate the score of that neural network. Initially, our fitness function was (max Hit Points – Ryu's current HP) * 2 + Ken's current HP but eventually evolved into a complex calculation that would penalize the neural network if certain buttons were pressed too many times (a.k.a., button mashing).

sfai-buttonmash.gif












Once the initial set of neural networks had been tested against the fitness function, we used a genetic algorithm to combine the top scoring networks and create the next generation of neural networks to test.

Unfortunately, mapping the entire screen onto the 8 by 8 grid and using both players’ positions as inputs created a considerable amount of possible inputs for the neural networks. The large amount of inputs led to extremely lengthy generation training times.

In order to reduce the training times, we reduced the grid of inputs to be 5 by 5 and changed Ken's position to be in the center of the grid and Ryu's position to be in relation to Ken's position.

Neural_Networks_3.png











The results of our Street Fighter artificial neural networks:

sfai-topscore.gif











We have future plans to further minimize training times by creating a scalable solution, incorporating recordings of humans playing as Ryu to use to train the neural networks, and possibly training the neural networks against themselves!

If you are interested in learning more about video games and neural networks, check out Mar I/O, our inspiration for this project, and the Lua scripts we based our code on. And for a more beginner overview of machine learning and artificial intelligence, check out our blog post on AlphaGo

Share:
Zach Zoeller

About Author Zach Zoeller

Zach Zoeller is a former Software Engineer with Omni. He has 8+ years in the Information Technology industry with 4 being focused in consulting. He is passionate about technology and enjoys the continuous learning his role provided. He has experience as a full stack developer and has also worked in the mobile space, creating cross-platform apps with Xamarin. Zach graduated from University of Wisconsin-Oshkosh with a bachelor's degree in Business and a minor in Computer Science. You can follow Zach on Twitter @zacharyzoeller.



Disclaimer:

Omni’s blog is intended for informational purposes only. Any views or opinions expressed on this site belong to the authors, and do not represent those held by people or organizations with which Omni is affiliated, unless explicitly stated.

Although we try to the best of our ability to make sure the content of this blog is original, accurate and up-to-date, we make no claims of complete accuracy or completeness of the information on this site/s to which we link. Omni is not liable for any unintended errors or omissions, or for any losses, injuries, or damages from the display or use of this information. We encourage readers to conduct additional research before making decisions based on the information in this blog.