EvolutionaryRNN

Using genetic algorithms and recurrent neural networks to evolve a sensory based agent in the task of navigating an unknown environment.

Overview

The agent is set in an unknown environment and given a destination. To get to the destination the agent must learn to navigate the environment purely based on its senses. These senses include:

• x, y scalar distance between the agent's current position and the destination
• 4 values set to 1 or 0 whether there is an adjacent block or not
• 2 inputs set to the last move taken in both x and y (optional)

These 6 (or 8) inputs are then used as an input to a Recurrent Neural Network. This network has 4 distinct outputs actions being the cardinal directions the block should move per turn. This model is then trained using a genetic algorithm to find the best possible solution.

Fitness function

The fitness function score is calculated by: SCORE = (MAX_TURNS - DISTANCE) * 0.95^MOVES_TAKEN

Where:

• DISTANCE = Manhattan distance between the agent's position and destination at the end of an episode
• MAX_TURNS = Maximum amount of turns per episode
• MOVES_TAKEN = Moves taken to get to the goal (Used to promote more direct solutions)

Running the project

To run the project use:

$ javac Main.java
$ java Main

Keyboard inputs:

• X - End training early and display best agent
• N - (hold) Display neural network of best agent
• P - (hold) Display current best agent
• SPACE - Save best agent's weights to out.txt once training is finished

Some hyper parameters in main.java and their descriptions:

int[] nShape      //Shape of neural network per-layer
int GENERATIONS   //Amount of generations to simulate
int TURNS         //Maximum turns per episode
int SAMPLES       //Amount of agents to simulate per generation
float MUTATION    //Rate of mutation of the agents
int MAP_SIZE      //Size of map
String INPUT      //Optional input for weight initialization, leave as "" for random initial values
String MAPNAME    //Name of .png map to be loaded
boolean TRAIN     //Train new generations or just display result

Java version: 1.8.0_121

Picture.java and StdDraw.java are used purely as a visual interface and input. Source: https://introcs.cs.princeton.edu/java/stdlib/