Gem Collector

The Gem Collector demo puts an agent on a small grid with a gem somewhere on it. Reach the gem and a new one appears in another spot, so the agent keeps navigating. The genetic algorithm evolves a small neural network to steer. Early agents drift into the edges. After a few generations they head straight for each gem and route around the blocks in the way.

It lives alongside the other live-agent games (Flappy Bird, Snake, the Self-Driving Car, Dino Runner, Breakout, Pong and Road Crossing) under the same Live Agent demo. Use the tabs at the top of the page to switch between them.

The Problem¶

The grid is 11 by 11 cells and the agent starts near the top-left corner. Single-cell blocks are scattered across it, 12 by default. They are always spaced apart, never touching each other even at a corner, so no cell can ever be boxed in. Every Start lays the blocks out in a fresh random arrangement, which then stays the same for that run.

The agent moves one cell up, right, down or left each step. The network reads six inputs: whether each of the four neighbouring cells is blocked (the grid edge counts as blocked), and whether the gem is to the left or right and above or below. One hidden layer of eight neurons feeds the four move outputs, and the largest one wins. The network has no memory and doesn't know where on the grid it is, so it has to learn a general way of reaching a gem rather than a fixed route.

Fitness gives a big bonus for each gem, a little for every step that gets closer and a little less for every step that moves away. Bumping into a block or the edge wastes the step and costs a small penalty, and stepping straight back onto the cell it just left costs a bit more, which discourages jittering back and forth. A run ends after six wasted steps in a row, after 80 steps without reaching a gem, or after 484 steps in total.

How a Run Looks¶

Click Start. The current best network plays a run live, and a panel in the corner shows the four move scores each step with the chosen one highlighted. The chart on the right tracks the best score, which is the number of gems collected, for each cycle.

The gems move to new spots every cycle while the blocks stay put, and each cycle trains and replays on the same gem positions. Reaching a new set of gems each time is how you can tell the agent learned to navigate rather than memorise one path.

Playing It Yourself¶

Switch the mode toggle from Agent to You, or press M, to take over. Move with the arrow keys or the on-screen pad. The agent only moves when you press a direction, and holding a key keeps it moving. In You mode there is no idle time-out, so you can stop and think as long as you like. Press P to pause or resume and Esc to reset. Switch back to Agent at any time and the evolved network picks up from where you left off.

Settings Worth Trying¶

Open Settings to tune the next run. These only affect Agent mode:

• Number of blocks. From 0 (an open grid) to 25. More blocks means more routing around obstacles.
• Population size. More agents per generation means more variety to select from, at the cost of a slower cycle. The default is 30.
• Generations per cycle. How much evolving happens between each live run you watch, from 1 to 15. The default is 10.
• Mutation rate. How much the networks are randomly perturbed each generation. A little keeps the search exploring; too much stops it from settling on a good policy.

What the Achievement Rewards¶

Signed-in users unlock the Gem Collector achievement the first time an evolved agent collects at least five gems in one of the runs you watch in Agent mode. Your own runs in You mode don't count toward it.

Gem Collector records and shares like the other demos. See Sharing and recording for how to capture a clip of a good run.

For more on how the demos catalog works, see What the Demos Do.