Pong

The Pong demo is the classic two-paddle game. The agent slides the left paddle up and down to return a bouncing ball, and an opponent paddle on the right sends it back. The genetic algorithm evolves a small neural network to control the agent's paddle. Early on the paddle sits still and the first ball slips past. After a few generations the paddle starts shadowing the ball and the rallies stretch out.

It lives alongside the other live-agent games (Flappy Bird, Snake, the Self-Driving Car, the Dino Runner, Breakout, Gem Collector and Road Crossing) under the same Live Agent demo. Use the tabs at the top of the page to switch between them.

The Problem¶

The opponent always returns the ball, so a rally only ends when the agent misses. The agent's whole job is to get its paddle in front of the ball every time it comes back.

The network reads seven inputs: the ball's position and velocity, its paddle's position and speed, and how far the paddle is from where the ball will arrive. One hidden layer of eight neurons feeds three outputs: push up, ease off, or push down. The largest output wins. The paddle has momentum, so a push speeds it up over a few frames and easing off brings it to a stop, the way a hand on a controller moves.

Fitness rewards how long the rally lasts, with a bonus for every successful return. Each time the paddle turns around costs a little, so between two equally safe networks the GA prefers the one that moves with purpose instead of wobbling around the ball. A miss ends the run. Where the ball meets the paddle bends its bounce, and the ball speeds up a little with each hit, so long rallies get gradually harder. The score you see is the number of returns.

How a Run Looks¶

Click Start. The current best network plays a run live, and a panel in the corner shows the three move scores each step with the chosen one highlighted. The chart on the right tracks the best score so far after each cycle.

Generation zero leaves the paddle parked and misses the first ball. Within a few cycles the paddle begins to track the ball, and a strong policy keeps rallies going until the run's step limit.

Playing It Yourself¶

Switch the mode toggle from Agent to You, or press M, to take over. Move the paddle with the up and down arrow keys, or the on-screen up and down buttons on a touch screen. Press P to pause or resume and Esc to reset. Switch back to Agent at any time and the evolved network picks up from where you left off.

Settings Worth Trying¶

Open Settings to tune the genetic algorithm for the next run. These only affect Agent mode:

• Population size. More candidates per generation means more variety to select from, at the cost of a slower cycle. The default is 30.
• Generations per cycle. How much evolving happens between each live run you watch. The default is 8.
• Mutation rate. How much the networks are randomly perturbed each generation. A little keeps the search exploring; too much stops it from settling on a good paddle.

What the Achievement Rewards¶

Signed-in users unlock the Pong achievement the first time an evolved agent makes at least eight returns in one of the runs you watch in Agent mode. Your own rallies in You mode don't count toward it.

Pong records and shares like the other demos. See Sharing and recording for how to capture a clip of a long rally.

For more on how the demos catalog works, see What the Demos Do.