Andy Zeng
Ghostbusters

Teach Pacman how to eat invisible ghosts!

Particle filters or Sequential Monte Carlo (SMC) methods are a set of on-line posterior density estimation algorithms that estimate the posterior density of the state-space by directly implementing the Bayesian recursion equations. SMC methods use a grid-based approach, and use a set of particles to represent the posterior density. These filtering methods make no restrictive assumption about the dynamics of the state-space or the density function. SMC methods provide a well-established methodology for generating samples from the required distribution without requiring assumptions about the state-space model or the state distributions. The state-space model can be non-linear and the initial state and noise distributions can take any form required. However, these methods do not perform well when applied to high-dimensional systems. SMC methods implement the Bayesian recursion equations directly by using an ensemble based approach. The samples from the distribution are represented by a set of particles; each particle has a weight assigned to it that represents the probability of that particle being sampled from the probability density function. Weight disparity leading to weight collapse is a common issue encountered in these filtering algorithms; however it can be mitigated by including a resampling step before the weights become too uneven. In the resampling step, the particles with negligible weights are replaced by new particles in the proximity of the particles with higher weights. This project includes Pacman agents that use sensors to locate and eat invisible ghosts using inference and particle filtering algorithms. Goal? Advance from locating single, stationary ghosts to hunting packs of multiple moving ghosts with ruthless efficiency.

Bayes' Nets and Exact Inference Observation

Notes:
- High posterior beliefs are represented by bright colors, while low beliefs are represented by dim colors. Pacman starts with a large cloud of belief that shrinks over time as more evidence accumulates.
- Correctly updates the agent's belief distribution over ghost positions given an observation from Pacman's sensors.
- Special case: when a ghost is eaten, the ghost will be sent to its prison cell.
- Squares converge to their final coloring after a number of updates.
- The Pacman agent has a separate inference module for each ghost it is tracking.
- Implemented the online belief update after observing new evidence. Note that before any readings, Pacman believes the ghost could be anywhere, hence distribution follows a uniform prior.
- No evidence values are stored; only beliefs.





Exact Inference Time Elapse

Notes:
- Previously, implemented belief updates for Pacman are based only on his observations. Realize that Pacman may also have knowledge about the ways that a ghost may move that we can take advantage of; namely that the ghost can not move through a wall or more than one space in one timestep.
- Consider the following scenario in which there is Pacman and one Ghost: Pacman receives many observations which indicate the ghost is very near, but then one which indicates the ghost is very far. The reading indicating the ghost is very far is likely to be the result of a buggy sensor. Pacman's prior knowledge of how the ghost may move will decrease the impact of this reading since Pacman knows the ghost could not move so far in only one move.
- Goal: to estimate a ghost's position given only its action distribution.
- Since Pacman is not utilizing any observations about the ghost, this means that Pacman will start with a uniform distribution over all spaces, and then update his beliefs according to how he knows the Ghost is able to move. Since Pacman is not observing the ghost, this means the ghost's actions will not impact Pacman's beliefs. Over time, Pacman's beliefs will come to reflect places on the board where he believes ghosts are most likely to be given the geometry of the board and what Pacman already knows about their valid movements.
- Lighter squares indicate that pacman believes a ghost is more likely to occupy that location, and darker squares indicate a ghost is less likely to occupy that location.
- The ghost in the right figure tends to move south so over time, and without any observations, notice how Pacman's belief distribution should begin to focus around the bottom of the board.


Exact Inference Full System

Notes:
- Implemented simple greedy search strategy - Pacman assumes that each ghost is in its most likely position according to its beliefs, then moves toward the closest ghost.
- Pacman finds the most likely position of each remaining (uncaptured) ghost, then choose an action that minimizes the distance to the closest ghost.

Approximate Inference

- Implemented similarly to exact, but provides much more noisy results.
- Special case: When all particles receive zero weight based on the evidence, resample all particles from the prior to recover.
- After implementation of approximate inference with time elapse, results are as effective as with exact inference.







Joint Particle Filter Observation

Notes:
- Implemented to find ghosts that choose actions dependent on the positions of other ghosts. Since the ghosts' transition models are no longer independent, all ghosts must be tracked jointly in a dynamic Bayes net.
- The Bayes net has the structure to the right, where the hidden variables G represent ghost positions and the emission variables E are the noisy distances to each ghost. This structure can be extended to more ghosts, but only two (a and b) are shown below.
- Each particle represents a tuple of ghost positions that is a sample of where all the ghosts are at the present time.
- Extracts marginal distributions about each ghost from the joint inference algorithm, so that belief clouds about individual ghosts can be displayed.
- Whole lists of particles are weighted and resampled based on new evidence.

Join Particle Filter Elapse Time

Notes:
- Using the same skeleton from the joint particle filter implemented above, particles are now resampled for the Bayes net.
- Each ghost draws a new position conditioned on the positions of all the ghosts at the previous time step.
- Notice that Pacman knows the ghosts will move to the sides of the gameboard.