COS 429 - Computer Vision

Fall 2019

Assignment 2: Face Detection and Model Fitting

Part III. Single-Scale Face Detection

Sliding-window: Now that you have a face classifier trained, the next step is to run the classifier on a bunch of locations within an image. A "sliding window" detector does this by brute force: it considers every (36x36, in our case) window of pixels, computes the HoG descriptor on it, and runs it through the classifier. In practice, advancing the detection window by one pixel at a time is very slow and probably unnecessary. So, you can have a "stride" parameter, which is how much the window should be advanced each time. For example, for a stride of 3, you would consider windows starting at (1,1), (4,1), (7,1), ..., (1,4), (4,4), (7,4), etc.

The output of this detector is a probability map similar to the one shown below.

Input image	Face probability map

You can now do the usual sorts of postprocessing: simple thresholding, nonmaximum suppression, etc. Here are the faces detected by thresholding at a probability of 0.95 (with stride = 3):

Notice the multiple overlapping detections: nonmaximum suppression would be necessary to get rid of them. Also notice the false-positive detections on Lurch's shirt. The shirt shows up quite strongly in the probability map, and the training data contains a number of low-contrast faces, so it might be difficult to eliminate this false positive.

Do this:

• First, in order to avoid re-training the classifier every time, save the results of your face classifier after you have run it for the final time, preferably with a large training set size. A simple way of doing this is to add the following line to test_face_classifier.py from part II, at the end of function test_face_classifier:

  np.savez('face_classifier.npz', params=params, orientations=orientations, wrap180=wrap180)

  These variables can then be read back via np.load('face_classifier.npz'), as is done by the starter code below.

• Next, download starter code and dataset (about 600 kB) for this part. It contains the following files:
  • find_faces_single_scale.py - you will modify this to perform the sliding-window detection.
  • test_single_scale.py - this function calls the above detector on all test images, and displays the results.
  • face_data/single_scale_scenes/*.jpg - 83 images from the CMU/MIT frontal face image database, rescaled such that the faces match our training scale of 36x36 pixels.
• Once you download the code, copy over logistic_prob.py and hog36.py from Part II, as well as your saved face_classifier.npz.

Do this and turn in:

1. Implement find_faces_single_scale.py, and test it on an image, like so:

   import cv2
   import numpy as np
   import matplotlib.pyplot as plt
   from find_faces_single_scale import find_faces_single_scale
   img = cv2.imread('face_data/single_scale_scenes/addams-family.jpg', cv2.IMREAD_GRAYSCALE)
   saved = np.load('face_classifier.npz')
   params, orientations, wrap180 = saved['params'], saved['orientations'], saved['wrap180']
   outimg, probmap = find_faces_single_scale(img, 3, 0.95, params, orientations, wrap180)
   plt.figure(1)
   plt.title('outimg')
   plt.imshow(outimg, cmap='gray')
   plt.show()
   plt.figure(2)
   plt.title('probmap')
   plt.imshow(probmap, cmap='gray')
   plt.show()

   Submit your file find_faces_single_scale.py
2. Once you have a working detector, run test_single_scale.py to get outputs on all test images. Save and submit your face detection results for 2 images showing particularly good and 2 images showing particularly bad performance.

Optional extra credit (up to 2 points each):
In your writeup, clearly specify which extra credit items you attempted. Be sure to submit all code, along with a description of your method and an analysis of the results. The amount of credit will depend on the sophistication and thoroughness of your implementation and analysis.

3. Implement nonmaximum suppression to eliminate overlapping detections.
4. Run the detector with a stride of 1 just in the vicinity of detected faces, in order to localize the maximum more precisely.
5. Go back and improve your scheme for selecting negative examples for your face classifier. Perhaps use more negative examples? Perhaps deliberately select negative examples that are close to the decision boundary?

• On to Part IV
• Back to the main page for Assignment 2