COS 526: Advanced
Computer Graphics
Fall 2008

General | Syllabus | Assignments

Assignment 1: Textures and Seams (due Thursday October 2)

This assignment deals with images: texture synthesis, in-painting, and resizing. For the assignment you may make use of any library for representing, reading and writing images, for example the skeleton code for the image processing assignment for COS 426.

You may also use any other source code as reference material, and also as library source, provided it doesn't directly solve the assigned problems. When you use other people's sources, please reference it in your writeup.

You program should run from the command line, taking one or more image file names and a set of parameters on the command line. It should read and write the files based on those parameters. You may create input examples any way you choose, using Photoshop or any other image-editing software (especially for mattes), grabbing them from the web or wherever.

Part 1: Texture Synthesis

Implement the Efros and Leung style texture synthesis, with the following features:
  1. Basic method: Synthesize larger examples from smaller examples. Replicate the examples shown here. Also try some of the different kinds of textures shown here. You can also find many examples on the web.
  2. Constrained synthesis: Extrapolate from a smaller rectangular image to fill a border area or a hole, as shown at the bottom of the page here. Try using the technique for in-painting to remove some "scratches" you put in a few photographs (by drawing on them in a drawing program, and creating a matte that identifies the scratches).
  3. Optional: Try to accelerate this very slow method somehow. Some ideas: When finding good matches, only compare against a small random subset of all patches; scan pixels in order and remember the best matches from the previous pixel, while also considering a some new randomly chosen locations. Or, try to accelerate the search for a good match by some kind of high-dimensional neighborhood search structure, such as K-D trees or vector quantization.

Part 2: Seam Carving

Implement the seam carving method of Avidan and Shamir for content aware image resizing.
  1. Basic method: make a command line tool that reads in a file and writes out a series of files that change the aspect ratio. The command line parameters should specify how the image shape should change and how many frames to write out. Only consider cases that make the image smaller in one dimension and keep the other dimension fixed. Suggestion: only handle on "vertical" cuts; to handle horizontal cuts rotate the image before (and after) by 90 degrees.
  2. Optional stretching: also allow the image to become larger.
  3. Optional arbitrary size: allow the image to change in both dimensions.
  4. Optional protection: also incorporate a matte that specifies regions that should not contain seams (e.g. faces).
  5. Optional movie: use any video sofware (e.g. ffmpeg or mencoder) to take the output sequence turn it into a short video clip.
  6. Optional fancy interface: if you want to get fancy with the interface, make a GUI that allows you to drag the window borders.

Part 3: Texture synthesis using image quilting

  1. Warm up "chaos mosaic": implement the method of Xu et al (PDF). This method simply pastes rectangles randomly from the source into the target. Do not bother with the specific random patterns they describe -- just use uniform random patch placement, ideally with a little blending at the edges.
  2. Basic image quilting: Implement the image quilting texture synthesis scheme of Efros and Freeman. This uses dynamic programming to do the minimal-error bounding cut at the boundaries. There are some examples of source code available on the web, for example here and here.
  3. Optional: do something like Efros and Freeman's texture transfer.

Part 4: (optional) Art Contest

In your writeup you may note one or two example images from your program that you wish to submit to our "art contest". They may be technically cool, or weird, or beautiful. We'll review the art contest images in class.