COS 526: Advanced
General | Syllabus | Assignments
Assignment 1: Textures and Seams (due Thursday October 2)
This assignment deals with images: texture synthesis, in-painting,
and resizing. For the assignment you may make use of any library
for representing, reading and writing images, for example the
skeleton code for the image
processing assignment for COS 426.
You may also use any other source code as reference material,
and also as library source, provided it doesn't directly
solve the assigned problems. When you use other people's sources,
please reference it in your writeup.
You program should run from the command line, taking one or more
image file names and a set of parameters on the command line. It
should read and write the files based on those parameters.
You may create input examples any way you choose, using
Photoshop or any other image-editing software (especially
for mattes), grabbing them from the web or wherever.
Part 1: Texture Synthesis
and Leung style texture synthesis, with the following features:
- Basic method: Synthesize larger examples from smaller examples.
Replicate the examples shown
Also try some of the different kinds of textures shown
can also find many examples on the web.
- Constrained synthesis: Extrapolate from a smaller rectangular image to fill a border area or a hole, as shown at the bottom of the page
Try using the technique for
to remove some "scratches" you put in a few photographs (by drawing on them
in a drawing program, and creating a matte that identifies the scratches).
- Optional: Try to accelerate this very slow method somehow.
Some ideas: When finding good matches, only compare against a small
random subset of all patches; scan pixels in order and remember the
best matches from the previous pixel, while also considering a some
new randomly chosen locations. Or, try to accelerate the search for
a good match by some kind of high-dimensional neighborhood search
structure, such as K-D trees or vector quantization.
Part 2: Seam Carving
Implement the seam carving method of
Avidan and Shamir
for content aware image resizing.
- Basic method: make a command line tool that reads in
a file and writes out a series of files that change the
aspect ratio. The command line parameters should specify
how the image shape should change and how many frames to
write out. Only consider cases that make the image smaller in
one dimension and keep the other dimension fixed.
Suggestion: only handle on "vertical" cuts; to handle horizontal
cuts rotate the image before (and after) by 90 degrees.
- Optional stretching: also allow the image to become larger.
- Optional arbitrary size: allow the image to change in both dimensions.
- Optional protection: also incorporate a matte that specifies
regions that should not contain seams (e.g. faces).
- Optional movie: use any video sofware (e.g. ffmpeg
to take the output sequence turn it into a short video clip.
- Optional fancy interface: if you want to get fancy
with the interface, make a GUI that allows you to drag
the window borders.
Part 3: Texture synthesis using image quilting
- Warm up "chaos mosaic": implement the method of Xu et al
This method simply pastes rectangles randomly from the source into the
target. Do not bother with the specific random patterns they describe --
just use uniform random patch placement, ideally with a little blending
at the edges.
- Basic image quilting:
Implement the image quilting texture synthesis scheme of
and Freeman. This uses dynamic
programming to do the minimal-error bounding cut at the boundaries.
There are some examples of source code available on the web, for
- Optional: do something like Efros and Freeman's texture transfer.
Part 4: (optional) Art Contest
In your writeup you may note one or two example images from your
program that you wish to submit to our "art contest". They may
be technically cool, or weird, or beautiful. We'll review the art
contest images in class.