COS 426: Computer Graphics Spring 2015

Overview

In this assignment you will create a simple image processing program. The operations that you implement will mostly be filters that take an input image, process the image, and produce an output image.

Getting Started

The image processing program has two modes: (1) an interactive mode where you can enable/disable various filters, adjust parameters to these filters, and see the result right away; and (2) a batch mode where all the filters and parameters are fixed. Each version run in a web page, and for the batch mode the filters and parameters are specified in the URL.

To get started, download this zip file and unzip it on your computer. Change to the subdirectory cos426-assign1 and run the command python -m SimpleHTTPServer which will launch a web server on your computer, rooted at this directory. Note that our assignments will not work if you simply open the html file in the browser directly using file://... due to the same-origin security policy. Therefore we run a local web server using python. Other options are discussed here.

Once you have started the local web server, direct your browser to http://localhost:8000 (8000 is the default port used by the python web server). You should see the web page for Assignment 1 showing a picture of a flower. In the upper right corner is a GUI, and the top control lets you choose a picture -- try it. Below that is a collection of controls. Click on the LUMINANCE OPERATIONS folder to open it, and choose the top item -- brightness. Try adjusting the brightness by pulling that slider around. It should change the brightness of the image (if not you have probably bumped up against the same-origin policy described above). Try adjusting the next slider down (contrast) and you should see a warning that that feature is not implemented yet. You will be implementing that feature as part of the assignment. Click ok on the warning box, and then "reset" at top of the GUI which will reset all the parameters to their default values.

Using your favorite text/code editor, edit the file js/student.js in that directory and fill in your name and NetID. Reload the web page in your browser, and now your information should appear above the image.

Now open the COLOR OPERATIONS folder and check the "gray" box. The image should turn gray. Adjust brightness again and you can see that both the grayscale filter and the brightness are active. These are applied in the order they appear in the GUI from top to bottom. The GUI only executes the filters where the parameter setting is different from the default. Now check the "exclusive" box just under "reset" and adjust brightness. Notice that only the brightness is adjusted and the image is no longer gray -- this feature means only one filter will be active at a time.

Set the brightness to 0.6. Now open the BATCH MODE folder, and click the top button guiToBatch. This opens a new tab in your browser and in this tab will be the "batch mode" version of the web page -- all filters are expressed in the URL. Notice it specifies brightness 0.6. Try changing the URL so that it sets brightness to -0.5. In the interactive GUI you can always open a batch version of the program with the current filter settings using the guiToBatch button. Close the batch mode tab and return to the tab that has the interactive mode. In the BATCH MODE folder click the brightnessAnim button. This opens a batch mode in a new tab but it will take a little while for it to compute. It mades an animated gif where it varies the brightness using brightness=(-1:0.2:1:100) -- meaning brightness starts at -1 (black), it iterates up to 1 (white) stepping by 0.2, and there is a 100ms delay between displaying the resulting frames. Note that it reverses direction in the loop so it goes dark... bright... dark... etc. You can make this kind of animation for any of the single-parameter operations; check out scale or swirl after you have implemented it!

Batch mode is useful for several things. First, it can be used for operations that are too slow for the interactive application (like computing this animation, or the morph described below). Second it is helpful for debugging because you can run the code repeatedly using exactly the same settings, pressing reload each time to start over as you edit your code. Third, you can set more parameters in the batch mode, by specifying them in the URL. For example, the bilateral filter can accept two sigma parameters. To see an example, try out BATCH MODE > newTab > bilateral. Another advantage to the batch mode is you can specify any image you like in addition to the ones in the GUI by naming the file in the URL. (Images are assumed to be in the "images" directory.) Finally, you can put as many operations as you want in the URL and they will be processed in the order they appear.

Implementing the Image Processing Filters

The assignment is worth 20 points. The following is a list of features that you may implement (listed roughly from easiest to hardest). The number in front of the feature corresponds to how many points the feature is worth for the full implementation. Partial or partially-correct solutions will receive partial credit. The features in bold face are required. The other ones are optional. Refer to the examples web page for example output images as well as some implementation hints.

• Luminance Operations
  • (0.0) Brightness: Change the brightness of an image by scaling RGB values towards black or white.
  • (0.5) Contrast: Change the contrast of an image using the method described in Graphica Obscura.
  • (0.5) Gamma: Apply a gamma correction to the image.
  • (0.5) Vignette: Apply a vignetting operator to the image.
  • (2.0) HistEq: Increase the contrast of the image by histogram equalization in the L channel.
• Color Operations
  • (0.0) Grayscale: Convert to gray levels by replacing each pixel with its luminance.
  • (0.5) Saturation: Change the color saturation of an image using the method described in Graphica Obscura.
  • (1.5) WhiteBalance: Shift all the colors so that a given RGB value becomes white using Von Kries' method.
  • (2.0) HistMatch: Histogram matching - match R,G,B channels to those of another image (easier if you already did HistEq).
• Filter Operations
  • (1.0) Gaussian: Blur an image by convolving it with a Gaussian low-pass filter.
  • (0.5) Sharpen: Apply a linear sharpening filter to the image.
  • (0.5) Edge: Detect edges in an image by convolving it with an edge detection kernel in each color channel.
  • (1.0) Median: Remove speckle noise using a median filter of given width
  • (2.0) Bilateral: Smooth while preserving sharp edges in the original. See here or here.
• Dithering Operations
  • (0.5) Quantize: Change the number of bits per channel of an image, using simple rounding.
  • (0.5) Random: Convert an image to a given number of bits per channel, using a random threshold.
  • (2.0) Ordered: Convert an image to a given number of bits per channel, using a 4x4 ordered dithering matrix.
  • (2.0) Floyd: Convert an image to a given number of bits per channel, using Floyd-Steinberg error diffusion.
• Resampling Operations
  • (0.5) Sampling: Implement bilinear and Gaussian sampling (in addition to the basic point sampling) for all resampling operations, including the morph below.
  • (1.0) Scale: Scale an image up or down in resolution by a real valued factor.
  • (1.5) Rotate: Rotate an image by a given angle.
  • (1.5) Swirl: Swirl the image around its center (see the examples web page).
• Composite Operations
  • (1.0) Composite: Compose one image with a second image using the over operator, with a third image as a matte (alpha).
  • (3.0) Morph: Use pairs of corresponding lines to morph a source image into a target image using a morph parameter t. See [Beier92]. Use the web page morphLines.html in your code directory to define the line correspondences.
• Miscellaneous
  • (3.0) Palette: Create a color palette from an image -- k colors that represent the color scheme of the image -- using basic k-means clustering (aka Lloyd's algorithm). The output should be a new image that contains the original image plus the palette drawn as k rectangles along the right side. See the examples web page.
  • (4.0) Paint: Implement the "paint by numbers" approach described by Haeberli to make the image look like a painting.
  • (4.0) XDoG: Implement the eXtended Difference-of-Gaussians compendium described by Winnemoeller for a range of artistic styles. Hint: use the approach of Kang to make the flow field (instead of the one in the XDoG paper) because it is based on bilateral filter that you already implemented.

Two of the features (brightness and grayscale) are already implemented for you ask examples. By implementing all the required features (in bold), you get 13 points. Full credit for this assignment is 20 points, so to complement the required features you may choose from the optional features listed above and participate in the art contest (which yields one point for participation and two for winning). It is possible to get more than 20 points for this assignment. Your final score will be calculated by adding:

• your score on the required features (up to 13 points)
• your participation in the art contest (up to 2 points)
• your score on the non-required features, calculated as follows.

The value of non-required features incurs diminishing returns after the first 7 points. For sums of non-required features (n>7), the score given will be 4 log(n-6) + 7.

How to Program the Filters

To implement the image processing features listed above, you only need to edit the file js/filters.js. Before starting on that, we recommend you take a quick look at the file js/image.js because it has some important helper code relating to images and pixels. You are welcome to look at any of the other files, but it should not be necessary and some of them have some pretty byzantine javascript magic. In js/filters.js, two of the image processing filters are already implemented for you as examples, so you should get the idea of how it works. Some of the filters (like brightness) can work directly in the image that is passed to them; however, other filters (like gaussian) need to allocate a new image to avoid overwriting the original pixel values needed in subsequent processing.

Submitting

You should submit your solution via CS dropbox here. The submitted zip file should preserve the directory structure of the skeleton code we provided in the zip file above.

The writeup.html file should be an HTML document demonstrating the effects of the features you have implemented and would like scored. For some features (e.g., sharpen), you can simply show the input and output of your program. (No need to show inputs for the default images in the images folder.) However, for features that take an input parameter (e.g., blur), you should provide a series of images showing at least two settings of the input parameter to demonstrate that your code is working properly.

You should start from the the example writeup.html provided. At the top of that file are a list of features that you might implement, linking to the section where you talk about them. Please remove any features that you do not implement, but otherwise leave this header section in tact. When you include an image, also include a link to the batch.html command that creates that image, as illustrated in the example writeup.

To save space, please submit images in png format. You may include one or a few gif files as well to show animations, but these files can be large so please try not to include lots of large gifs.

Note that you are expected to use good programming style at all times, including meaningful variable names, a comment or three describing what the code is doing, etc. Partial credit may not be assigned for code without comments. We have mostly tried to conform to the idiomatic JS style conventions.

Hints

A few hints:

• Do the simplest filters first!
• Look at the example page.
• Post a message to Piazza if you have questions.
• Take note of the late policy and the collaboration policy.

FAQ

Here are some answers to frequently asked questions. Check back here occasionally, as we may add FAQs to this list:

• When filtering / sampling / convolving etc, I need pixels outside the image boundary. What do I do?
  There are several ways to deal with that. You can clamp the lookup, that is, you use the closest coordinate that is still in bounds. Suppose your image is 10 pixels wide, and you want to retrieve a pixel at x position 12. In clamping mode, you would read the pixel with the highest available coordinate (that is, the one at index 9). You can also omit pixels outside the boundary from a convolution, but make sure that the remaining weights sum to 1 (for kernels that need this property). When implementing error diffusion dithering, you do not need to diffuse errors to pixels outside the image.

• My blur filter is darkening the image. What is wrong?
  Make sure that the entries in your kernel add up to 1. Be especially careful with rounding errors: although small in each entry, collectively they can be large enough to cause this undesired darkening effect.