Andy ZengCS 194-26

Overview

The objective of this project is to play around with the concepts involved in image warping, and try out the application of image mosaicing. Using homographies and morphing/blending techniques that we've learned in the past, we can take a collection of images and stitch them together to form a wide angled panorama.

Recover Homographies

In order to warp an image into alignment with another image during the stitching process, we need to compute the transformation between the two images - in our case this is the homography matrix: p'=Hp where H is a 3x3 matrix with 8 degrees of freedom and (p',p) is a set of corresponding point pairs between the two images. To do this we solve for the 8 unknown values of the matrix H using a linear system of equations and the given information from the point pairs. At least 4 point correspondences are necessary in order to solve the system - any more than 4 point correspondences is solved with least-squares.

Warp the Images

Using the homorgraphy matrix H, we can forwardly transform the four corners of the original image to generate the four corners of the warped image. We then interpolate in between the corner points and inversely find in the corresponding pixel values in order to generate our final warped image. With transforms, we do some pretty neat things, like rectifying rectangles in images!

Signs at Bakery Square
Rectified Front-View of the Signs
Random Side-walk
Bird's Eye-View of the Side-walk
Random Snack Box
Rectified Front-View of the Box

Stitching and Blending

To approach mosaic generating, I stitched each image one by one into a set of combined and stitched images. In order to stitch two images together, we compute the homography matrix to warp the first image into the second image. Using an offset, we simply overlap the two images and perform some blending over the overlapped regions in order to remove visual artifacts. I compared the aesthetic results of the following blending techniques: no blending, non-weighted averaging, linear blending, and Laplacian pyramids. Here are some sample results:

Image 1
Image 2
No Blending
Non-weighted Average
Linear Blending
Laplacian Pyramid
Image 1
Image 2
No Blending
Non-weighted Average
Linear Blending
Laplacian Pyramid

Blending Technique Observations

No blending was fast, but for obvious reasons it did not generate nice looking results. Non-weighted average blending looked acceptable sometimes, but there were still some blurred artifacts that would show up from poor alignments from inaccurate point correspondences. There would also be terrible lighting differences on the edges of the images that would show up as sharp artifacts (which only slightly improved from when there was no blending performed at all). Linear blending was able to mitigate most of the noise caused by image lighting differences, but there were still some pretty annoying blurring artifacts that could arise from misalignment problems (i.e. see the example of linear blending on the 2nd to last image above - some objects are quite blurry). Laplacian blending over the image boundaries of the overlapped regions did well to remove the artifacts caused from misalignment, but the lighting differences (although blurred for the most part) can still be a problem without more pyramid levels (see the last image above where the image boundary is still quite visible on the side-walk). I personally liked the results from linear blending the most for my mosaics, since the lighting and color balancing felt much more natural to me. In my mosaics, I also had larger 'chunks' of areas in the image where the lighting needed to be consistently linear in order for the stitching to feel natural (the large body of water in the Tahoe mosaic, or the large road in the sidewalk mosaic below). So for my mosaics, I chose to use linear blending.

Mosaics

Here are some sample mosaics that I generated!

LeConte Hall 1
LeConte Hall 2
LeConte Hall 3
LeConte Hall Mosaic
Random Road 1
Random Road 2
Random Road 3
Random Road Mosaic
Lake Tahoe 1
Lake Tahoe 2
Lake Tahoe 3
Lake Tahoe Mosaic

What I Learned

I learned that homography is a very powerful concept that can be useful for many different applications. I had a lot of fun experimenting with and generating mosaics! The results were gorgeous.