Szymon Rusinkiewicz [Picture]
Office CS building, room 406
Email smr at princeton
Phone +1 609 258 7479
Fax +1 609 258 1771
Address   Department of Computer Science
35 Olden Street
Princeton University
Princeton, NJ  08540
Teaching
2008-2009 (On leave)

 Older classes...

Spring, 2008 COS 226 - Algorithms and Data Structures (Preceptor)
Fall, 2007 COS 597B - Reading and Writing Papers in Computer Graphics
Spring, 2007 COS 598C / HLS 598 - Reconstructing the Thera Frescoes
Fall, 2006 FRS 123 - Technology in Art and Cultural Heritage
Spring, 2006 COS 323 - Computing for the Physical and Social Sciences
Fall, 2005 COS 429 - Computer Vision
Spring, 2005 COS 323 - Computing for the Physical and Social Sciences
Fall, 2004 COS 526 - Advanced Computer Graphics
Spring, 2004 COS 429 - Computer Vision
Fall, 2003 COS 526 - Advanced Computer Graphics
Spring, 2003 COS 496 - Computer Vision
Fall, 2002 COS 597B - 3D Photography
Spring, 2002 COS 496 - Computer Vision
Fall, 2001 COS 597D - Sensing for Graphics
Research

I am associate professor in the Computer Science Department and a member of the Princeton Computer Graphics Group and PIXL. My research concentrates on techniques that incorporate large, measured datasets of surface geometry and appearance, as well as elements of human perception.

Here are some of my recent projects:

  • Rendering with multi-light image collections and images with normals. We are investigating acquisition, analysis, and rendering techniques that begin with datasets inherently more powerful than single images, yet easier to acquire with high quality than full 3D scans. The Multiscale Shape and Detail Enhancement paper begins with just a few images of an object under varying lighting (a Multi-Light Image Collection or "MLIC"), and produces a non-photorealistic high-detail illustation by performing local statistical analyses at multiple scales. The Factored Time-Lapse Video paper attempts to separate the contributions of shadowing, reflectance, sunlight, and skylight from an uncalibrated full-day time-lapse video of an exterior scene.

    I am also very interested in the power of images with a per-pixel color and normal ("RGBN" images), and our recent RGBN NPR paper shows how to adapt signal processing, segmentation, and stylized depiction algorithms to this new datatype. Earlier work with these datasets included a paper on combining normal maps with scanned 3D data to produce high-quality high-resolution 3D models.

  • New techniques for range scanning. Although 3D scanning has become a widely-used technique, most commercially available systems acquire data at relatively low rates and require careful control over the setup and calibration of cameras and/or active light sources. I am interested in techniques for making range scanning systems faster, more flexible, more robust, and easier to use. One system I worked on lets you take an object, wave it around by hand in front of the scanner, and get a full 3D model of the object, while being able to see the model as it is being constructed. This combines a real-time structured-light rangefinder with algorithms for fast registration, merging, and display.

    I am also interested in "filling in the gaps" between traditionally separate methods for acquisition of 3D shape. For example, our spacetime stereo paper argues that methods such as structured light and traditional stereo may be considered points on a continuum of techniques that obtain depth via triangulation from 2 viewpoints (that may contain cameras or light sources), and that obtain correspondences over windows in space, time, or both. This suggests several hybrid methods, such as temporal stereo, that achieve high accuracy while increasing ease of use by not requiring camera-to-projector synchronization and calibration. More recent work has focused on taking advantage of multiview coding constraints to produce a multi-camera system that is guaranteed to not have matching ambiguity.

    My interests in range scanning began with my work on Digital Michelangelo, which was an epic effort to acquire high-resolution three-dimensional models of some well-known sculptures. As part of this project, I spent nine months in Italy doing things like heavy lifting, debugging hardware and software, heavy lifting, doing some actual scanning, and heavy lifting. My particular specialty was the color processing pipeline (some results are here and here). I also worked on the kiosk that displays our 3D models in the Galleria dell'Accademia, right next to the big guy himself.

  • Shape analysis, registration, and matching. 3D scans obtained from a rangescanner typically must be aligned to each other before a full model may be reconstructed. I am interested in both rigid-body registration (including stability analysis) and non-rigid methods, the latter being especially applicable when scanner miscalibration has resulted in warped scans. Using this technique we are finally making inroads into producing high-quality versions of the Digital Michelangelo scans. Together with Benedict Brown and Misha Kazhdan, we taught a course on "3D Scan Matching and Registration" at ICCV 2005.

    I have also worked on shape analysis projects such as curvature estimation, shape matching and retrieval, and detection of perfect and imperfect symmetries, both through the center of mass and through arbitrary planes. Using these detected symmetries, we can perform remeshing that yields attractive low-polygon-count models that preserve and strengthen the existing symmetries.

  • Shape depiction, including line drawings with suggestive contours. Line drawings made from 3D models usually contain strokes placed at the contours (or silhouettes - locations of depth discontinuities) and at creases or sharp bends on the surface. Suggestive contours are a different family of lines that complement contours and add a significant amount of visual information that helps dramatically in the perception of shape. The SIGGRAPH 2003 paper on suggestive contours is here. Our NPAR 2004 paper focuses on analyzing the stability of suggestive contours under changes in viewpoint and on fast extraction from triangular meshes, while our SIGGRAPH 2005 paper looks at suggestive contours in volumetric data. Sample code for rendering suggestive contours is available, and check out the suggestive contour gallery. Doug DeCarlo, Adam Finkelstein, and I also taught a course on "Line Drawings from 3D Models" at SIGGRAPH 2005.

    A recent project involves "exaggerated shading", which adjusts the effective light position for different areas of the surface. It reveals detail regardless of surface orientation and, by operating at multiple scales, is designed to convey detail at all frequencies simultaneously. The SIGGRAPH 2006 paper is here, and source code is also available.

  • Measurement and representation of realistic surface appearance. The appearance of a surface can be characterized by the amount of light it reflects at each surface position, for each pair of directions of incident and reflected light. (For translucent materials, the function must also include the position at which the light leaves the surface.) This is a complete description of an object's interaction with light, and embodies enough information to reconstruct its appearance in any environment. The difficulties with capturing and representing this reflectance are many: the datasets are large (gigabytes for even a sparse sampling of the spatially-varying reflectance), their underlying structure is often only exposed through complex reparameterization, and it is difficult to compress them while both retaining visual fidelity and maintaining the ability to use the data with algorithms for photorealistic lighting simulation.

    We have recently been focusing on factored reflectance representations, in which the high-dimensional reflectance function is first reparameterized using a change-of-basis transform that exposes its underlying structure, then is factored into low-dimensional pieces. Our SIGGRAPH 2004 paper focuses on a factorization specially designed to make importance sampling (in the context of a photorealistic global illumination system) efficient. More recently, we have focused on a complete hierarchical "Inverse Shade Tree" decomposition of spatially-varying reflectance, in which a material's appearance is broken down into intuitive, editable lower-dimensional components. Together with researchers at KU Leuven and MERL, we have also developed methods for decomposing heterogeneous subsurface scattering in materials such as veined marble.

    My previous work related to appearance includes helping design the Stanford spherical gantry, a computer-controlled, multi-axis measurement apparatus for measuring surface geometry and reflectance, and the bv program for displaying a variety of analytic BRDF functions.

  • QSplat. One of the results of the Digital Michelangelo Project was a series of meshes ranging from 100 million to 1 billion points. This is significantly more than can be displayed interactively on current hardware. QSplat is a rendering system that combines a multiresolution representation based on a bounding sphere hierarchy with splat rendering, resulting in a system capable of rendering huge scanned meshes at interactive frame rates. The QSplat software is publically available, and a description of the algorithm may be found in a SIGGRAPH 2000 paper. It is also possible to make QSplat stream data across the net.
Publications

   Older publications...
Other Writings, Tutorials, Pictures, etc.
Software
  • Xshade is an implementation of Exaggerated Shading.
  • RTSC is a sample implementation of Real-Time Suggestive Contours.
  • trimesh2 is a library and a set of utilities for reading, writing, and manipulating 3D triangle meshes. An older version is also available.
  • qsplat is a multiresolution point rendering system for large scanned models.
  • bv is a browser for various analytical BRDF models.
Collaborators
Current Ph.D. students: Past Ph.D. students: Others at Princeton: Other collaborators: