Szymon Rusinkiewicz [Picture]
Office CS building, room 406
Email smr at princeton
Phone +1 609 258 7479
Fax +1 609 258 1771
Address   Department of Computer Science
Princeton University
35 Olden St.
Princeton, NJ  08540-5233
PAC

I am director of the undergraduate Certificate Program in Applications of Computing (PAC). Please read the webpage, and contact me at smr+pac at cs princeton edu if you have any questions about the program.

Teaching
Fall, 2013 COS 323 - Computing for the Physical and Social Sciences

 Older classes...

Spring, 2013 COS 598A - 3D Capture and Fabrication
Fall, 2012 COS 323 - Computing for the Physical and Social Sciences
Spring, 2012 COS 426 - Computer Graphics
Spring, 2012 COS/ART/HLS 495 - Modeling the Past - Technologies and Excavations in Polis, Cyprus
Fall, 2011 COS 429 - Computer Vision
Spring, 2011 COS 426 - Computer Graphics
Fall, 2010 COS 323 - Computing for the Physical and Social Sciences
Spring, 2010 COS 226 - Algorithms and Data Structures (Preceptor)
Fall, 2009 COS 429 - Computer Vision
Spring, 2008 COS 226 - Algorithms and Data Structures (Preceptor)
Fall, 2007 COS 597B - Reading and Writing Papers in Computer Graphics
Spring, 2007 COS/HLS 598C - Reconstructing the Thera Frescoes
Fall, 2006 FRS 123 - Technology in Art and Cultural Heritage
Spring, 2006 COS 323 - Computing for the Physical and Social Sciences
Fall, 2005 COS 429 - Computer Vision
Spring, 2005 COS 323 - Computing for the Physical and Social Sciences
Fall, 2004 COS 526 - Advanced Computer Graphics
Spring, 2004 COS 429 - Computer Vision
Fall, 2003 COS 526 - Advanced Computer Graphics
Spring, 2003 COS 496 - Computer Vision
Fall, 2002 COS 597B - 3D Photography
Spring, 2002 COS 496 - Computer Vision
Fall, 2001 COS 597D - Sensing for Graphics
Research

I am professor in the Computer Science Department and a member of the Princeton Computer Graphics Group and PIXL. My work focuses on the interface between computers and the visual and tangible world: acquisition, representation, analysis, and fabrication of 3D shape, motion, surface appearance, and scattering. I am interested in algorithms for processing geometry and reflectance, including registration, matching, completion, hierarchical decomposition, symmetry analysis, sampling, and depiction. Applications of this work include documentation of cultural heritage artifacts and sites, appearance and performance capture for digital humans, and depiction through line drawings and non-photorealistic shading models.

Here are some of my recent projects:

  • Reassembling the Thera Frescoes. We are engaged in a project to assist archaeologists and conservators in reconstructing frescoes excavated from the late-bronze-age settlement of Akrotiri, on the Aegean island of Thera (modern-day Santorini). The project incorporates systems for digitization of high-quality 3D geometry, normal maps, and color, as well as computer-aided matching of fragments on the basis of shape (3D edge profile), thickness, color, surface impressions, provenance, and other cues. This project is ongoing (the project page is here), and an initial publication on the digitization and matching systems is here.
  • Rendering with multi-light image collections and images with normals. We are investigating acquisition, analysis, and rendering techniques that begin with datasets inherently more powerful than single images, yet easier to acquire with high quality than full 3D scans. The Multiscale Shape and Detail Enhancement paper begins with just a few images of an object under varying lighting (a Multi-Light Image Collection or "MLIC"), and produces a non-photorealistic high-detail illustation by performing local statistical analyses at multiple scales. The Factored Time-Lapse Video paper attempts to separate the contributions of shadowing, reflectance, sunlight, and skylight from an uncalibrated full-day time-lapse video of an exterior scene.

    I am also very interested in the power of images with a per-pixel color and normal ("RGBN" images), and our recent RGBN NPR paper shows how to adapt signal processing, segmentation, and stylized depiction algorithms to this new datatype. Earlier work with these datasets included a paper on combining normal maps with scanned 3D data to produce high-quality high-resolution 3D models.

  • New techniques for range scanning. Although 3D scanning has become a widely-used technique, most commercially available systems acquire data at relatively low rates and require careful control over the setup and calibration of cameras and/or active light sources. I am interested in techniques for making range scanning systems faster, more flexible, more robust, and easier to use. One system I worked on lets you take an object, wave it around by hand in front of the scanner, and get a full 3D model of the object, while being able to see the model as it is being constructed. This combines a real-time structured-light rangefinder with algorithms for fast registration, merging, and display.

    I am also interested in "filling in the gaps" between traditionally separate methods for acquisition of 3D shape. For example, our spacetime stereo paper argues that methods such as structured light and traditional stereo may be considered points on a continuum of techniques that obtain depth via triangulation from 2 viewpoints (that may contain cameras or light sources), and that obtain correspondences over windows in space, time, or both. This suggests several hybrid methods, such as temporal stereo, that achieve high accuracy while increasing ease of use by not requiring camera-to-projector synchronization and calibration. More recent work has focused on taking advantage of multiview coding constraints to produce a multi-camera system that is guaranteed to not have matching ambiguity.

    My interests in range scanning began with my work on Digital Michelangelo, which was an epic effort to acquire high-resolution three-dimensional models of some well-known sculptures. As part of this project, I spent nine months in Italy doing things like heavy lifting, debugging hardware and software, heavy lifting, doing some actual scanning, and heavy lifting. My particular specialty was the color processing pipeline (some results are here and here). I also worked on the kiosk that displays our 3D models in the Galleria dell'Accademia, right next to the big guy himself.

  • Shape analysis, registration, and matching. 3D scans obtained from a rangescanner typically must be aligned to each other before a full model may be reconstructed. I am interested in both rigid-body registration (including stability analysis) and non-rigid methods, the latter being especially applicable when scanner miscalibration has resulted in warped scans. Using this technique we are finally making inroads into producing high-quality versions of the Digital Michelangelo scans. Together with Benedict Brown and Misha Kazhdan, we taught a course on "3D Scan Matching and Registration" at ICCV 2005.

    I have also worked on shape analysis projects such as curvature estimation, shape matching and retrieval, and detection of perfect and imperfect symmetries, both through the center of mass and through arbitrary planes. Using these detected symmetries, we can perform remeshing that yields attractive low-polygon-count models that preserve and strengthen the existing symmetries.

  • Shape depiction, including line drawings with suggestive contours. Line drawings made from 3D models usually contain strokes placed at the contours (or silhouettes - locations of depth discontinuities) and at creases or sharp bends on the surface. Suggestive contours are a different family of lines that complement contours and add a significant amount of visual information that helps dramatically in the perception of shape. The SIGGRAPH 2003 paper on suggestive contours is here. Our NPAR 2004 paper focuses on analyzing the stability of suggestive contours under changes in viewpoint and on fast extraction from triangular meshes, while our SIGGRAPH 2005 paper looks at suggestive contours in volumetric data. Sample code for rendering suggestive contours is available, and check out the suggestive contour gallery. Doug DeCarlo, Adam Finkelstein, and I also taught a course on "Line Drawings from 3D Models" at SIGGRAPH 2005.

    A recent project involves "exaggerated shading", which adjusts the effective light position for different areas of the surface. It reveals detail regardless of surface orientation and, by operating at multiple scales, is designed to convey detail at all frequencies simultaneously. The SIGGRAPH 2006 paper is here, and source code is also available.

  • Measurement and representation of realistic surface appearance. The appearance of a surface can be characterized by the amount of light it reflects at each surface position, for each pair of directions of incident and reflected light. (For translucent materials, the function must also include the position at which the light leaves the surface.) This is a complete description of an object's interaction with light, and embodies enough information to reconstruct its appearance in any environment. The difficulties with capturing and representing this reflectance are many: the datasets are large (gigabytes for even a sparse sampling of the spatially-varying reflectance), their underlying structure is often only exposed through complex reparameterization, and it is difficult to compress them while both retaining visual fidelity and maintaining the ability to use the data with algorithms for photorealistic lighting simulation.

    We have recently been focusing on factored reflectance representations, in which the high-dimensional reflectance function is first reparameterized using a change-of-basis transform that exposes its underlying structure, then is factored into low-dimensional pieces. Our SIGGRAPH 2004 paper focuses on a factorization specially designed to make importance sampling (in the context of a photorealistic global illumination system) efficient. More recently, we have focused on a complete hierarchical "Inverse Shade Tree" decomposition of spatially-varying reflectance, in which a material's appearance is broken down into intuitive, editable lower-dimensional components. Together with researchers at KU Leuven and MERL, we have also developed methods for decomposing heterogeneous subsurface scattering in materials such as veined marble.

    My previous work related to appearance includes helping design the Stanford spherical gantry, a computer-controlled, multi-axis measurement apparatus for measuring surface geometry and reflectance, and the bv program for displaying a variety of analytic BRDF functions.

  • QSplat. One of the results of the Digital Michelangelo Project was a series of meshes ranging from 100 million to 1 billion points. This is significantly more than can be displayed interactively on current hardware. QSplat is a rendering system that combines a multiresolution representation based on a bounding sphere hierarchy with splat rendering, resulting in a system capable of rendering huge scanned meshes at interactive frame rates. The QSplat software is publically available, and a description of the algorithm may be found in a SIGGRAPH 2000 paper. It is also possible to make QSplat stream data across the net.
Publications

  Recent

   Older...
Other Writings, Tutorials, Pictures, etc.
Software
  • Xshade is an implementation of Exaggerated Shading.
  • RTSC is a sample implementation of Real-Time Suggestive Contours.
  • trimesh2 is a library and a set of utilities for reading, writing, and manipulating 3D triangle meshes. An older version is also available.
  • qsplat is a multiresolution point rendering system for large scanned models.
  • bv is a browser for various analytical BRDF models.
Collaborators

Current Ph.D. advisees:

Others at Princeton:

Past Ph.D. advisees:

Past Masters advisees:

  • Siyu Liu
  • Tobechukwu Nwanna

Past postdocs:

Other collaborators:


Princeton University