LiSens, Photometric stereo using BRDF dictionaries, CS-MUVI

Aswin Sankaranarayanan just let me know of the following papers:

Hi Igor
Hope you are well.

I wanted to point out a few recent papers.


1) LiSens http://arxiv.org/abs/1503.04267

This paper describes a multi-pixel extension to the single pixel camera. We show that it is possible to obtain videos at spatial resolution of nearly mega-pixel resolution and at 10 frames-per-second using a sensor array with 1000s of pixel.

2) Photometric stereo using BRDF dictionaries. http://arxiv.org/abs/1503.04265

This paper is on shape estimation of non-Lambertian objects. While not mainstream CS work, at the heart of this work is a bilinear problem that we solve using some interesting techniques.

3) CS-MUVI (journal version) http://arxiv.org/abs/1503.02727

This paper uses motion-flow models for video CS using the single-pixel camera. We demonstrate many results on real data obtained from our lab prototype. This is the extended version of a conference publication in 2012.

Both LiSens and the photometric stereo work will appear in the proceedings of Intl. conf. computational photography (ICCP), 2015.

Am hoping this will be interesting to you and the broader nuit-blanche readership. 

Outstanding ! thanks Aswin

LiSens --- A Scalable Architecture for Video Compressive Sensing by Jian Wang, Mohit Gupta, Aswin C. Sankaranarayanan

The measurement rate of cameras that take spatially multiplexed measurements by using spatial light modulators (SLM) is often limited by the switching speed of the SLMs. This is especially true for single-pixel cameras where the photodetector operates at a rate that is many orders-of-magnitude greater than the SLM. We study the factors that determine the measurement rate for such spatial multiplexing cameras (SMC) and show that increasing the number of pixels in the device improves the measurement rate, but there is an optimum number of pixels (typically, few thousands) beyond which the measurement rate does not increase. This motivates the design of LiSens, a novel imaging architecture, that replaces the photodetector in the single-pixel camera with a 1D linear array or a line-sensor. We illustrate the optical architecture underlying LiSens, build a prototype, and demonstrate results of a range of indoor and outdoor scenes. LiSens delivers on the promise of SMCs: imaging at a megapixel resolution, at video rate, using an inexpensive low-resolution sensor.

A Dictionary-based Approach for Estimating Shape and Spatially-Varying Reflectance  by Zhuo Hui, Aswin C. Sankaranarayanan

We present a technique for estimating the shape and reflectance of an object in terms of its surface normals and spatially-varying BRDF. We assume that multiple images of the object are obtained under fixed view-point and varying illumination, i.e, the setting of photometric stereo. Assuming that the BRDF at each pixel lies in the non-negative span of a known BRDF dictionary, we derive a per-pixel surface normal and BRDF estimation framework that requires neither iterative optimization techniques nor careful initialization, both of which are endemic to most state-of-the-art techniques. We showcase the performance of our technique on a wide range of simulated and real scenes where we outperform competing methods.  

Video Compressive Sensing for Spatial Multiplexing Cameras using Motion-Flow Models by Aswin C. Sankaranarayanan, Lina Xu, Christoph Studer, Yun Li, Kevin Kelly, Richard G. Baraniuk

Spatial multiplexing cameras (SMCs) acquire a (typically static) scene through a series of coded projections using a spatial light modulator (e.g., a digital micro-mirror device) and a few optical sensors. This approach finds use in imaging applications where full-frame sensors are either too expensive (e.g., for short-wave infrared wavelengths) or unavailable. Existing SMC systems reconstruct static scenes using techniques from compressive sensing (CS). For videos, however, existing acquisition and recovery methods deliver poor quality. In this paper, we propose the CS multi-scale video (CS-MUVI) sensing and recovery framework for high-quality video acquisition and recovery using SMCs. Our framework features novel sensing matrices that enable the efficient computation of a low-resolution video preview, while enabling high-resolution video recovery using convex optimization. To further improve the quality of the reconstructed videos, we extract optical-flow estimates from the low-resolution previews and impose them as constraints in the recovery procedure. We demonstrate the efficacy of our CS-MUVI framework for a host of synthetic and real measured SMC video data, and we show that high-quality videos can be recovered at roughly 60× compression.

  Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.