Going up the Technology Readiness Level ladder with CS-MUVI

The world does not change at 30Hz! Aswin Sankaranarayanan just sent me the following:

Hi Igor 

We recently finished up a completely new video CS algorithm for the single pixel camera.

Here is a link to the project page: http://www.ece.rice.edu/~as48/research/csmuvi/

A key realization behind the algorithm in the paper is the fact that, when we sense time-varying scenes with a SPC, each compressive measurement is obtained from a slightly-different scene. We cannot ignore this fact especially for scenes with fast moving objects. Many (including me) have papers on video CS where we simulated CS data from 30 fps videos. This creates quite a mismatch from reality --- as we are simulating a world that abruptly changes only once every 1/30 second. In reality, the scene is constantly changing --- and addressing this is key if we want to get something that works in practice.

With Kevin Kelly's group, we now have this algorithm working on the SPC hardware and have been getting very nice results on real data; we'll be releasing that very soon. In all, quite excited about this. On a partially related note, I wanted to touch-base on a post from a month back

http://nuit-blanche.blogspot.com/2012/02/whose-heart-doesnt-sink-at-thought-of.htmli think, we can move the SPC a notch-up the ladder ;-)

And finally, as always, your time and effort on Nuit Blanche is highly appreciated.

-as

Aswin, maybe you moved up the TRL ladder, but you are also probably changing some of our readers' thinking when they talk about compressive sensing system and videos.Here is the introduction of the page:

CS-MUVI

Compressive sensing (CS)-based spatial-multiplexing cameras (SMCs) sample a scene through a series of coded projections using a spatial light modulator and a few optical sensor elements. SMC architectures are particularly useful when imaging at wavelengths for which full-frame sensors are too cumbersome or expensive. While existing recovery algorithms for SMCs perform well for static images, they typically fail for time-varying scenes (videos). We propose a novel CS multi-scale video (CS-MUVI) sensing and recovery framework for SMCs. Our framework features a co-designed video CS sensing matrix and recovery algorithm that provide an efficiently computable low-resolution video preview. We estimate the scene's optical flow from the video preview and feed it into a convex-optimization algorithm to recover the high-resolution video. We demonstrate the performance and capabilities of the CS-MUVI framework for different scenes.

CS-MUVI: Video Compressive Sensing for Spatial-Multiplexing Cameras by  Aswin Sankaranarayanan , Christoph Studer, and Richard Baraniuk. The abstract reads:

Compressive sensing (CS)-based spatial-multiplexing cameras (SMCs) sample a scene through a series of coded projections using a spatial light modulator and a few optical sensor elements. SMC architectures are particularly useful when imaging at wavelengths for which full-frame sensors are too cumbersome or expensive. While existing recovery algorithms for SMCs perform well for static images, they typically fail for time-varying scenes (videos). In this paper, we propose a novel CS multi-scale video (CS-MUVI) sensing and recovery framework for SMCs. Our framework features a co-designed video CS sensing matrix and

recovery algorithm that provide an efficiently computable low-resolution video preview. We estimate the scene’s optical flow from the video preview and feed it into a convexoptimization algorithm to recover the high-resolution video. We demonstrate the performance and capabilities of the CSMUVI framework for different scenes.

Thanks Aswin !


Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.