FAQ for compressive sensing ?, Dark Matter in Neuroscience, power laws, MIA'09 redux, a Terry Tao Video, Orders of Magnitude 2010

Is it time for a FAQ in compressive sensing ? Have you had people coming to you and ask you questions about compressive and wished there was a FAQ somewhere ?. Here is my first throw:

What is compressive sensing to a linear algebra person ?

What is compressive sensing to an electrical engineer ?

What is compressive sensing to ....

How do you invert an underdetermined linear system in compressive sensing ?

Do you still have to sample at a high rate ?

How does one recognize that an under-determined linear equation can be solved with nonlinear solvers of compressive sensing ?

How do I recognize whether compressive sensing should be useful in my sensing system ?

How does the one pixel camera at rice work ?

Are there any other hardware implementing compressive sensing

Please help me strengthen the list as I am sure you have been asked similar questions by friends, colleagues, students,....





The author of this small ($300) movie just got a deal from Hollywood for 30 million dollars.




How can that be ? If this is not an expression of the long tail argument, I don't know what is. In the same category I would put Arduino in the same category and sometimes think that some compressive sensing projects ought to follow the same route.

While we are on the subject of power laws and sparsity, back at MIA'09, David Field made the comment that we have a lots of neurons but that in effect, our brain is using very few of them. He called that problem the dark matter problem of neuroscience. I asked him offline if he had references on this and here is what he had to say:

There is one I wrote with Bruno Olshausen.
BA Olshausen, DJ Field. What is the other 85% of V1 doing. - Problems in
Systems Neuroscience, 2004
This is on my website. A version was also published in Neural Computation.
http://redwood.psych.cornell.edu/

How close are we to understanding V1?
BA Olshausen, DJ Field - Neural Computation, 2005

This was extended by Shoham et al.
How silent is the brain: is there a “dark matter” problem in neuroscience?
S Shoham, DH O'Connor, R Segev - Journal of Comparative Physiology A: …, 2006

Thanks David !

I was also very much impressed at how one can compute intrinsic dimension and entropy for images (and much more really) as presented in this paper.

Chandler DM, and Field DJ. (2007). "Estimates of the Information Content and Dimensionality of Natural Scenes From Proximity Distributions." Journal of the Optical Society of America A, Vol. 24, Issue 4, pp. 922-941. [LINK] [PDF]

Other power laws and sparse sampling observed this week on the interwebs include:

• Insurgent Violence
• Judging people

And while we are on the subject of MIA'09, here are additional presentations now available:

• Pierre Vandergheynst, In Praise of Sparsity, Diverse and Sometimes Surprising Applications in Image Processing.
• Sandrine Anthoine, Content-based image retrieval using sparse multiscale patches
• Joachim Weickert, Image Compression with Partial Differential Equations
• Thomas Batard, Heat Equations on Vector Bundles : Application to Images Regularization
• Yannick Berthoumieu, Texture Discrimination Based Upon Stochastic Modeling Francis Bach, Non-local Sparse Models for Image Restoration

I did not see Pierre's presentation but I remember him reminding us about the number Ramesh Raskar (in his ECTV'08 presentation entitled Computational Photography: Epsilon to Coded Imaging) about the type of cameras that grew the largest in recent years (no it's not the cameras in cellphones!). Since those cameras are low resolution but plentiful, I wonder if a cheap Compressive Sensing project should be attempted around that platform. I put the last slide of Pierre's presentation to get a sense of the amount of data we are about to deal with. Let us contrast this to some of the numbers that were large back in 2004 (also here). To makes things simple to understand, in 2004, a movie was taking 10 times as much memory as a movie produced three years before (2001). In 2009, people are producing roughly, 250 exabytes of images, or 5 millions times the amount of pictures needed for a movie produced five years before.

While we are talking about videos, Terry Tao just made a new presentation on Compressed Sensing at University of Oregon. It is here and will be featured shortly in the Compressive Sensing Videos page.


As you all know, I have called Compressive Sensing , the George Constanza 'Do the opposite sampling scheme'. Well, since today is December 23rd, Happy Festivus everybody!