Sunday Morning Insight: Readers' comments and an insight on living on the edge

Insights come in different shapes, one of them is in the comments left on this blog, here are a few and a recent blog entry: 

In Sparse Estimation From Noisy Observations of an Overdetermined Linear System / On the Randomized Kaczmarz Algorithm

Anonymous commented 

Seems to me that you aren't seeing these types of phase transition diagrams, because the Wu-Verdu result shows that you can get stable recovery for far lower measurement rates. The regime discussed here is interesting only because the noise is large.

I commented

I am sure I have seen it, this one come to my mind: http://nuit-blanche.blogspot.com/2013/08/convex-optimization-approaches-for.html

Igor.

but taking a cue from the commenter, the following phase transition are from Optimal Phase Transitions in Compressed Sensing by Yihong Wu, Sergio Verdú. Also of further interest

• Shannon Theory for Compressed Sensing, Yihong Wu

In Organizing Meetups: Some Tips

Joshua Stults commented

Wow Igor, thanks for this! We use meetup for our hackerspace here in Dayton, Ohio. I had the sneaking suspicion we were not using it to full potential, and you confirmed it. Lots of good ideas here. Thanks for sharing.

Martin Jaggi whom we featured here before, has started the Zurich Machine Learning Group and they had their first meetup this past week. I am looking forward to the video.

In Sunday Morning Insight: A Compressive Sensing Approach to Quanta Image Sensor (QIS), Thomas Arildsen commented

I have not properly read up on the QIS principle, but the photon counting approach you explain makes me wonder if this work on Poisson compressed sensing could be relevant to it:

http://people.ee.duke.edu/~willett/papers/PCS.pdf

http://people.ee.duke.edu/maxim/pubs/raginsky_willett_harmany_marcia_TSP10.pdf

(An anonymous reviewer of a paper of mine was kind enough to point those references out to me in another context.)

I also notice another detail: there seems to be a non-linear function involved in the measuring in that the detectors simply seem to count *one or more* photons, so any positive number of photons is truncated (quantized) to one. I am not sure if this matters, though, as you seem to suggest that these measured detections are themselves the sparse data one would hope to reconstruct from fewer measurements through compressed sensing.

Finally, I just noticed this paper http://www.sciencemag.org/content/343/6166/58 which sounds somewhat related. I have only read the abstract so far.

I commented

Thanks Thomas.

Yes the Poisson noise will be there but it is part of the measurement process so there is nothing that can be done about it.

As regards to the nonlinear detection, in the QIS itself it is only one photon being counted, there is the possibility of being nonlinear but it is not contemplated in the QIS work if I understand correctly.

Igor

In Do Deep Nets Really Need to be Deep?

Paul commented

Really Interesting, but it seems the SNN with the same number of parameters performs worse than the DNN or CNN. To get comparable performance from the SNN they have to substantially increase the number of parameters.

In Nonlinear Compressive Sensing - request -

yhli commented

Nonlinear regression problems with sparse parameters might be viewed as nonlinear compressive sensing problems. Then there are some results from the statistics community.

Finally, Dustin Mixon provided some insight on a paper in Living on the edge: A geometric theory of phase transitions in convex optimization. Go read it, I'll wait. You may recall that paper back when it was covered here:

• Living on the edge: A geometric theory of phase transitions in convex optimization
• Sunday Morning Insight: The Map Makers
• Sunday Morning Insight: Randomization is not a dirty word

and the attendant videos: 

• Living on the Edge - Phase Transitions in Random Convex Programs Joel Tropp ( other videos of the ROKS meeting )
• SAHD; Living on the edge: A geometric theory of phase transitions in convex optimization - Joel Tropp

Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.