Nuit Blanche: Sunday Morning Insight: Readers' comments and an insight on living on the edge

Sunday, March 02, 2014

Sunday Morning Insight: Readers' comments and an insight on living on the edge

Insights come in different shapes, one of them is in the comments left on this blog, here are a few and a recent blog entry:

In Sparse Estimation From Noisy Observations of an Overdetermined Linear System / On the Randomized Kaczmarz Algorithm

Anonymous commented

Seems to me that you aren't seeing these types of phase transition diagrams, because the Wu-Verdu result shows that you can get stable recovery for far lower measurement rates. The regime discussed here is interesting only because the noise is large.

I commented

I am sure I have seen it, this one come to my mind: http://nuit-blanche.blogspot.com/2013/08/convex-optimization-approaches-for.html

Igor.

but taking a cue from the commenter, the following phase transition are from Optimal Phase Transitions in Compressed Sensing by Yihong Wu, Sergio Verdú. Also of further interest

Shannon Theory for Compressed Sensing, Yihong Wu

In Organizing Meetups: Some Tips

Joshua Stults commented

Wow Igor, thanks for this! We use meetup for our hackerspace here in Dayton, Ohio. I had the sneaking suspicion we were not using it to full potential, and you confirmed it. Lots of good ideas here. Thanks for sharing.

Martin Jaggi whom we featured here before, has started the Zurich Machine Learning Group and they had their first meetup this past week. I am looking forward to the video.

In Sunday Morning Insight: A Compressive Sensing Approach to Quanta Image Sensor (QIS), Thomas Arildsen commented

I have not properly read up on the QIS principle, but the photon counting approach you explain makes me wonder if this work on Poisson compressed sensing could be relevant to it:

http://people.ee.duke.edu/~willett/papers/PCS.pdf

http://people.ee.duke.edu/maxim/pubs/raginsky_willett_harmany_marcia_TSP10.pdf

(An anonymous reviewer of a paper of mine was kind enough to point those references out to me in another context.)

I also notice another detail: there seems to be a non-linear function involved in the measuring in that the detectors simply seem to count *one or more* photons, so any positive number of photons is truncated (quantized) to one. I am not sure if this matters, though, as you seem to suggest that these measured detections are themselves the sparse data one would hope to reconstruct from fewer measurements through compressed sensing.

Finally, I just noticed this paper http://www.sciencemag.org/content/343/6166/58 which sounds somewhat related. I have only read the abstract so far.

I commented

Thanks Thomas.

Yes the Poisson noise will be there but it is part of the measurement process so there is nothing that can be done about it.

As regards to the nonlinear detection, in the QIS itself it is only one photon being counted, there is the possibility of being nonlinear but it is not contemplated in the QIS work if I understand correctly.

Igor

In Do Deep Nets Really Need to be Deep?

Paul commented

Really Interesting, but it seems the SNN with the same number of parameters performs worse than the DNN or CNN. To get comparable performance from the SNN they have to substantially increase the number of parameters.

In Nonlinear Compressive Sensing - request -

yhli commented

Nonlinear regression problems with sparse parameters might be viewed as nonlinear compressive sensing problems. Then there are some results from the statistics community.

Finally, Dustin Mixon provided some insight on a paper in Living on the edge: A geometric theory of phase transitions in convex optimization. Go read it, I'll wait. You may recall that paper back when it was covered here:

and the attendant videos: