A Short Review of 2014 on Nuit Blanche: On Compressive Sensing, Machine Learning, Advanced Matrix Factorization and many beautiful things.

2014 was an exceptional year! One that seemed focused on exploration.

First and foremost, in order to get a real sense of what was accomplished you probably need to read the last 12 Nuit Blanche Monthly Reviews.

This year saw the Ten Year Anniversary of compressive sensing.

The most important event of the year, is in my view the release of two genome sequencers that enable nearly any polynomial time algorithms to perform sequence alignment (Sunday Morning Insight: Crossing into P territory). It is quite simply a momentous event. In terms of data and algorithm most think of it as a faraway mountain range that will eventually need to be explored and studied. It's the opposite, think of it as the wave on Miller. Which is why one of the most fascinating video of the year was that of Craig Venter entitled Life at the Speed of Light. More on that later.

In terms of physical exploration, Philae landed on a comet, but one of the image I saw on my twitter feed this year provided me with the equivalent of what Nature does: Bacteriophage

 

The human made robot bumped three times to eventually land on the comet (we also lost it) but in Nature, do Bacteriophages have similar issues ? I'd love to know more about what we know about the  bacteriophage navigation algorithm!

Anyway, several trends could be witessed from a few of the 536 entries of 2014. Let me note the arrival of a few Machine Learning subjects as there seems to be a convergence between some supervised learning and compressive sensing (and my involvement as a co-organizer of one of the largest Machine Learning meetup).Without further ado, here is my biased take, we witnessed:

• A narrowing gap between reconstruction solvers in compressive sensing and the current deep (or not) architectures used in Machine learning to perform classification. the possibility of one-bit compressive sensing to provide some way of understanding neural networks or how real Neurons could be thought as Signal Processing Devices.
• An increase in Matrix and Tensor Factorizations of all kinds with their attendant phase diagrams (MF, Tensor , RandNLA tag on Nuit Blanche)
• We noticed a larger number of mapmakers, people who, while developing algorithms, also realized that these have bounds / limits could be used as a positive tool. But first and foremost, we saw that empirically some of these bounds could allow us to explore nonlinear reconstructions solvers such as those found in Machine Learning ( Some thoughts on invertibility: Signal recovery from Pooling Representations, Determination of Nonlinear Genetic Architecture using Compressed Sensing ). Sharp phase transition such as Donoho-Tanner was also seen as a way to probe good from bad neural networks in Compressive Sensing and Short Term Memory / Visual Nonclassical Receptive Field Effects or provide Sharp Performance Bounds for Graph Clustering . See also the more in-depth: Sunday Morning Insight: It's Complicated ...On Games, NP-hardness, Grothendieck, Learning and Friends. It is quite interesting to see these moments where pure mathematics has a real an immediate impact on actual engineering feats (Another Donoho-Tao Moment ? ). 
• We saw the growth of Multiple Regularizers (and recent results presented by Maryam Fazel)
• Randomization as a key for designing new algorithms.
• The actual rise of Convolutional Neural Networks as per their winning entries in Kaggle challenges. This indicates an increased robustness of these algorithms (a higher technology readiness level). But there is a high level of activity that aims at figuring out if those a really necessary (See the regularization architecture) or how they can be robustified through adverserial noise.
• Kaggle Challenges point to Random Forests as a first technique to use ( see also Why Kaggle Changes Everything yet don't really allow exploration)
• Vowpal wabbit (as seen in the recent score on the Tradeshift Kaggle challenge)
• Julia as a programming language used in our areas.
• A growing interest in extracting causality from correlation measurements.
• The Paris Machine Learning Meetup has had over 20 meetups already and can now boast to be the largest Machine Learning meetup outside the US (Top 6 overall). I have tried to scheduled a few research subjects that have a particular bearing on some of the theme mentioned here on Nuit Blanche. In at least one case, I tried to clinch the magic of at least one of these meetups on Unfiltered Access. Another meetup stands in my mind, it is the one we organized in June (Europe Wide Machine Learning Meetup and Paris Machine Learning #12: Season 1 Finale, Andrew Ng and More... ). The organization alone was interesting to say the least. But what's not to like when you are being made aware of the intricacies of decyphering Dolphin communications. All the slides and some videos of these meetups can be found here. I'll try to do a summary of what we earned from all these meetups next year.
• The mention and comment of the panel video entitled Is Deep Learning the Final Frontier and the End of Signal Processing ? produced a strong reaction from one of the leaders in Deep Learning. His response can be found in Yoshua Bengio's view on Deep Learning. 
• When asked about how Compressive Sensing helped in scientific discovery, I had a few ideas but here is another one: avoid a 200 Million Dollar Assumption
• The rise of data driven sensors ( Sunday Morning Insight: Zero Knowledge Sensor Design / Data Driven Sensor Design , Sunday Morning Insight: Physics Driven Sensor Design ? , Quick Panorama of Sensing from Direct Imaging to Machine Learning, Blind Deconvolution tag) as well as additional compressive sensing related hardware (including hyperspectral sensors)
• The potential for new fields to be explored (Sunday Morning Insight: Note By Note Cooking)

Personally, I also got back into publishing mode. Our paper also resulted in a few blog entries:

• Imaging With Nature: Compressive Imaging Using a Multiply Scattering Medium
• The Post Publication Peer Review of "Imaging With Nature: Compressive Imaging Using a Multiply Scattering Medium"
• Additional Photos and Video to "Imaging With Nature: Compressive Imaging Using a Multiply Scattering Medium"
• Follow up on "Imaging With Nature: Compressive Imaging Using a Multiply Scattering Medium"

I also did an open review (... There will be a "before" and "after" this paper ...) and did a poster presentation at JIONC on the connection between Direct Imaging to Machine Learning. I also provided some tips on Organizing Meetups and got Nuit Blanche featured on La Recherche of February.

Some additional stats can be found in:

• The Long Distance Blogger
• Nuit Blanche in Numbers
• Three and a half million page views: a million here, a million there and soon enough we're talking real readership...
• 1250 posts on the CompressiveSensing Subreddit.
• CitingNuitBlanche tag.

 Credits:
- Courtesy of NASA/SDO and the AIA, EVE, and HMI science teams.
- "PhageExterior" by Adenosine - Own work. Licensed under CC BY-SA 3.0 via Wikimedia Commons - http://commons.wikimedia.org/wiki/File:PhageExterior.svg#mediaviewer/File:PhageExterior.svg
-  ESA / Wikipedia
  Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.