Saturday Morning Video: Deep Learning Meets Sparse Coding, Chandra Sekhar Seelamantula

Chandra just sent me the following:

Hi Igor, 

I would like to draw your attention to the following work of ours: 

https://arxiv.org/abs/1705.07290 

The lecture video related to the paper is up on youtube: 

https://www.youtube.com/watch?v=r3q05c1PIRg 

I request you to kindly share these with Nuit Blanche readers, if possible.
Hope you find it interesting.
Thank you so much.
Best regards,
Chandra
==

Chandra, this is great ! Here is the video:

Deep Sparse Coding Using Optimized Linear Expansion of Thresholds by Debabrata Mahapatra, Subhadip Mukherjee, Chandra Sekhar Seelamantula

We address the problem of reconstructing sparse signals from noisy and compressive measurements using a feed-forward deep neural network (DNN) with an architecture motivated by the iterative shrinkage-thresholding algorithm (ISTA). We maintain the weights and biases of the network links as prescribed by ISTA and model the nonlinear activation function using a linear expansion of thresholds (LET), which has been very successful in image denoising and deconvolution. The optimal set of coefficients of the parametrized activation is learned over a training dataset containing measurement-sparse signal pairs, corresponding to a fixed sensing matrix. For training, we develop an efficient second-order algorithm, which requires only matrix-vector product computations in every training epoch (Hessian-free optimization) and offers superior convergence performance than gradient-descent optimization. Subsequently, we derive an improved network architecture inspired by FISTA, a faster version of ISTA, to achieve similar signal estimation performance with about 50% of the number of layers. The resulting architecture turns out to be a deep residual network, which has recently been shown to exhibit superior performance in several visual recognition tasks. Numerical experiments demonstrate that the proposed DNN architectures lead to 3 to 4 dB improvement in the reconstruction signal-to-noise ratio (SNR), compared with the state-of-the-art sparse coding algorithms.

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.