CSVideoNet: A Real-time End-to-end Learning Framework for High-frame-rate Video Compressive Sensing - implementation -

The Great Convergence continues: Compressive Sensing allowed the encoder to be non iterative and now a set of neural networks (CNNs, LSTMs) make the decoding also non iterative. Woohoo !

Fengbo who just received an interesting NSF award also sent me the following:

Dear Igor, 

Hope all is well.
Recently, we proposed a deep learning based encoding/decoding framework (CSVideoNet) for high-frame-rate video compressive sensing. We believe you and your blog’s readers would be interested in this research. The preview of the manuscript can be found here: https://arxiv.org/abs/1612.05203

CSVideoNet: A Real-time End-to-end Learning Framework for High-frame-rate Video Compressive Sensing by Kai Xu, Fengbo Ren

This paper addresses the real-time encoding-decoding problem for high-frame-rate video compressive sensing (CS). Unlike prior works that perform reconstruction using iterative optimization-based approaches, we propose a non-iterative model, named "CSVideoNet". CSVideoNet directly learns the inverse mapping of CS and reconstructs the original input in a single forward propagation. To overcome the limitations of existing CS cameras, we propose a multi-rate CNN and a synthesizing RNN to improve the trade-off between compression ratio (CR) and spatial-temporal resolution of the reconstructed videos. The experiment results demonstrate that CSVideoNet significantly outperforms the state-of-the-art approaches. With no pre/post-processing, we achieve 25dB PSNR recovery quality at 100x CR, with a frame rate of 125 fps on a Titan X GPU. Due to the feedforward and high-data-concurrency natures of CSVideoNet, it can take advantage of GPU acceleration to achieve three orders of magnitude speed-up over conventional iterative-based approaches. We share the source code at this https URL

Thanks,
================================
Fengbo​ Ren
Director, Parallel Systems and Computing Laboratory (PSCLab)
​Assistant Professor, School of Computing, Informatics, and Decision Systems Engineering (CIDSE)
Arizona State University (ASU)
========================​​=====​​===

Thanks Fengbo !


Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.