Approximate Message Passing with Restricted Boltzmann Machine Priors

The great convergence is upon us. From the following paper:

The present paper demonstrates the first steps in incorporating deep learned priors into generalized linear problems such as compressed sensing.

 

Approximate Message Passing with Restricted Boltzmann Machine Priors by Eric W. Tramel, Angélique Drémeau, Florent Krzakala

Approximate Message Passing (AMP) has been shown to be an excellent statistical approach to signal inference and compressed sensing problem. The AMP framework provides modularity in the choice of signal prior; here we propose a hierarchical form of the Gauss-Bernouilli prior which utilizes a Restricted Boltzmann Machine (RBM) trained on the signal support to push reconstruction performance beyond that of simple iid priors for signals whose support can be well represented by a trained binary RBM. We present and analyze two methods of RBM factorization and demonstrate how these affect signal reconstruction performance within our proposed algorithm. Finally, using the MNIST handwritten digit dataset, we show experimentally that using an RBM allows AMP to approach oracle-support performance.  

    Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Gaussian Phase Transitions and Conic Intrinsic Volumes: Steining the Steiner Formula

 

 

Stéphane Chrétien just sent me the following:

 

Dear Igor,
 

I hope you are doing fine !
 

I recently read the following paper:

http://arxiv.org/abs/1411.6265

which uses techniques such as the Stein method for proving a CLT explaining precisely the asymptotical phase transition phenomenon in compressed sensing with gaussian measurements. The readers of Nuit Blanche might also be interested in this research.

Best regards, 

Stéphane

Thank you Stéphane !

Gaussian Phase Transitions and Conic Intrinsic Volumes: Steining the Steiner Formula by Larry Goldstein, Ivan Nourdin, Giovanni Peccati

Intrinsic volumes of convex sets are natural geometric quantities that also play important roles in applications, such as linear inverse problems with convex constraints, and constrained statistical inference. It is a well-known fact that, given a closed convex cone C⊂Rd, its conic intrinsic volumes determine a probability measure on the finite set {0,1,...d}, customarily denoted by L(VC). The aim of the present paper is to provide a Berry-Esseen bound for the normal approximation of L(VC), implying a general quantitative central limit theorem (CLT) for sequences of (correctly normalised) discrete probability measures of the type L(VCn), n≥1. This bound shows that, in the high-dimensional limit, most conic intrinsic volumes encountered in applications can be approximated by a suitable Gaussian distribution. Our approach is based on a variety of techniques, namely: (1) Steiner formulae for closed convex cones, (2) Stein's method and second order Poincar\'e inequality, (3) concentration estimates, and (4) Fourier analysis. Our results explicitly connect the sharp phase transitions, observed in many regularised linear inverse problems with convex constraints, with the asymptotic Gaussian fluctuations of the intrinsic volumes of the associated descent cones. In particular, our findings complete and further illuminate the recent breakthrough discoveries by Amelunxen, Lotz, McCoy and Tropp (2014) and McCoy and Tropp (2014) about the concentration of conic intrinsic volumes and its connection with threshold phenomena. As an additional outgrowth of our work we develop total variation bounds for normal approximations of the lengths of projections of Gaussian vectors on closed convex sets.  

  Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Tutorial: Submodularity in Machine Learning Applications

Here is a tutorial on Submodularity in Machine Learning Applications by Jeff Bilmes.   Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

RSVDPACK: Subroutines for computing partial singular value decompositions via randomized sampling on single core, multi core, and GPU architectures - implementation -

RSVDPACK: Subroutines for computing partial singular value decompositions via randomized sampling on single core, multi core, and GPU architectures by Sergey Voronin, Per-Gunnar Martinsson

This document describes an implementation in C of a set of randomized algorithms for computing partial Singular Value Decompositions (SVDs). The techniques largely follow the prescriptions in the article "Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions," N. Halko, P.G. Martinsson, J. Tropp, SIAM Review, 53(2), 2011, pp. 217-288, but with some modifications to improve performance. The codes implement a number of low rank SVD computing routines for three different sets of hardware: (1) single core CPU, (2) multi core CPU, and (3) massively multicore GPU.  

The implementations are on Sergey Voronin's GitHub: More specifically here.

  Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Unsupervised Learning of Acoustic Features Via Deep Canonical Correlation Analysis

When Random Features compete with Deep neural networks:  Unsupervised Learning of Acoustic Features Via Deep Canonical Correlation Analysis by Weiran Wang, Raman Arora, Karen Livescu, Jeff A. Bilmes

It has been previously shown that, when both acoustic and articulatory training data are available, it is possible to improve phonetic recognition accuracy by learning acoustic features from this multi-view data with canonical correlation analysis (CCA). In contrast with previous work based on linear or kernel CCA, we use the recently proposed deep CCA, where the functional form of the feature mapping is a deep neural network. We apply the approach on a speaker-independent phonetic recognition task using data from the University of Wisconsin X-ray Microbeam Database. Using a tandem-style recognizer on this task, deep CCA features improve over earlier multi-view approaches as well as over articulatory inversion and typical neural network-based tandem features. We also present a new stochastic training approach for deep CCA, which produces both faster training and better-performing features.

  Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Scalable audio separation with light Kernel Additive Modelling - implementation -

Using randomization to scale some capabilities on low power systems, this is what we have today: Scalable audio separation with light Kernel Additive Modelling by Antoine Liutkus, Derry Fitzgerald, Zafar Rafii

Recently, Kernel Additive Modelling (KAM) was proposed as a unified framework to achieve multichannel audio source separation. Its main feature is to use kernel models for locally describing the spectrograms of the sources. Such kernels can capture source features such as repetitivity, stability over time and/or frequency, self-similarity, etc. KAM notably subsumes many popular and effective methods from the state of the art, including REPET and harmonic/percussive separation with median filters. However, it also comes with an important drawback in its initial form: its memory usage badly scales with the number of sources. Indeed, KAM requires the storage of the full-resolution spectrogram for each source, which may become prohibitive for full-length tracks or many sources. In this paper, we show how it can be combined with a fast compression algorithm of its parameters to address the scalability issue, thus enabling its use on small platforms or mobile devices.  

The implementation is one Antoine Liutkus' KAML page.

    Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Crossing the P-river (follow-up)

A follow up to yesterday's blog entry. How much time did it take to gather all the information from the entire bacterial genome using a USB connected instrument that looks like a large memory stick and featured in the paper in Crossing the P-river: A complete bacterial genome assembled de novo using only nanopore sequencing data ?

@IgorCarron This was 4 runs worth which took 48 hours each, so 8 days with 1 MinION (or 2 days with 4 MinIONs!)
— Nick Loman (@pathogenomenick) February 24, 2015

 

Wow ! You're seeing history in the making.

 

Credit photo: this forum thread

 

Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Compressive spectrum sensing of radar pulses based on photonic techniques

 

 

Here is another instance of Compressive Sensing hardware: Compressive spectrum sensing of radar pulses based on photonic techniques by Qiang Guo, Yunhua Liang, Minghua Chen, Hongwei Chen, and Shizhong Xie

We present a photonic-assisted compressive sampling (CS) system which can acquire about 106 radar pulses per second spanning from 500 MHz to 5 GHz with a 520-MHz analog-to-digital converter (ADC). A rectangular pulse, a linear frequency modulated (LFM) pulse and a pulse stream is respectively reconstructed faithfully through this system with a sliding window-based recovery algorithm, demonstrating the feasibility of the proposed photonic-assisted CS system in spectral estimation for radar pulses. 

  Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Crossing the P-river: A complete bacterial genome assembled de novo using only nanopore sequencing data

 

 

We interrupt our regularly scheduled program, you probably remember this entry from fifteen days ago:

• Sunday Morning Insight: What Happens When You Cross into P Territory ?

 Here is a new piece of that puzzle, with the right sensor (the Oxford Nanopre MinION instrument a long read sequencer), things that used to be difficult are enabling the flooding of a new tsunami: A complete bacterial genome assembled de novo using only nanopore sequencing data by Nicholas James Loman , Joshua Quick , Jared T Simpson

A method for de novo assembly of data from the Oxford Nanopore MinION instrument is presented which is able to reconstruct the sequence of an entire bacterial chromosome in a single contig. Initially, overlaps between nanopore reads are detected. Reads are then subjected to one or more rounds of error correction by a multiple alignment process employing partial order graphs. After correction, reads are assembled using the Celera assembler. We show that this method is able to assemble nanopore reads from Escherichia coli K-12 MG1655 into a single contig of length 4.6Mb permitting a full reconstruction of gene order. The resulting assembly has 98.4% nucleotide identity compared to the finished reference genome. 

The software pipeline used to generate these assemblies is freely available online at https://github.com/jts/nanocorrect.

 

 

 

Related blog entries:

• Sunday Morning Insight: The Stuff of Discovery 
• Sunday Morning Insight: Crossing into P territory

 

 

        Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

X-Ray Vision with Only WiFi Power Measurements Using Rytov Wave Models

We continue our series of Hardware related compressive sensing blog entries with this tomography using WiFi and robots to cover the area of interest from different angles.

X-Ray Vision with Only WiFi Power Measurements Using Rytov Wave Models by Saandeep Depatla, Lucas Buckland and Yasamin Mostofi

In this paper, unmanned vehicles are tasked with seeing a completely unknown area behind thick walls based on only wireless power measurements using WLAN cards. We show that a proper modeling of wave propagation that considers scattering and other propagation phenomena can result in a considerable improvement in see-through imaging. More specifically, we develop a theoretical and experimental framework for this problem based on Rytov wave models, and integrate it with sparse signal processing and robotic path planning. Our experimental results show high-resolution imaging of three different areas, validating the proposed framework. Moreover, they show considerable performance improvement over the state-of-the-art that only considers the Line Of Sight (LOS) path, allowing us to image more complex areas not possible before. Finally, we show the impact of robot positioning and antenna alignment errors on our see-through imaging framework. 

In the end, I note their use of TV as a way to regularize their signals and I wonder if by modulating the power level of the emitters and having more than one receivers (they seem to have only two robots, one emitting the other receiving), then the results might be even more beautiful than what they have. While the system is indeed linear, it seems to rely on much trajectory design for both robots. With additional receivers, one would think that some of the trajectory issues might become less important but then additional receivers woud also mean that the system is not linear anymore as is the case of CT (traditional CT is linear, source coding CT is not).

On a totally unrelated note, I wonder if using monocular camera SLAM would simplify some of the embedded coding that goes into this current set up.

The project page is here.

      another Wifi related project:

• Wi-Vi: See Through Walls with Wi-Fi Signals by Fadel Adib Dina Katabi  

Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Reference-less measurement of the transmission matrix of a highly scattering material using a DMD and phase retrieval techniques - implementation -

Here is a follow up of our recent work on Compressive Sensing. I am not a co-author of this new paper so I feel freer to make a comment on it. In the previous approach, we defined the transmission matrix through measurements that required us to go through at least 4 different measurements with a phase difference of pi/4 in order to finally figure out each coefficient of the transmission matrix. Actually, this was the paper that originally used this technique. Our work put a spin on this by looking at it as a compressive sensing system (as opposed to using a reconstruction based on Tikhonov solver). But SLMs that produce those tiny phase shifts are expensive. For that reason, the authors of today's paper used a Texas Instrument DMD that loses this phase information. The paper shows that with enough of these phaseless information, we can recover the transmission matrix. In turn this knowledge is used to produce peaks through these media. Woohoo !

 

Reference-less measurement of the transmission matrix of a highly scattering material using a DMD and phase retrieval techniques by Angelique Dremeau, Antoine Liutkus, David Martina, Ori Katz, Christophe Schulke, Florent Krzakala, Sylvain Gigan, Laurent Daudet

This paper investigates experimental means of measuring the transmission matrix (TM) of a highly scattering medium, with the simplest optical setup. Spatial light modulation is performed by a digital micromirror device (DMD), allowing high rates and high pixel counts but only binary amplitude modulation. We used intensity measurement only, thus avoiding the need for a reference beam. Therefore, the phase of the TM has to be estimated through signal processing techniques of phase retrieval. Here, we compare four different phase retrieval principles on noisy experimental data. We validate our estimations of the TM on three criteria : quality of prediction, distribution of singular values, and quality of focusing. Results indicate that Bayesian phase retrieval algorithms with variational approaches provide a good tradeoff between the computational complexity and the precision of the estimates.

the solver is on Angelique's page under this paper:

• A. Drémeau, F. Krzakala - Phase recovery from a Bayesian point of view: the variational approach - Accepted at IEEE Int'l Conference on Acoustics, Speech and Signal Processing (ICASSP), (preprint) arXiv:1410.1368, Brisbane, Australia, April 2015. 

  Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Thesis: Recovery of Continuous Quantities from Discrete and Binary Data with Applications to Neural Data

We featured Karin Knudson's work before here in One-bit compressive sensing with norm estimation - implementation - well, Karin graduated and here is her very interesting thesis. It is interesting in part because it is part of the continuum of studies between traditional compressive sensing and neural networks. Gigem, uh....Congratulations Karin ! here is the thesis: Recovery of Continuous Quantities from Discrete and Binary Data with Applications to Neural Data

We consider three problems, motivated by questions in computational neuroscience, related to recovering continuous quantities from binary or discrete data or measurements in the context of sparse structure. First, we show that it is possible to recover the norms of sparse vectors given one-bit compressive measurements, and provide associated guarantees. Second, we present a novel algorithm for spike-sorting in neural data, which involves recovering continuous times and amplitudes of events using discrete bases. This method, Continuous Orthogonal Matching Pursuit, builds on algorithms used in compressive sensing. It exploits the sparsity of the signal and proceeds greedily,achieving gains in speed and accuracy over previous methods. Lastly, we present a Bayesian method making use of hierarchical priors for entropy rate estimation from binary sequences.

   

N00235465.jpg was taken on February 16, 2015 and received on Earth February 17, 2015. The camera was pointing toward SATURN, and the image was taken using the CL1 and CL2 filters. This image has not been validated or calibrated. Image Credit: NASA/JPL/Space Science Institute

  Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Weighted SGD for ℓp Regression with Randomized Preconditioning

Following up on Tuesday's related development (53 pages of good stuff), here is some more on the use of randomized projection for preconditionning. Proposition 13 is interesting: Weighted SGD for ℓp Regression with Randomized Preconditioning by Jiyan Yang, Yin-Lam Chow, Christopher Ré, Michael W. Mahoney

In recent years, stochastic gradient descent (SGD) methods and randomized linear algebra (RLA) algorithms have been applied to many large-scale problems in machine learning and data analysis. SGD methods are easy to implement and applicable to a wide range of convex optimization problems. In contrast, RLA algorithms provide much stronger performance guarantees but are applicable to a narrower class of problems. In this paper, we aim to bridge the gap between these two methods in solving overdetermined linear regression problems---e.g., ℓ2 and ℓ1 regression problems. We propose a hybrid algorithm that uses RLA techniques for preconditioning and constructing an importance sampling distribution, and then performs an SGD-like iterative process with weighted sampling on the preconditioned system. We prove that this algorithm inherits faster convergence rates that only depend on the lower dimension of the linear system, while maintaining low computation complexity. The effectiveness of such algorithms is illustrated numerically, and the results are consistent with our theoretical findings. Finally, we also provide lower bounds on the coreset complexity for more general regression problems, indicating that still new ideas will be needed to extend similar RLA preconditioning ideas to weighted SGD algorithms for more general regression problems.  

    Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Implementing Randomized Matrix Algorithms in Parallel and Distributed Environments

For some problems that are typically using Hadoop and related techniques, RandNLA is an enabler  in  that " the improved scalability often comes due to restricted communications, rather than improvements in FLOPS"

Implementing Randomized Matrix Algorithms in Parallel and Distributed Environments by Jiyan Yang, Xiangrui Meng, Michael W. Mahoney

In this era of large-scale data, distributed systems built on top of clusters of commodity hardware provide cheap and reliable storage and scalable processing of massive data. Here, we review recent work on developing and implementing randomized matrix algorithms in large-scale parallel and distributed environments. Randomized algorithms for matrix problems have received a great deal of attention in recent years, thus far typically either in theory or in machine learning applications or with implementations on a single machine. Our main focus is on the underlying theory and practical implementation of random projection and random sampling algorithms for very large very overdetermined (i.e., overconstrained) ℓ1 and ℓ2 regression problems. Randomization can be used in one of two related ways: either to construct sub-sampled problems that can be solved, exactly or approximately, with traditional numerical methods; or to construct preconditioned versions of the original full problem that are easier to solve with traditional iterative algorithms. Theoretical results demonstrate that in near input-sparsity time and with only a few passes through the data one can obtain very strong relative-error approximate solutions, with high probability. Empirical results highlight the importance of various trade-offs (e.g., between the time to construct an embedding and the conditioning quality of the embedding, between the relative importance of computation versus communication, etc.) and demonstrate that ℓ1 and ℓ2 regression problems can be solved to low, medium, or high precision in existing distributed systems on up to terabyte-sized data.

 a related presentation from MMS2012

    Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Kernel Probabilistic Programming (KPP): Computing Functions of Random Variables via Reproducing Kernel Hilbert Space Representations

Random Features used as a means of speeding up Kernel Probabilistic Programming:

Computing Functions of Random Variables via Reproducing Kernel Hilbert Space Representations by Bernhard Schölkopf, Krikamol Muandet, Kenji Fukumizu, Jonas Peters

We describe a method to perform functional operations on probability distributions of random variables. The method uses reproducing kernel Hilbert space representations of probability distributions, and it is applicable to all operations which can be applied to points drawn from the respective distributions. We refer to our approach as {\em kernel probabilistic programming}. We illustrate it on synthetic data, and show how it can be used for nonparametric structural equation models, with an application to causal inference.

this is funny ref 34 ( http://www.learning-with-kernels.org/ ) does not lead to the intended site.

Of relevance:
Kernel Mean Estimation and Stein's Effect by Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, Arthur Gretton, Bernhard Schölkopf

A mean function in reproducing kernel Hilbert space, or a kernel mean, is an important part of many applications ranging from kernel principal component analysis to Hilbert-space embedding of distributions. Given finite samples, an empirical average is the standard estimate for the true kernel mean. We show that this estimator can be improved via a well-known phenomenon in statistics called Stein's phenomenon. After consideration, our theoretical analysis reveals the existence of a wide class of estimators that are better than the standard. Focusing on a subset of this class, we propose efficient shrinkage estimators for the kernel mean. Empirical evaluations on several benchmark applications clearly demonstrate that the proposed estimators outperform the standard kernel mean estimator.

      Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

CRISP - Towards Compressive Information Processing Systems - implementation -

Giulio just sent me the following the other day:

 

 

Dear Igor, 

my name is Giulio Coluccia and I'm a PostDoc at Politecnico di Torino, Italy. I'm involved in an Eurpean ERC Project, named CRISP, entirely dedicated to Compressed Sensing. Its website is www.crisp-erc.eu and the Principal Investigator is Prof. Enrico Magli (enrico.magli@polito.it http://www1.tlc.polito.it/oldsite/sas-ipl/Magli/). The website contains all the material produced within the project, including the full text of the papers and the software to reproduce the results. I just wanted to address you the website of our project and, in case you find it interesting, to encourage you to contact either me or directly Prof. Magli for more details.
Kind regards,Giulio

Giulio Coluccia, PhD
Politecnico di Torino
Dipartimento di Elettronica e Telecomunicazioni
(Sede Storica - Corso Montevecchio)
Room S4-MB-15
corso Duca degli Abruzzi 24
I-10129, Torino, Italy 

Here is the publication page with attendant links to papers and implementations:

 

2014

• Matamoros, J. , Fosson, S., Magli, E., Antón-Haro, C., Distributed ADMM for in-network reconstruction of sparse signals with innovations, IEEE Global Conference on Signal and Information Processing (GlobalSIP), Atlanta, Georgia, USA, December 3-5, 2014
• Bioglio, V., Bianchi, T., Magli, E., Secure Compressed Sensing over Finite Fields, IEEE Workshop on Information Forensics and Security (WIFS), Atlanta, Georgia, USA, December 3-5, 2014
• Bianchi, T., Magli, E., Analysis of the Security of Compressed Sensing with Circulant Matrices, IEEE Workshop on Information Forensics and Security (WIFS), Atlanta, Georgia, USA, December 3-5, 2014
• Bioglio, V., Coluccia, G., Magli, E., Sparse Image Recovery Using Compressed Sensing Over Finite Alphabets, IEEE International Conference on Image Processing (ICIP) 2014, Paris, France, October 27-30, 2014
• Coluccia, G., Roumy, A., Magli, E., Operational Rate-Distortion Performance of Single-source and Distributed Compressed Sensing, to appear in IEEE Transactions on Communications, 2014
• Coluccia, G., Roumy, A., Magli, E., Exact Performance Analysis of the Oracle Receiver for Compressed Sensing Reconstruction, 39th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Firenze, Italy, May 4-9, 2014
• Kamdem Kuiteing, S., Coluccia, G., Barducci, A., Barni, M., Magli, E., Compressive Hyperspectral Imaging Using Progressive Total Variation, 39th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Firenze, Italy, May 4-9, 2014
• Valsesia, D., Magli, E., Compressive Signal Processing with Circulant Sensing Matrices, 39th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Firenze, Italy, May 4-9, 2014
• Bianchi, T., Bioglio, V., Magli, E., On the Security of Random Linear Measurements, 39th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Firenze, Italy, May 4-9, 2014
• Ravazzi, C., Fosson, S.M., Magli, E., Energy-saving Gossip Algorithm for Compressed Sensing in Multi-agent Systems, 39th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Firenze, Italy, May 4-9, 2014
• Fosson, S.M., Matamoros, J., Antón-Haro, C., Magli, E., Distributed Support Detection of Jointly Sparse Signals, 39th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Firenze, Italy, May 4-9, 2014

2013

• Valsesia, D., Coluccia, G., Magli, E. Joint recovery algorithms using difference of innovations for distributed compressed sensing, Conference Record of the Forty Seventh Asilomar Conference on Signals, Systems and Computers (ASILOMAR), 2013
• Ravazzi, C. Fosson S.M., Magli, E. Distributed soft thresholding for sparse signal recovery, IEEE GLOBECOM 2013, December 9-13, Atlanta, Georgia, USA
• Fiandrotti, A, Fosson S.M., Ravazzi C., Magli E. PISTA: Parallel Iterative Soft Thresholding Algorithm for Sparse Image Recovery, 30th Picture Coding Symposium, December 8-11, San Jose, California, USA
• Björklund, T., Magli, E. A Parallel Compressive Imaging Architecture for One-Shot Acquisition, 30th Picture Coding Symposium, December 8-11, San Jose, California, USA
• Coluccia, G., Valsesia, D., Magli, E. Smoothness-Constrained Image Recovery from Block-Based Random Projections, 15th International Workshop on Multimedia Signal Processing, September 30-October 3, 2013, Pula, Italy, pp. 129-134
• Valsesia, D., Magli, E. Spatially Scalable Compressed Image Sensing with Hybrid Transform and Inter-layer Prediction Model, 15th International Workshop on Multimedia Signal Processing, September 30-October 3, 2013, Pula, Italy, pp. 373-378
• Bay, A., Fragneto,P., Grella, M., Fosson, S.M., Ravazzi, C., Magli, E. Sparsity-based Indoor Localization in Wireless Sensor Networks, Demo Session at 15th International Workshop on Multimedia Signal Processing, September 30-October 3, 2013, Pula, Italy
• Valsesia, D, Coluccia, G., Magli, E. Graded Quantization: Democracy for Multiple Descriptions in Compressed Sensing, 38th International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vancouver, Canada, May 26 - 31, 2013, pp. 5825-5829

2012

• Coluccia, G., Kuiteing, S. K., Abrardo, A., Barni, M., Magli, E. Progressive compressed sensing and reconstruction of multidimensional signals using hybrid transform/prediction sparsity model. IEEE Journal on emerging and Selected Topics in Circuits and Systems, 2(3), 340-352.
• Coluccia, G., Magli, E. A Novel Progressive Image Scanning and Reconstruction Scheme Based on Compressed Sensing and Linear Prediction. In 2012 IEEE International Conference on Multimedia and Expo (ICME),  (pp. 866-871). IEEE.

 

  Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Saturday Morning Videos: Nando de Freitas' Machine Learning course, ICLR2014

I meant to post this yesterday. Oh well.

Nando de Freitas is currently teaching a course at Oxford on Machine Learning and it is videotaped:

• Deep learning at Oxford 2015
• Deep learning - Introduction
• Deep learning - linear models
• Deep learning - maximum likelihood and information
• Deep learning - regularization, model complexity and data complexity (part 1)
• Deep learning - Optimization
• Deep learning - Deep learning - regularization, model complexity and data complexity (part 1)

All the slides should be here.

I also found this series of ICLR videos from last year:

• ICLR14: J Bruna: Spectral Networks and Locally Connected Networks on Graphs
• ICLR14: J Masci: Sparse similarity-preserving hashing
• ICLR14: P Sprechmann (for Qiang Qiu): Learning Transformations for Classification Forests
• ICLR14: P Sermanet: OverFeat: Integrated Recognition, Localization and Detection using ConvNets
• ICLR14: D Kingma: Auto-Encoding Variational Bayes
• ICLR14: A Klami: Group-sparse Embeddings in Collective Matrix Factorization
• ICLR14 Invited Talk: Vincent Vanhoucke: "Learning visual representations at scale"
• ICLR14 Workshop: S Wang: Relaxations for inference in RBM
• ICLR14 Workshop: A Modi: Learning Semantic Script Knowledge with Event Embeddings
• ICLR14 Workshop: A Szlam: Unsupervised Feature Learning by Deep Sparse Coding
• ICLR14: M Norouzi: Zero-Shot Learning by Convex Combination...
• ICLR14 Invited Talk: Hynek Hermansky "Speech representations: 'knowledge' or data?" 
• ICLR14: A Saxe: Exact solutions to the nonlinear dynamics of learning... 
• ICLR14: T Schaul: Unit Tests for Stochastic Optimization 
• ICLR14: R Pascanu: Revisiting Natural Gradient for Deep Networks 
• ICLR14 Invited talk: Rich Sutton "Myths of Representation Learning". 

    Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Abstract for "collège doctoral" seminar at IFPEN, March 2015

A while back, Laurent Duval kindly asked me to do a presentation at the "collège doctoral" seminar at IFPEN at the end of March.  

Based on my co-hosting of the Paris Machine Learning meetup and the themes covered on Nuit Blanche, I (foolishly ?) decided to say yes. Here is my first (and probably last) draft abstract (so now it's public, I cannot walk away from this :-)

Title: "Ca va être compliqué": Islands of knowledge, Mathematician-Pirates and the Great Convergence

Igor Carron,
https://www.linkedin.com/in/IgorCarron
http://nuit-blanche.blogspot.com

In this talk, we will survey the different techniques that have led to recent changes in the way we do sensing and how to make sense of that information. In particular, we will talk about problem complexity and attendant algorithms, compressive sensing, advanced matrix factorization, sensing hardware and machine learning and how all these seemingly unrelated issues are of importance to the praticing engineer. In particular, we'll draw some parallel between some of the techniques currently used in machine learning as used by internet companies and the upcoming convergence that will occur in many fields of Engineering and Science as a result.

    Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Abstracts: Information Theory and Applications (ITA 2014)

 

 

As I was reading this presentation Universal Denoising and Approximate Message Passing by Dror Baron , Yanting Ma and Junan Zhu, I realized I had not covered the ITA conference yet.

Here are some of the abstracts of the presentations made there, some of which have been covered here on Nuit Blanche already. Enjoy !

Zhu

• Capacity-Approaching Sparse-Graph Code for Compressive Phase Retrieval Ramtin Pedarsani, UC Berkeley; Kangwook Lee, UC Berkeley; Kannan Ramchandran*, UC Berkeley
• Finding globally optimal k-means clusterings through convex relaxation Rachel Ward*, UT Austin
• Limits on Support Recovery with Probabilistic Models: An Information-Spectrum Approach Jonathan Scarlett*, EPFL; Volkan Cevher, EPFL
• Locating outliers in large matrices with adaptive compressive sampling Xingguo Li, University Of Minnesota; Jarvis Haupt*, University Of Minnesota
• Compressive modeling of stationary autoregressive processes Georg Kail, Delft University Of Technology; Geert Leus*, Delft University Of Technology
• The benefits of nonconvex optimization for compressed sensing, Galen Reeves*, Duke
• A totally unimodular view of structured sparsity Volkan Cevher*, Swiss Federal Institute of Technology in Lausanne
• Stable Super-Resolution of Positive Sources Veniamin Morgenshtern*, Stanford; Emmanuel Candes, Stanford
• Information guided sequential sensing Yao Xie*, Georgia Tech; Sebastian Pokutta, Georgia Tech; Gabor Braun, Georgia Tech
• Doubly sparse compressed sensing Grigory Kabatiansky*, IITP RAS; Serge Vladuts, Aix-Marseille University, France; Cedric Tavernier, Assystem, France
• Sub-linear Time Compressed Sensing using Sparse-Graph Codes, Xiao Li*, UC Berkeley; Sameer Pawar, UC Berkeley; Kannan Ramchandran, UC Berkeley
• Universal Denoising and Approximate Message Passing, Dror Baron*, NC State
• Universal compressed sensing of Markov sources Shirin Jalali*, Princeton; H. Vincent Poor, Princeton
• Generalized Compressive Interferometry for Optical Modal Analysis George Atia*, University of Central Florida; Ayman Abouraddy, University of Central Florida; Davood Mardani, University of Central Florida
• Optimal Data-Dependent Hashing for Nearest Neighbors Alexandr Andoni*, Simons Institute for the Theory of Computing, UC Berkeley; Ilya Razenshteyn, MIT
• Streaming and space efficient approximation algorithms for Subset Sum Anna Gal*, UT Austin; Jing-Tang Jang, Google; Nutan Limaye, IIT Bombay; Meena Mahajan, Institute of Mathematical Sciences Chennai; Karteek Sreenivasaiah, Max Planck Institute for Informatics
• Nearly optimal and deterministic sparse Hadamard transform Mahdi Cheraghchi*, UC Berkeley; Piotr Indyk, MIT
• On the stability of collaborative dictionary learning Waheed Bajwa*, Rutgers
• Learning Minimal Latent Directed Information Polytrees Negar Kiyavash*, UIUC
• Exploration-Exploitation: a low rank matrix completion solution and performance bounds Urbashi Mitra*, USC; Sunav Choudhary, USC
• Low-rank methods to address the big data issues in the synchrophasor measurements in the power system Meng Wang*, RPI; Pengzhi Gao, RPI; Joe H. Chow, RPI; Scott Ghiocel, RPI; Bruce Fardanesh, New York Power Authority; George Stefopoulos, New York Power Authority
• THE NETFLIX RECOMMENDER SYSTEM: WHAT IT IS AND HOW WE IMPROVE IT Carlos Gomez-Uribe*, Netflix, Inc.
• Optimal CUR Matrix Decompositions, Christos Boutsidis*, Yahoo; David Woodruff, IBM
• Single pass spectral sparsification in dynamic streams, Michael Kapralov*, IBM T.J Watson; Yin-Tat Lee, MIT; Cameron Musco, MIT; Christopher Musco, MIT; Aaron Sidford, MIT
• Sketching for M-Estimators: A Unified Approach to Robust Regression Ken Clarkson*, IBM Research - Almaden; David Woodruff, IBM Almaden
• Subspace embeddings and learning applications Haim Avron, IBM T.J Watson; Jelani Nelson, Harvard; Huy Nguyen*, UC Berkeley; David Woodruff, IBM Almaden
• Elementary estimators for high-dimensional statistical models Eunho Yang, IBM T.J Watson; Aurelie Lozano, IBM T.J Watson; Pradeep Ravikumar*, UT Austin
• Inference in High-Dimensional Varying Coefficient Models Mladen Kolar*, University Of Chicago; Damian Kozbur, ETH
• Stochastic Iterative Greedy Algorithms for Sparse Reconstruction Nam Nguyen, MIT; Deanna Needell*, Claremont McKenna College; Tina Woolf, Claremont Graduate University
• High dimensional variable selection: a decision theoretic approach Oluwasanmi Koyejo*, Stanford; Rajiv Khanna, University Of Texas; Joydeep Ghosh, University Of Texas; Russell Poldrack, Stanford
• Big Tensor Subspace Learning for Dynamic MRI Georgios B. Giannakis*, University Of Minnesota
• Clustering with Distributed Data: A Consensus+Innovations Approach Soummya Kar*, CMU
• Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints Ben Recht*, UC Berkeley; Laurent Lessard, UC Berkeley; Andrew Packard, UC Berkeley
• Non-convex Robust PCA Praneeth Netrapalli, Microsoft Research; U. Niranjan, UC Irvine; Sujay Sanghavi*, UT Austin; Anima Anandkumar, UC Irvine; Prateek Jain, Microsoft Research
• Simple, Efficient and Neural Algorithms for Sparse Coding Ankur Moitra*, MIT
• Correlation decay, Phase transitions, and Counting Alistair Sinclair, UC Berkeley; Piyush Srivastava*, Caltech; Daniel Stefankovic, University Of Rochester; Yitong Yin, Nanjing University
• K-clustering of Semi-Random graphs, Alexandra Kolla*, UIUC
• Incremental clustering: The case for extra clusters, Margareta Ackerman*, Florida State University; Sanjoy Dasgupta, UC San Diego
• The Lovász Local Lemma as a Random Walk, Dimitris Achlioptas*, UC Santa Cruz; Fotis Iliopoulos, UC Berkeley
• Inverse Rendering using Color+Depth Camera, Ha Nguyen, EPFL; Minh Do*, University Of Illinois
• Environmental information from noise, Peter Gerstoft*, UC San Diego
• Clustering techniques applied to the human seismic footprint, Nima Riahi*, UC San Diego; Peter Gerstoft, UC San Diego
• Earthquake Ground Motion Prediction using Seismic Noise, Marine Denolle*, UC San Diego
• Parametric Bilinear Generalized Approximate Message Passing, Phil Schniter*, Ohio State; Jason Parker, AFRL
• Capacity-achieving sparse regression codes via approximate message passing decoding Cynthia Rush, Yale; Adam Greig, Cambridge; Ramji Venkataramanan*, Cambridge
• Algorithms for Parameter Estimation in Mixture Models, Aditya Bhaskara*, Google
• Control-based analog-to-digital conversion without sampling and quantization, Hans-Andrea Loeliger*, ETH; Georg Wilckens, ETH
• Sub-Nyquist Detection and Estimation Using Equivalent-time Sampling, Tarig Ballal, King Abdullah University of Science and Technology ; Tareq Al-Naffouri*, King Abdullah University of Science and Technology
• Estimation with Norm Regularization, Arindam Banerjee*, University Of Minnesota; Sheng Chen, University Of Minnesota; Farideh Fazayeli, University Of Minnesota; Vidyashankar Sivakumar, University Of Minnesota
• Learning Probability Distributions over Structured Spaces, Arthur Choi, UC Los Angeles; Guy Van den Broeck, KU Leuven; Adnan Darwiche*, UC Los Angeles
• Polar codes for the broadcast channel with confidential messages and constrained randomization, Rémi Chou, Georgia Tech; Matthieu Bloch*, Georgia Tech
• Pipelining for Accuracy in Randomized Digital Computation, Marc Riedel*, University Of Minnesota
• Learning from pairwise comparisons, Mark Davenport*, Georgia Tech
• See-Through Imaging and Occupancy Estimation with RF Signals, Yasamin Mostofi*, UC Santa Barbara
• Clustering multi-way data: A novel algebraic approach Eric Kernfeld, University Of Washington; Misha Kilmer, Tufts University; Shuchin Aeron*, Tufts University 
• A graph sampling perspective for semi-supervised learning Aly El Gamal*, USC; Aamir Anis, USC; Salman Avestimehr, USC; Antonio Ortega, USC
• Stochastic Iterative Greedy Algorithms for Sparse Reconstruction, Nam Nguyen, MIT; Deanna Needell*, Claremont McKenna College; Tina Woolf, Claremont Graduate University
• High dimensional variable selection: a decision theoretic approach, Oluwasanmi Koyejo*, Stanford; Rajiv Khanna, University Of Texas; Joydeep Ghosh, University Of Texas; Russell Poldrack, Stanford
• Elementary estimators for high-dimensional statistical models Eunho Yang, IBM T.J Watson; Aurelie Lozano, IBM T.J Watson; Pradeep Ravikumar*, UT Austin
• Inference in High-Dimensional Varying Coefficient Models, Mladen Kolar*, University Of Chicago; Damian Kozbur, ETH
• Landmarking Manifolds with Gaussian Processes, John Paisley*, Columbia
• Learning in Repeated Auctions Kareem Amin, UPenn; Afshin Rostamizadeh*, Google; Umar Syed, Google
• Machine Learning in Apache Spark Ameet Talwalkar*, UC Los Angeles
• Optimistic Concurrency Control in the Design and Analysis of Parallel Learning Algorithms, Joseph Gonzalez*, UC Berkeley; Xinghao Pan, UC Berkeley; Stefanie Jegelka, MIT; Joseph Bradley, UC Berkeley; Michael Jordan, UC Berkeley
• Subspace Clustering with Ordered Weighted L1 minimization Ulas Ayaz*, Brown University
• Correlation Clustering on Big Graphs, Dimitris Papailiopoulos*, UC Berkeley; Xinghao Pan, UC Berkeley; Benjamin Recht, UC Berkeley; Kannan Ramchandran, UC Berkeley; Michael I. Jordan, UC Berkeley
• Pessimistic active learning using robust bias-aware prediction Anqi Liu, University Of Illinois Chicago; Lev Reyzin*, University Of Illinois Chicago; Brian Ziebart, University Of Illinois Chicago

Here are some videos presentations of some of the talks:

• Interference in finite-sized highly dense millimeter wave networks, Kiran Venugopal, UT Austin Matthew Valenti, West Virginia University Robert Heath, UT Austin
• Information guided sequential compressed sensing, Yao Xie, Georgia Tech
• A Reusable Holdout for Guaranteed Classifier Validation, Moritz Hardt, IBM Research - Almaden
• Unified scaling of polar codes: error exponent, scaling exponent, moderate deviations, and error floors, Marco Mondelli, EPFL Seyed Hamed Hassani, ETH Ruediger Urbanke, EPFL

 

 

  Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.