Nuit Blanche: K-SVD/IPR: Learning Incoherent Dictionaries for Sparse Approximation using Iterative Projections and Rotations

Friday, May 10, 2013

K-SVD/IPR: Learning Incoherent Dictionaries for Sparse Approximation using Iterative Projections and Rotations - implementation -

Back in July of last year, I mentioned the following work Learning Incoherent Dictionaries for Sparse Approximation using Iterative Projections and Rotations by Daniele Barchiesi, Mark D. Plumbley. The abstract reads:

This article deals with learning dictionaries for sparse approximation whose atoms are both adapted to a training set of signals and mutually incoherent. To meet this objective, we employ a dictionary learning scheme consisting of sparse approximation followed by dictionary update and we add to the latter a decorrelation step in order to reach a target mutual coherence level. This step is accomplished by an iterative projection method followed by a rotation of the dictionary. Experiments on musical audio data illustrate that the proposed algorithm can learn highly incoherent dictionaries while providing a sparse approximation with good signal to noise ratio. We show that using dictionaries with low mutual coherence leads to a faster rate of decay of the residual when using MP for sparse approximation.

Daniele reminded me of the location of its implementation. The attendant incoherent dictionary learning implementation is here. It is also part of SMALLbox, a framework for sparse representation and dictionary learning. From the page:

The SMALLbox, a framework for sparse representation and dictionary learning.

SMALLbox is a new foundational framework for processing signals using adaptive sparse structured representations.

The main aim of SMALLbox is to become a test ground for exploration of new provably good methods to obtain inherently data-driven sparse models, which are able to cope with large-scale and complicated data.

The main focus of research in the area of sparse representations is in developing reliable algorithms with provable performance and bounded complexity. Yet, such approaches are simply inapplicable in many scenarios for which no suitable sparse model is known. Moreover, the success of sparse models heavily depends on the choice of a “dictionary” to reflect the natural structures of a class of data. Inferring a dictionary from training data is a key to the extension of sparse models for new exotic types of data.

The SMALLbox provides an easy way to evaluate these methods against state-of-the art alternatives in a variety of standard signal processing problems. This is achieved trough a unifying interface that enables a seamless connection between the three types of modules: problems, dictionary learning algorithms and sparse solvers.

In addition, it provides interoperability between existing state-of-theart toolboxes. As an open source MATLAB toolbox, the SMALLbox can be seen as a tool for reproducible research in the sparse representations research community.