Approximating Kernels at the Speed of Light

It could be one of those Hamming's time blog entries, except it's not. There will be some announcements about this in the coming future. In the meantime and without further ado, here is the preprint:

Random Projections through multiple optical scattering: Approximating kernels at the speed of light  by Alaa Saade, Francesco Caltagirone, Igor Carron, Laurent Daudet, Angélique Drémeau, Sylvain Gigan, Florent Krzakala

Random projections have proven extremely useful in many signal processing and machine learning applications. However, they often require either to store a very large random matrix, or to use a different, structured matrix to reduce the computational and memory costs. Here, we overcome this difficulty by proposing an analog, optical device, that performs the random projections literally at the speed of light without having to store any matrix in memory. This is achieved using the physical properties of multiple coherent scattering of coherent light in random media. We use this device on a simple task of classification with a kernel machine, and we show that, on the MNIST database, the experimental results closely match the theoretical performance of the corresponding kernel. This framework can help make kernel methods practical for applications that have large training sets and/or require real-time prediction. We discuss possible extensions of the method in terms of a class of kernels, speed, memory consumption and different problems.

Some of my co-authors ( sylvain Gigan, Laurent Daudet, Alaa Saade , Krzakala Florent ) mentioned the news of the preprint on their twitter feed:


Need fast random projections for your favorite machine learning algorithm? We made them (literally) at the speed of light! arXiv:1510.06664

— Krzakala Florent (@KrzakalaF) October 23, 2015 ‏

We are building hardware to speed up Machine Learning w/ light https://t.co/I7aeWubAFt @IgorCarron @Laurent_Daudet @KrzakalaF @sylvaingigan

— Alaa Saade (@saade_alaa) October 23, 2015


"Random Projections through multiple optical scattering: Approximating kernels at the speed of light"@IgorCarron @KrzakalaF @Laurent_Daudet

— sylvain Gigan (@sylvaingigan) October 23, 2015


Credit: NASA/NOAA

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

50 years of Data Science by David Donoho

The genesis of Nuit Blanche is intimately linked to Dave Donoho's work (and its webpage). Today is a preprint based on a keynote speech Dave gave at the John W. Tukey 100th Birthday Celebration at Princeton University that took place on September 18, 2015, on defining Data Science. Here is the attendant preprint: 50 years of Data Science by David Donoho

More than 50 years ago, John Tukey called for a reformation of academic statistics. In `The Future of Data Analysis', he pointed to the existence of an as-yet unrecognized science, whose subject of interest was learning from data, or `data analysis'. Ten to twenty years ago, John Chambers, Bill Cleveland and Leo Breiman independently once again urged academic statistics to expand its boundaries beyond the classical domain of theoretical statistics; Chambers called for more emphasis on data preparation and presentation rather than statistical modeling; and Breiman called for emphasis on prediction rather than inference. Cleveland even suggested the catchy name \Data Science" for his envisioned field. A recent and growing phenomenon is the emergence of \Data Science" programs at major universities, including UC Berkeley, NYU, MIT, and most recently the Univ. of Michigan, which on September 8, 2015 announced a $100M \Data Science Initiative" that will hire 35 new faculty. Teaching in these new programs has signi cant overlap in curricular subject matter with traditional statistics courses; in general, though, the new initiatives steer away from close involvement with academic statistics departments. This paper reviews some ingredients of the current \Data Science moment", including recent commentary about data science in the popular media, and about how/whether Data Science is really diff erent from Statistics. The now-contemplated field of Data Science amounts to a superset of the fi elds of statistics and machine learning which adds some technology for `scaling up' to `big data'. This chosen superset is motivated by commercial rather than intellectual developments. Choosing in this way is likely to miss out on the really important intellectual event of the next fty years. Because all of science itself will soon become data that can be mined, the imminent revolution in Data Science is not about mere `scaling up', but instead the emergence of scienti c studies of data analysis science-wide. In the future, we will be able to predict how a proposal to change data analysis work ows would impact the validity of data analysis across all of science, even predicting the impacts fi eld-by- field. Drawing on work by Tukey, Cleveland, Chambers and Breiman, I present a vision of data science based on the activities of people who are `learning from data', and I describe an academic eld dedicated to improving that activity in an evidence-based manner. This new field is a better academic enlargement of statistics and machine learning than today's Data Science Initiatives,while being able to accommodate the same short-term goals. 

 There are many passages I liked, including this one:

"..Machine Translation research fi nally re-emerged decades later from the Piercian limbo, but only because it found a way to avoid a susceptibility to Pierce's accusations of glamor and deceit. A research team led by Fred Jelinek at IBM, which included true geniuses like John Cocke, began to make de nite progress towards machine translation based on an early application of the common task framework. A key resource was data: they had obtained a digital copy of the so-called Canadian Hansards, a corpus of government documents which had been translated into both English and French. By the late 1980's DARPA was convinced to adopt the CTF as a new paradigm for machine translation research. NIST was contracted to produce the sequestered data and conduct the refereeing, and DARPA challenged teams of researchers to produce rules that correctly classifi ed under the CTF...."

 h/t Diego and Victoria

 

David Donoho reflects on "50 Years of #DataScience" >https://t.co/ntUnWMaQpR #Statistics #BigData HT @victoriastodden pic.twitter.com/dwZ0yd0S2g

— Dr. Diego Kuonen (@DiegoKuonen) October 13, 2015

Credit: NASA/Johns Hopkins University Applied Physics Laboratory/Southwest Research Institute

Pluto's Blue Sky
Release Date: October 8, 2015
Keywords: MVIC, Pluto, RalphPluto's haze layer shows its blue color in this picture taken by the New Horizons Ralph/Multispectral Visible Imaging Camera (MVIC). The high-altitude haze is thought to be similar in nature to that seen at Saturn’s moon Titan. The source of both hazes likely involves sunlight-initiated chemical reactions of nitrogen and methane, leading to relatively small, soot-like particles (called tholins) that grow as they settle toward the surface. This image was generated by software that combines information from blue, red and near-infrared images to replicate the color a human eye would perceive as closely as possible.



Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Clustering Noisy Signals with Structured Sparsity Using Time-Frequency Representation - implementation -

Or just let me know of his recent arxiv preprint:

Hi Igor,


Hope all is well,

The following manuscript on sparse clustering using time-frequency representation by students Tom Hope, Avishai Wagner and me might be of interest to your blog's reader:
http://arxiv.org/abs/1510.05214

Best,
Or

 Thanks Or ! Here it is:Clustering Noisy Signals with Structured Sparsity Using Time-Frequency Representation by Tom Hope, Avishai Wagner, Or Zuk

We propose a simple and efficient time-series clustering framework particularly suited for low Signal-to-Noise Ratio (SNR), by simultaneous smoothing and dimensionality reduction aimed at preserving clustering information. We extend the sparse K-means algorithm by incorporating structured sparsity, and use it to exploit the multi-scale property of wavelets and group structure in multivariate signals. Finally, we extract features invariant to translation and scaling with the scattering transform, which corresponds to a convolutional network with filters given by a wavelet operator, and use the network's structure in sparse clustering. By promoting sparsity, this transform can yield a low-dimensional representation of signals that gives improved clustering results on several real datasets.

SPARCWave is on Github: https://github.com/avishaiwa/SPARCWave 

    Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

CSJob: Postdoc positions in Graph Signal Processing / Dimension Reduction / Compressive Learning, Rennes, France

  Here is an announcement of interest:

Postdoc positions in Graph Signal Processing / Dimension Reduction / Compressive Learning

The PANAMA team is welcoming applications from highly qualified candidates for postdoctoral positions in the context of the project PLEASE (Projection, Learning and Sparsity for Efficient data processing).

The PLEASE project gathers a small research group to explore the frontiers of Signal Processing and Machine Learning under the auspices of sparsity and low-dimensional projections. With its task force of graduate students and postdocs with different scientific backgrounds, and its program of international visitors, PLEASE develops the mathematical and algorithmic foundations of new ways to acquire, analyze and process the information content of complex data, streams and collections.

The investigations within the PLEASE project put a special focus on the theoretical foundations of compressive statistical learning, the mathematical analysis of provably good and efficient dictionary learning, and the investigation of the emerging field of graph signal processing. The recruited postdocs are expected to contribute to these objectives by proposing new approaches with solid underpinning mathematics, demonstrating their impact on selected applications, and disseminating them through software.

The recruited postdocs will conduct a vigorous research program within the scope of the project, and are expected to show independence and team working attitude at the same time. The successful candidates, which can come from different areas (applied mathematics and statistics, signal processing, machine learning, information theory, computer science) are expected to bring their expertise to the PLEASE task force and will be encouraged to develop collaborations within the PANAMA team (audio signal processing) as well as with other groups at IRISA / Inria Rennes. The positions are endowed with travel, computing, and experimental resources.

To apply: applicants are requested to send a detailed CV, a list of publications and a brief statement of research interests. This material, together with two letters of reference, shall be sent to Remi.Gribonval@inria.fr (cc: Stephanie.lemaile@inria.fr).

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Dual Principal Component Pursuit

Some new phase transition, now in Principal Component Pursuit:

 

Dual Principal Component Pursuit by Manolis C. Tsakiris, Rene Vidal

We consider the problem of outlier rejection in single subspace learning. Classical approaches work directly with a low-dimensional representation of the subspace. Our approach works with a dual representation of the subspace and hence aims to find its orthogonal complement. We pose this problem as an $\ell_1$-minimization problem on the sphere and show that, under certain conditions on the distribution of the data, any global minimizer of this non-convex problem gives a vector orthogonal to the subspace. Moreover, we show that such a vector can still be found by relaxing the non-convex problem with a sequence of linear programs. Experiments on synthetic and real data show that the proposed approach, which we call Dual Principal Component Pursuit (DPCP), outperforms state-of-the art methods, especially in the case of high-dimensional subspaces.

    Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

CSjob: Postdoc, Statistical Inference, Compressed Sensing, Information Theory, Machine learning and Statistical Physics, ENS Paris, France

Florent let me know of this postdoc opportunity in his group

Dear friends and colleagues,

I would like to invite applications for postdoctoral positions in my group in Ecole Normale Superieure in Paris, in the context of the project SPARCS (Statistical Physics Approach to Reconstruction in Compressed Sensing) The appointments are intended to start in the fall of 2016 and will be for 1+1 years.

The candidates can come from different areas (Statistical Physics, Signal Processing, Applied Mathematics, Information Theory, Statistical Inference and Machine learning).

For more information, please visit the post-doc announcement page: postdoc.krzakala.org

Do not hesitate to spread the information around you. I apologize if you receive this mail more than once.

Best regards,

Florent Krzakala

From the page:

Postdoctoral positions in Krzakala's group in Ecole Normale Superieure, Paris:

Statistical Inference, Compressed Sensing, Information Theory, Machine learning and Statistical Physics

I would like to invite applications for postdoctoral positions funded by the European Research Council Starting Grant program, in the context of the project SPARCS (Statistical Physics Approach to Reconstruction in Compressed Sensing) in my group in Ecole Normale in Paris. The appointments are intended to start in the fall of 2016 and will be for 1+1 years.

The SPARCS project is building a research group in the very center of Paris in Ecole Normale, concentrated on inverse problems, information theory, graphical models, compressed sensing and statistical physics (but also open to other issues in inference, clustering, community detection, machine learning, neural networks and all aspects of the statistical physics of disordered and complex systems). A strong visitor program and a series of small workshops are also organized and several graduate students have already joined the group. The project is developed in collaboration with Marc Mezard (Ecole Normale Superieure), Lenka Zdeborova (CEA Saclay) and many groups in the Parisian region.

This project places an emphasis on interdisciplinarity, and aims to achieve progress by bringing together postdocs with different scientific backgrounds. More specifically, the candidates can come from different areas (signal processing, applied mathematics, statistical physics, information theory, inference, machine learning and neural networks) and are expected to bring their expertise. Successful candidates will thus conduct a vigorous research program within the scope of the project, and are expected to show independence and team working attitude at the same time. Click here to see recent works from the SPARCS team or click here to see the group webpage.

The SPARCS members also keep close contacts with other researchers of the Ecole Normale Superieure, of the University Paris-Sud and of the CEA Saclay, all of them based closed by. The ENS is conveniently located in the very center of Paris. The positions are endowed with travel and computing resources.

Keywords: Machine Learning, Signal processing, information theory, graphical models, Bayesian inference, compressed sensing, error correcting codes, spatial coupling, Belief Propagation, Message Passing, Tomography. Statistical physics - glasses, spin glasses, random optimization problems, cavity method, replica method - c, c++, matlab, julia, python.

Deadline for applying: 31st December 2015.

To apply, and for further information: florent.krzakala@ens.fr. The candidates should send their detailed cv (including list of publication, presentations, citations etc.), and 1 page letter of motivation explaining why they want to work on this subject, what is their related experience, and present a short project. Preselected candidates should be ready to provide two letters of recommendation at a later stage, and are expected to be available to come to Paris for an interview.

  Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Tensor vs Matrix Methods: Robust Tensor Decomposition under Block Sparse Perturbations

I wonder how these tensor techniques will be doing when used in combination with a randomization a la GoDec. We all know that this technique is fast in robust PCA for the matrix form, so I would expect similar speed-up for a tensot approach. Without further ado: Tensor vs Matrix Methods: Robust Tensor Decomposition under Block Sparse Perturbations by Anima Anandkumar, Prateek Jain, Yang Shi, U.N. Niranjan

Robust tensor CP decomposition involves decomposing a tensor into low rank and sparse components. We propose a novel non-convex iterative algorithm with guaranteed recovery. It alternates between low-rank CP decomposition through gradient ascent (a variant of the tensor power method), and hard thresholding of the residual. We prove convergence to the globally optimal solution under natural incoherence conditions on the low rank component, and bounded level of sparse perturbations. We compare our method with natural baselines, viz., which apply robust matrix PCA either to the flattened tensor, or to the matrix slices of the tensor. Our method can provably handle a far greater level of perturbation when the sparse tensor is block-structured. This naturally occurs in many applications such as the activity detection task in videos. Our experiments validate these findings. Thus, we establish that tensor methods can tolerate a higher level of gross corruptions compared to matrix methods

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

CSjob: Postdoc, Computational Sensing Strategies for Low-Complexity Signal Models, Louvain, Belgium

Laurent Jacques let me know of this postdoc opportunity. From the page:
 

Applications are collected till December 15th 2015, last limit.

Open postdoctoral position:

ALTERSENSE: Computational Sensing Strategies for Low-Complexity Signal Models

Discipline keywords: Compressive classification, Non-linear/1-bit/quantized compressive sensing, convex optimization, resource management, hyperspectral imaging, towards low-power sensor design

Location: ISPGroup, UCL, Belgium.

Introduction

The research group of Prof. Laurent Jacques in the Image and Signal Processing Group (ISPGroup) of the University of Louvain-la-Neuve in Belgium (UCL) opens one position for a postdoctoral researcher to work on "ALTERSENSE" ("Computational Sensing Strategies for Low-Complexity Signal Models") a new project funded by the Belgian Fund for Scientific Research - FNRS.

AlterSense Project

With the steady development of technology in numerous scientific fields such as biomedical sciences, astronomy, optics or computer vision, big challenges are raised by the design of new and efficient data acquisition systems. These must often comply with contradictory goals such as sampling high dimensional domains, devising fast and low-complexity recording processes, reaching low-power consumption sensors, facing limited capacity communication channels and being at the same time robust against multiple noise sources.

Noticeably, the final objective of those sensors is invariably the same: providing at the very end of the data sensing chain processed and interpretable information, either for human or for automatic (machine) processing. This is the case, for instance,

• in Satellite or Biomedical Imaging: for numerous imaging technologies, such as Hyperspectral Imaging, Magnetic Resonance Imaging, Computed Tomography, or Positron-Electron Tomography, segmenting images or data volumes in a few categories of connected pixel areas (e.g., spectral endmembers, biological tissues) is of particular interest for simplifying information;
• in Low-Power Dynamic Sensing: in the general development of the Internet-of-Things (IoT) or of ultra-low energy sensors (e.g., biomedical).

As already formalized by Kolmogorov in the 60’s, all those applications are possible since “meaningful signals follow low-complexity descriptions”: their informative content is materialized by highly structured “patterns” whose intrinsic parameterization is considerably reduced compared to the high dimensionality of the ambient domain. By contrast, purely noisy signals often carry no information content, they are highly unstructured and require much more parameters to be characterized.

Leveraging the paradigm shift introduced by the Compressed Sensing theory where signal sensing is adjusted to prior signal models, ALTERSENSE aims to develop a “Computational Sensing” framework where, departing from the mere signal reconstruction objective, the sensing stage is adapted and simplified to perform specific computational tasks ahead of the final data processing. We will pursue this objective:

• for ubiquitous data processing tasks: for detecting, segmenting or classifying informative signals;
• for high-dimensional signals (e.g., hyperspectral or dynamic images) following low-complexity descriptions such as sparse/low-rank signal models or linear dynamical systems (LDS);
• for conveniently balancing sensing time/complexity, data quantization and transmission (as in 1-bit CS), final data processing accuracy and data processing time as any other limited resources.

ALTERSENSE will also instantiate this theoretical research on two case studies with high scientific impacts, i.e., we will define computational sensing strategies for:

• hyperspectral data volumes whose high dimensionality poses real challenges both in sensor design and data processing, with applications in biomedical and satellite imaging;
• spatio-temporal event processing, in the general development of low-power (compressed sensing) sensors, e.g., for millimetric 2-D grid of EEG/ECG probes, efficiently detecting specific events (such as strokes).

Applicant's profile:

• PhD in applied mathematics, electrical engineering or theoretical physics
• Strong background in signal processing, compressed sensing and inverse problem solving.
• Knowledge in measure concentration phenomenon, signal detection, classification methods, high dimensional data processing, convex optimisation.
• Excellent programming skills in a numerical language (matlab or python);
• Good communications skills, both written and oral, in English.
• Knowing French *is not* required (the research group is international)
• Mobility criterion: The position is open to all nationalities, including Belgian, but the condition is to have spent less than 12 months over the last three years in Belgium.

More information about the project can be obtained upon request.

We offer:

• A research position in a dynamic environment, working on leading-edge theories and applications with international contacts;
• A research team constituted of one professor, another postdoctoral researcher, and 3 PhD students on topics related to AlterSense;
• A 24-month position funded by the Belgian NSF (FNRS)
• The funding is a scholarship and Visa will be needed for a non-EU researcher.

Application:

Applications should include:

1. a detailed resume (in pdf) + list of publications;
2. 2-page research statement (in pdf) explaining also why the candidate is interested in working in the research topics described above and how it is connected to his/her PhD background
3. the names and complete addresses of two reference persons that can be contacted

Please send applications by email to:

• Prof. Laurent Jacques, laurent.jacquesuclouvain.be

Candidate Selection

• Pre-selection of candidates based on their application files
• (remote) Interview of the short-listed candidates
• The position remains open until selection of a good candidate
• The successful candidate can be hired from January 1st 2016, but we have some flexibility to start the position (a bit) later (to be discussed)

 

 

    Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

ℓ1-regularized Neural Networks are Improperly Learnable in Polynomial Time

 

 

Here is a very interesting find:

Theorem 3 presents a more general result, showing that any activation function that is sigmoid-like or ReLU-like leads to the computational hardness, even if the loss function ℓis convex

and from the conclusion:

Although the recursive kernel method doesn’t outperform the LeNet5 model, the experiment demonstrates that it does learn better predictors than fully connected neural networks such as the multi-layer perceptron. The LeNet5 architecture encodes prior knowledge about digit recogniition via the convolution and pooling operations; thus its performance is better than the generic architectures

ℓ1-regularized Neural Networks are Improperly Learnable in Polynomial Time by Yuchen Zhang, Jason D. Lee, Michael I. Jordan

We study the improper learning of multi-layer neural networks. Suppose that the neural network to be learned has k hidden layers and that the ℓ1-norm of the incoming weights of any neuron is bounded by L. We present a kernel-based method, such that with probability at least 1−δ, it learns a predictor whose generalization error is at most ϵ worse than that of the neural network. The sample complexity and the time complexity of the presented method are polynomial in the input dimension and in (1/ϵ,log(1/δ),F(k,L)), where F(k,L) is a function depending on (k,L) and on the activation function, independent of the number of neurons. The algorithm applies to both sigmoid-like activation functions and ReLU-like activation functions. It implies that any sufficiently sparse neural network is learnable in polynomial time.

 
 

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Neural Networks with Few Multiplications / BinaryConnect - implementation -

fastml extra did all the work !

The reddit discussion is heavy on the hardware implementation mostly because having fewer multiplications makes FPGA a reasonable solution (multiplication in FPGA takes space and time). The fascinating part of this approach is the quantization in both the weight and the backpropagation scheme. Without further ado: Neural Networks with Few Multiplications by Zhouhan Lin, Matthieu Courbariaux, Roland Memisevic, Yoshua Bengio

For most deep learning algorithms training is notoriously time consuming. Since most of the computation in training neural networks is typically spent on floating point multiplications, we investigate an approach to training that eliminates the need for most of these. Our method consists of two parts: First we stochastically binarize weights to convert multiplications involved in computing hidden states to sign changes. Second, while back-propagating error derivatives, in addition to binarizing the weights, we quantize the representations at each layer to convert the remaining multiplications into binary shifts. Experimental results across 3 popular datasets (MNIST, CIFAR10, SVHN) show that this approach not only does not hurt classification performance but can result in even better performance than standard stochastic gradient descent training, paving the way to fast, hardware-friendly training of neural networks.

From the conclusion:

Directions for future work include exploring actual implementations of this approach (for example, using FPGA), seeking more efficient ways of binarization, and the extension to recurrent neural networks.

The implementation for BinaryConnect which is revisited in this preprint can be found at: https://github.com/AnonymousWombat/BinaryConnect

BinaryConnect was mentioned in this reference: Courbariaux, M., Bengio, Y., and David, J.-P. (2015). Binaryconnect:Training deep neural networks with binary weights during propagations.but as Yoav Goldberg, I cannot find on the interwebs.

  Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.