Hamming's Time: The Desperate Researchers of Computer Vision

Image Credit: NASA/JPL-Caltech/Space Science Institute

Back in August 2014, I mentioned this roundtable at the Technion ( Is Deep Learning the Final Frontier and the End of Signal Processing ? ). Then Yoshua provided his view on Deep Learning. Two years and half later, Miki just wrote the following op-ed in SIAM News: Deep, Deep Trouble: Deep Learning’s Impact on Image Processing, Mathematics, and Humanity. It is difficult to understate how much of The Great Convergence we are witnessing. This is how it starts.

I am really confused. I keep changing my opinion on a daily basis, and I cannot seem to settle on one solid view of this puzzle. No, I am not talking about world politics or the current U.S. president, but rather something far more critical to humankind, and more specifically to our existence and work as engineers and researchers. I am talking about…deep learning.....

.....Now back to the main question: should we be pleased about emerging solutions based on deep learning? Is our frustration justified? What is the role of deep learning in imaging science? These questions present themselves when researchers in the community meet at conferences, and the answers are diverse and confusing. The facts speak loudly for themselves; in most cases, deep learning-based solutions lack mathematical elegance and offer very little interpretability of the found solution or understanding of the underlying phenomena. On the positive side, however, the performance obtained is terrific. This is clearly not the school of research we have been taught, and not the kind of science we want to practice. Should we insist on our more rigorous ways, even at the cost of falling behind in terms of output quality? Or should we fight back and seek ways to fuse ideas from deep learning into our more solid foundations?
To further complicate this story, certain deep learning-based contributions bear some elegance that cannot be dismissed. Such is the case with the style-transfer problem, which yielded amazingly beautiful results, and with inversion ideas of learned networks used to synthesize images out of thin air, as Google’s Deep Dream project does. A few years ago we did not have the slightest idea how to formulate such complicated tasks; now they are solved formidably as a byproduct of a deep neural network trained for the completely extraneous task of visual classification. 

The whole thing is here.
h/t Ulugbeck

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Spectral Ergodicity in Deep Learning Architectures via Surrogate Random Matrices - implementation -

Mehmet just sent me the following

Hello Igor,

Hope you are all well.

The recent manuscript [1,3] might be of interest to nuit-blanche blog readers. The manuscript has developed an ergodicity measure for spectral analysis using KL divergence and applied to circular ensembles. The software package is also available along with the Python notebook.

Please feel free to use the material in your blog.

Many regards,
Mehmet

[1] https://arxiv.org/abs/1704.08303
[2] https://pypi.python.org/pypi/bristol
https://github.com/msuzen/bristol/blob/master/works/spectralErgodicity/ergodicity_random_matrix_17a.ipynb

[3] Spectral Ergodicity in Deep Learning Architectures via Surrogate Random Matrices by Mehmet Süzen, Cornelius Weber, Joan J. Cerdà

Using random matrix ensembles, mimicking weight matrices from deep and recurrent neural networks, we investigate how increasing connectivity leads to higher accuracy in learning with a related measure on eigenvalue spectra. For this purpose, we quantify spectral ergodicity based on the Thirumalai-Mountain (TM) metric and Kullbach-Leibler (KL) divergence. As a case study, different size circular random matrix ensembles, i.e., circular unitary ensemble (CUE), circular orthogonal ensemble (COE), and circular symplectic ensemble (CSE), are generated. Eigenvalue spectra are computed along with the approach to spectral ergodicity with increasing connectivity size. As a result, it is argued that success of deep learning architectures attributed to spectral ergodicity conceptually, as this property prominently decreases with increasing connectivity in surrogate weight matrices. 

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Nuit Blanche in Review (April 2017)

April 2017 was pretty eventful. At LightOn, we just came out with our second newsletter. We attended ICLR in Toulon, and much like Libby , we noticed the millenia CPU paper as well as the current thought processes going on in finding rules of thumbs for new architectures on the algorithm side (see Understanding section below) and on the hardware side (see MLHardware section). We also noted that some of these insights might be automated (see Architecture section). Kudos to the ICLR organizers for streamlining and eventually putting the videos of the event in real time. Other highlights included what is going on in sequencing (see Sunday Morning Insight), a thesis and some jobs. Enjoy !

Nuit Blanche in Review for March 2017 is here.

Sunday Morning Insight:

• Sunday Morning Insight: "No Need for the Map of a Cat, Mr Feynman" or The Long Game in Nanopore Sequencing.

Implementation (implementation tag)

• CSVideoNet: A Real-time End-to-end Learning Framework for High-frame-rate Video Compressive Sensing - implementation -
• Improved Training of Wasserstein GANs - implementation -
• AutoGP: Exploring the Capabilities and Limitations of Gaussian Process Models - implementation -

Architectures (MetaLearning tag)

• Designing Neural Network Architectures using Reinforcement Learning
• Random Feature Expansions for Deep Gaussian Processes / AutoGP: Exploring the Capabilities and Limitations of Gaussian Process Models - implementation -

ML Hardware (MLHardware tag)

• In-Datacenter Performance Analysis of a Tensor Processing Unit​ (TM)
• MLHardware: Understanding and Optimizing Asynchronous Low-Precision Stochastic Gradient Descent / WRPN: Training and Inference using Wide Reduced-Precision Networks
• It's all about convolutions: YodaNN, TrueNorth computing, FPGAs and a comparison between HOG and CNNs on Hardware

CSHardware (CSHardware tag)

• Lensless Imaging with Compressive Ultrafast Sensing

In-depth

• Online Natural Gradient as a Kalman Filter
• Stochastic Gradient Descent as Approximate Bayesian Inference
• Phase Transitions of Spectral Initialization for High-Dimensional Nonconvex Estimation
• F2F: A Library For Fast Kernel Expansions

Understanding

• Understanding Shallower Networks
• Understanding Deep Neural Networks; Opening the Black Box, Energy Propagation in DNNs, Exploring loss function topology

Thesis (Thesis tag)

• Thesis: Randomized Algorithms for Large-Scale Data Analysis by Farhad Pourkamali-Anaraki

Highly Technical Reference page:

• Highly Technical Reference Page: The Incredible PyTorch

Videos:

• Saturday Morning Videos: #ICLR2017 videos

Conferences

• #ICLR2017 Monday Morning Program
• #ICLR2017 Monday Afternoon Program
• #ICLR2017 Tuesday Morning Program
• #ICLR2017 Tuesday Afternoon Program
• ICLR2017, third and last day.

Jobs: (CSjobs tag)

• Two jobs, Research Scientists, NVIDIA

Credit photo: NASA, JPL-Caltech, SSI,  enhanced by Sophia Nasr

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Designing Neural Network Architectures using Reinforcement Learning

At ICLR, I noted these figures below that tells the story of the need for certain operations in neural networks as a function of their depth location: 

Designing Neural Network Architectures using Reinforcement Learning by Bowen Baker, Otkrist Gupta, Nikhil Naik, Ramesh Raskar

At present, designing convolutional neural network (CNN) architectures requires both human expertise and labor. New architectures are handcrafted by careful experimentation or modified from a handful of existing networks. We introduce MetaQNN, a meta-modeling algorithm based on reinforcement learning to automatically generate high-performing CNN architectures for a given learning task. The learning agent is trained to sequentially choose CNN layers using Q-learning with an ϵ-greedy exploration strategy and experience replay. The agent explores a large but finite space of possible architectures and iteratively discovers designs with improved performance on the learning task. On image classification benchmarks, the agent-designed networks (consisting of only standard convolution, pooling, and fully-connected layers) beat existing networks designed with the same layer types and are competitive against the state-of-the-art methods that use more complex layer types. We also outperform existing meta-modeling approaches for network design on image classification tasks.

Models found by MetaQNN are located here: https://bowenbaker.github.io/metaqnn/


Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Online Natural Gradient as a Kalman Filter

So Kalman Filters can help in hyperparameter search in Recurrent Learning.

Online Natural Gradient as a Kalman Filter by Yann Ollivier

We establish a full relationship between Kalman filtering and Amari's natural gradient in statistical learning. Namely, using an online natural gradient descent on data log-likelihood to evaluate the parameter of a probabilistic model from a series of observations, is exactly equivalent to using an extended Kalman filter to estimate the parameter (assumed to have constant dynamics).
In the i.i.d. case, this relation is a consequence of the "information filter" phrasing of the extended Kalman filter. In the recurrent (state space, non-i.i.d.) case, we prove that the joint Kalman filter over states and parameters is a natural gradient on top of real-time recurrent learning (RTRL), a classical algorithm to train recurrent models.
This exact algebraic correspondence provides relevant settings for natural gradient hyperparameters such as learning rates or initialization and regularization of the Fisher information matrix.

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Saturday Morning Videos: #ICLR2017 videos

Here are the videos of this year's ICLR2017 meeting.

Monday April 24, 2017

Morning Session – Session Chair: Dhruv Batra 

Opening remarks, video starts at 12:15

9.00 - 9.40 Invited talk 1: Eero Simoncelli Elucidating and testing hierarchical sensory models through synthesis, Video starts at 26:00

9.40 - 10.00 Contributed talk 1: End-to-end Optimized Image Compression Video starts at 1:08:00

10.00 - 10.20 Contributed talk 2: Amortised MAP Inference for Image Super-resolution Video starts at 1:26:30

10.30 - 12.30 Poster Session 1 (Conference Papers, Workshop Papers)

 

h/t Libby for catching this one. From this paper.

Afternoon Session – Session Chair: Joan Bruna (sponsored by Baidu)

14.30 - 15.10 Invited talk 2: Benjamin Recht What can Deep Learning learn from linear regression Video starts at 18:30

15.10 - 15.30 Contributed Talk 3: Understanding deep learning requires rethinking generalization - BEST PAPER AWARD, Video starts at 53:30

15.30 - 15.50 Contributed Talk 4: Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima Video starts at 1;12:30

15.50 - 16.10 Contributed Talk 5: Towards Principled Methods for Training Generative Adversarial Networks Video starts at 1;30:00

16.30 - 18.30 Poster Session 2 (Conference Papers, Workshop Papers)

Tuesday April 25, 2017

Morning Session – Session Chair: Tara Sainath (sponsored by Google)

9.00 - 9.40 Invited talk 1: Chloé-Agathe Azencott High dimensional feature selection in precision medicine Video starts at 13;24
9.40 - 10.00 Contributed talk 1: Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data - BEST PAPER AWARD ,Video starts at 53:10
10.00 - 10.20 Contributed talk 2: Learning Graphical State Transitions  Video starts at 1;14;50
10.30 - 12.30 Poster Session 1 (Conference Papers, Workshop Papers)

Afternoon Session – Session Chair: Raia Hasdell (sponsored by Amazon)

14.00 - 16.00 Poster Session 2 (Conference Papers, Workshop Papers)
16.15 - 17.00 Invited talk 2: Riccardo Zecchina Video starts at 7:05
17.00 - 17.20 Contributed Talk 3: Learning to Act by Predicting the Future Video starts at 53:50
17.20 - 17.40 Contributed Talk 4: Reinforcement Learning with Unsupervised Auxiliary Tasks Video starts at 1:15:30
17.40 - 18.00 Contributed Talk 5: Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic Video starts at 1;37;10

18.00 - 18.30 Group photo at the RCT Stadium

Morning Session – Session Chair: Slav Petrov
9.00 - 9.40 Invited talk 1: Regina Barzilay Moving beyond supervised realm Video starts at 3;15 + last 8 minutes of this presentation on this video
9.40 - 10.00 Contributed talk 1: Learning End-to-End Goal-Oriented Dialog Video starts at 27:34
10.00 - 10.20 Contributed talk 2: Multi-Agent Cooperation and the Emergence of (Natural) Language Video starts at 6:00
10.30 - 12.30 Poster Session 1 (Conference Papers, Workshop Papers)

Afternoon Session – Session Chair: Navdeep Jaitly
14.30 - 15.10 Invited talk 2: Alex Graves, New Direction for Recurent Neural Networks, Video starts at 4:10

15.10 - 15.30 Contributed Talk 3: Making Neural Programming Architectures Generalize via Recursion - BEST PAPER AWARD , Video starts at 50:12

15.30 - 15.50 Contributed Talk 4: Neural Architecture Search with Reinforcement Learning Video starts at 1:08:44

15.50 - 16.10 Contributed Talk 5: Optimization as a Model for Few-Shot Learning Video starts at 1:26:30

16.30 - 18.30 Poster Session 2 (Conference Papers, Workshop Papers)

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Thesis: Randomized Algorithms for Large-Scale Data Analysis by Farhad Pourkamali-Anaraki

Image 1

Stephen just sent me the following:

Hi Igor, 

It's a pleasure to write to you again and announce the graduation of my PhD student Farhad Pourkamali-Anaraki.

His thesis is online "Randomized Algorithms for Large-Scale Data Analysis" at https://www.researchgate.net/publication/316070128_Randomized_Algorithms_for_Large-Scale_Data_Analysis

It contains a lot of good things, some published some not. In particular (see attached image 1) he has great work on a 1-pass algorithm for K-means that seems to be one of the only 1-pass algorithms to accurately estimate cluster centers (implementation at https://github.com/stephenbeckr/SparsifiedKMeans ), and also has very recent work on efficient variations of the Nystrom method for approximating kernel matrices that seems to give the high-accuracy of the clustered Nystrom method at a fraction of the computational cost (see image 2). 

Best,
Stephen

Image 2

Thanks Stephen but I think the following paper also does 1-pass for K-Means (Keriven N., Tremblay N., Traonmilin Y., Gribonval R., "Compressive K-means" and its implementation SketchMLbox: A MATLAB toolbox for large-scale mixture learning ) even though the contruction seems different. Both of these implementations will be added to the Advanced Matrix Factorization Jungle page.

Anyway, congratulations Dr. Pourkamali-Anaraki !
Randomized Algorithms for Large-Scale Data Analysis, Farhad Pourkamali-Anaraki The abstract reads :

Massive high-dimensional data sets are ubiquitous in all scientific disciplines. Extract- ing meaningful information from these data sets will bring future advances in fields of science and engineering. However, the complexity and high-dimensionality of modern data sets pose unique computational and statistical challenges. The computational requirements of analyzing large-scale data exceed the capacity of traditional data analytic tools. The challenges surrounding large high-dimensional data are felt not just in processing power, but also in memory access, storage requirements, and communication costs. For example, modern data sets are often too large to fit into the main memory of a single workstation and thus data points are processed sequentially without a chance to store the full data. Therefore, there is an urgent need for the development of scalable learning tools and efficient optimization algorithms in today’s high-dimensional data regimes.

A powerful approach to tackle these challenges is centered around preprocessing high-dimensional data sets via a dimensionality reduction technique that preserves the underlying geometry and structure of the data. This approach stems from the observation that high- dimensional data sets often have intrinsic dimension which is significantly smaller than the ambient dimension. Therefore, information-preserving dimensionality reduction methods are valuable tools for reducing the memory and computational requirements of data analytic tasks on large-scale data sets.

Recently, randomized dimension reduction has received a lot of attention in several fields, including signal processing, machine learning, and numerical linear algebra. These methods use random sampling or random projection to construct low-dimensional representations of the data, known as sketches or compressive measurements. These randomized methods are effective in modern data settings since they provide a non-adaptive data- independent mapping of high-dimensional data into a low-dimensional space. However, such methods require strong theoretical guarantees to ensure that the key properties of original data are preserved under a randomized mapping.

This dissertation focuses on the design and analysis of efficient data analytic tasks using randomized dimensionality reduction techniques. Specifically, four efficient signal processing and machine learning algorithms for large high-dimensional data sets are proposed: covariance estimation and principal component analysis, dictionary learning, clustering, and low-rank approximation of positive semidefinite kernel matrices. These techniques are valu- able tools to extract important information and patterns from massive data sets. Moreover, an efficient data sparsification framework is introduced that does not require incoherence and distributional assumptions on the data. A main feature of the proposed compression scheme is that it requires only one pass over the data due to the randomized preconditioning transformation, which makes it applicable to streaming and distributed data settings.

The main contribution of this dissertation is threefold: (1) strong theoretical guarantees are provided to ensure that the proposed randomized methods preserve the key properties and structure of high-dimensional data; (2) tradeoffs between accuracy and memory/computation savings are characterized for a large class of data sets as well as dimensionality reduction methods, including random linear maps and random sampling; (3) extensive numerical experiments are presented to demonstrate the performance and benefits of our proposed methods compared to prior works.

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

ICLR2017, third and last day.

This is the last day of ICLR 2017. The meeting is be featured live on Facebook here at: https://www.facebook.com/iclr.cc/ . If you want to say hi, I am around.and we're hiring.


Morning Session – Session Chair: Slav Petrov
7.30 – 9.00 Registration
9.00 - 9.40 Invited talk 1: Regina Barzilay
9.40 - 10.00 Contributed talk 1: Learning End-to-End Goal-Oriented Dialog
10.00 - 10.20 Contributed talk 2: Multi-Agent Cooperation and the Emergence of (Natural) Language
10.20 - 10.30 Coffee Break
10.30 - 12.30 Poster Session 1 (Conference Papers, Workshop Papers)
12.30 - 14.30 Lunch provided by ICLR

Afternoon Session – Session Chair: Navdeep Jaitly
14.30 - 15.10 Invited talk 2: Alex Graves
15.10 - 15.30 Contributed Talk 3: Making Neural Programming Architectures Generalize via Recursion - BEST PAPER AWARD
15.30 - 15.50 Contributed Talk 4: Neural Architecture Search with Reinforcement Learning
15.50 - 16.10 Contributed Talk 5: Optimization as a Model for Few-Shot Learning
16.10 - 16.30 Coffee Break
16.30 - 18.30 Poster Session 2 (Conference Papers, Workshop Papers)

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

#ICLR2017 Tuesday Afternoon Program

 

ICLR 2017 continues this afternoon in Toulon, there will be a blog post for each half day that features directly links to papers from the Open review section. The meeting will be featured live on Facebook here at: https://www.facebook.com/iclr.cc/ . If you want to say hi, I am around.and we're hiring.

 

14.00 - 16.00 Poster Session 2 (Conference Papers, Workshop Papers)
16.00 - 16.15 Coffee Break
16.15 - 17.00 Invited talk 2: Riccardo Zecchina
17.00 - 17.20 Contributed Talk 3: Learning to Act by Predicting the Future
17.20 - 17.40 Contributed Talk 4: Reinforcement Learning with Unsupervised Auxiliary Tasks
17.40 - 18.00 Contributed Talk 5: Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic
18.00 - 18.10 Group photo at the Stade Félix Mayol
19.00 - 24.00 Gala dinner offered by ICLR

C1: Sigma Delta Quantized Networks 
( code)
C2: Paleo: A Performance Model for Deep Neural Networks
C3: DeepCoder: Learning to Write Programs
C4: Topology and Geometry of Deep Rectified Network Optimization Landscapes
C5: Incremental Network Quantization: Towards Lossless CNNs with Low-precision Weights
C6: Learning to Perform Physics Experiments via Deep Reinforcement Learning
C7: Decomposing Motion and Content for Natural Video Sequence Prediction
C8: Calibrating Energy-based Generative Adversarial Networks
C9: Pruning Convolutional Neural Networks for Resource Efficient Inference
C10: Incorporating long-range consistency in CNN-based texture generation
( code )
C11: Lossy Image Compression with Compressive Autoencoders
C12: LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation
C13: Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
C14: Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data
C15: Mollifying Networks
C16: beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework
C17: Categorical Reparameterization with Gumbel-Softmax
C18: Online Bayesian Transfer Learning for Sequential Data Modeling
C19: Latent Sequence Decompositions
C20: Density estimation using Real NVP
C21: Recurrent Batch Normalization
C22: SGDR: Stochastic Gradient Descent with Restarts
C23: Variable Computation in Recurrent Neural Networks
C24: Deep Variational Information Bottleneck
C25: SampleRNN: An Unconditional End-to-End Neural Audio Generation Model
C26: TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency
C27: Frustratingly Short Attention Spans in Neural Language Modeling
C28: Offline Bilingual Word Vectors, Orthogonal Transformations and the Inverted Softmax
C29: LEARNING A NATURAL LANGUAGE INTERFACE WITH NEURAL PROGRAMMER
C30: Designing Neural Network Architectures using Reinforcement Learning
C31: Metacontrol for Adaptive Imagination-Based Optimization (spaceship dataset )
C32: Recurrent Environment Simulators
C33: EPOpt: Learning Robust Neural Network Policies Using Model Ensembles

W1: Lifelong Perceptual Programming By Example
W2: Neu0
W3: Dance Dance Convolution
W4: Bit-Pragmatic Deep Neural Network Computing
W5: On Improving the Numerical Stability of Winograd Convolutions
W6: Fast Generation for Convolutional Autoregressive Models
W7: THE PREIMAGE OF RECTIFIER NETWORK ACTIVITIES
W8: Training Triplet Networks with GAN
W9: On Robust Concepts and Small Neural Nets
W10: Pl@ntNet app in the era of deep learning
W11: Exponential Machines
W12: Online Multi-Task Learning Using Biased Sampling
W13: Online Structure Learning for Sum-Product Networks with Gaussian Leaves
W14: A Theoretical Framework for Robustness of (Deep) Classifiers against Adversarial Samples
W15: Compositional Kernel Machines
W16: Loss is its own Reward: Self-Supervision for Reinforcement Learning
W17: REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
W18: Precise Recovery of Latent Vectors from Generative Adversarial Networks
W19: Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization (code)
 

 
 
 

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

#ICLR2017 Tuesday Morning Program

 

 

 

So ICLR 2017 continues today in Toulon, there will be a blog post for each half day that features directly links to papers from the Open review section. The meeting will be featured live on Facebook here at: https://www.facebook.com/iclr.cc/ . If you want to say hi, I am around.and we're hiring.

7.30 – 9.00 Registration
9.00 - 9.40 Invited talk 1: Chloé-Agathe Azencott
9.40 - 10.00 Contributed talk 1: Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data - BEST PAPER AWARD
10.00 - 10.20 Contributed talk 2: Learning Graphical State Transitions
10.20 - 10.30 Coffee Break
10.30 - 12.30 Poster Session 1 (Conference Papers, Workshop Papers)

 Conference posters (1st floor)

 

C1: DeepDSL: A Compilation-based Domain-Specific Language for Deep Learning (code)
C2: A SELF-ATTENTIVE SENTENCE EMBEDDING
C3: Deep Probabilistic Programming
C4: Lie-Access Neural Turing Machines
C5: Learning Features of Music From Scratch
C6: Mode Regularized Generative Adversarial Networks
C7: End-to-end Optimized Image Compression (web)
C8: Variational Recurrent Adversarial Deep Domain Adaptation
C9: Steerable CNNs
C10: Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning (code)
C11: PixelVAE: A Latent Variable Model for Natural Images
C12: A recurrent neural network without chaos
C13: Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
C14: Tree-structured decoding with doubly-recurrent neural networks
C15: Introspection:Accelerating Neural Network Training By Learning Weight Evolution
C16: Hyperband: Bandit-Based Configuration Evaluation for Hyperparameter Optimization (page)
C17: Quasi-Recurrent Neural Networks (Keras)
C18: Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources in the same domain
C19: A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
C20: Trusting SVM for Piecewise Linear CNNs
C21: Maximum Entropy Flow Networks
C22: The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
C23: Unrolled Generative Adversarial Networks
C24: A Simple but Tough-to-Beat Baseline for Sentence Embeddings (blog entry)
C25: Query-Reduction Networks for Question Answering (code)
C26: Machine Comprehension Using Match-LSTM and Answer Pointer (code)
C27: Words or Characters? Fine-grained Gating for Reading Comprehension
C28: Dynamic Coattention Networks For Question Answering (code)
C29: Multi-view Recurrent Neural Acoustic Word Embeddings
C30: Episodic Exploration for Deep Deterministic Policies for StarCraft Micromanagement
C31: Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning
C32: Generalizing Skills with Semi-Supervised Reinforcement Learning
C33: Improving Policy Gradient by Exploring Under-appreciated Rewards

 

3rd Floor

 

W1: Programming With a Differentiable Forth Interpreter
W2: Unsupervised Feature Learning for Audio Analysis
W3: Neural Functional Programming
W4: A Smooth Optimisation Perspective on Training Feedforward Neural Networks
W5: Synthetic Gradient Methods with Virtual Forward-Backward Networks
W6: Explaining the Learning Dynamics of Direct Feedback Alignment
W7: Training a Subsampling Mechanism in Expectation
W8: Deep Kernel Machines via the Kernel Reparametrization Trick
W9: Encoding and Decoding Representations with Sum- and Max-Product Networks
W10: Embracing Data Abundance
W11: Variational Intrinsic Control
W12: Fast Adaptation in Generative Models with Generative Matching Networks
W13: Efficient variational Bayesian neural network ensembles for outlier detection
W14: Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols
W15: Adaptive Feature Abstraction for Translating Video to Language
W16: Delving into adversarial attacks on deep policies
W17: Tuning Recurrent Neural Networks with Reinforcement Learning
W18: DeepMask: Masking DNN Models for robustness against adversarial samples
W19: Restricted Boltzmann Machines provide an accurate metric for retinal responses to visual stimuli
 

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.