So ICLR 2017 continues today in Toulon, there will be a blog post for each half day that features directly links to papers from the Open review section. The meeting will be featured live on Facebook here at: https://www.facebook.com/iclr.cc/ . If you want to say hi, I am around.and we're hiring.

7.30 – 9.00 Registration

9.00 - 9.40 Invited talk 1: ChloĆ©-Agathe Azencott

9.40 - 10.00 Contributed talk 1: Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data - BEST PAPER AWARD

10.00 - 10.20 Contributed talk 2: Learning Graphical State Transitions

10.20 - 10.30

*Coffee Break*

10.30 - 12.30 Poster Session 1 (Conference Papers, Workshop Papers)

Conference posters (1st floor)

C1: DeepDSL: A Compilation-based Domain-Specific Language for Deep Learning (code)

C2: A SELF-ATTENTIVE SENTENCE EMBEDDING

C3: Deep Probabilistic Programming

C4: Lie-Access Neural Turing Machines

C5: Learning Features of Music From Scratch

C6: Mode Regularized Generative Adversarial Networks

C7: End-to-end Optimized Image Compression (web)

C8: Variational Recurrent Adversarial Deep Domain Adaptation

C9: Steerable CNNs

C10: Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning (code)

C11: PixelVAE: A Latent Variable Model for Natural Images

C12: A recurrent neural network without chaos

C13: Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

C14: Tree-structured decoding with doubly-recurrent neural networks

C15: Introspection:Accelerating Neural Network Training By Learning Weight Evolution

C16: Hyperband: Bandit-Based Configuration Evaluation for Hyperparameter Optimization (page)

C17: Quasi-Recurrent Neural Networks (Keras)

C18: Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources in the same domain

C19: A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks

C20: Trusting SVM for Piecewise Linear CNNs

C21: Maximum Entropy Flow Networks

C22: The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables

C23: Unrolled Generative Adversarial Networks

C24: A Simple but Tough-to-Beat Baseline for Sentence Embeddings (blog entry)

C25: Query-Reduction Networks for Question Answering (code)

C26: Machine Comprehension Using Match-LSTM and Answer Pointer (code)

C27: Words or Characters? Fine-grained Gating for Reading Comprehension

C28: Dynamic Coattention Networks For Question Answering (code)

C29: Multi-view Recurrent Neural Acoustic Word Embeddings

C30: Episodic Exploration for Deep Deterministic Policies for StarCraft Micromanagement

C31: Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning

C32: Generalizing Skills with Semi-Supervised Reinforcement Learning

C33: Improving Policy Gradient by Exploring Under-appreciated Rewards

C2: A SELF-ATTENTIVE SENTENCE EMBEDDING

C3: Deep Probabilistic Programming

C4: Lie-Access Neural Turing Machines

C5: Learning Features of Music From Scratch

C6: Mode Regularized Generative Adversarial Networks

C7: End-to-end Optimized Image Compression (web)

C8: Variational Recurrent Adversarial Deep Domain Adaptation

C9: Steerable CNNs

C10: Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning (code)

C11: PixelVAE: A Latent Variable Model for Natural Images

C12: A recurrent neural network without chaos

C13: Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

C14: Tree-structured decoding with doubly-recurrent neural networks

C15: Introspection:Accelerating Neural Network Training By Learning Weight Evolution

C16: Hyperband: Bandit-Based Configuration Evaluation for Hyperparameter Optimization (page)

C17: Quasi-Recurrent Neural Networks (Keras)

C18: Attend, Adapt and Transfer: Attentive Deep Architecture for Adaptive Transfer from multiple sources in the same domain

C19: A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks

C20: Trusting SVM for Piecewise Linear CNNs

C21: Maximum Entropy Flow Networks

C22: The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables

C23: Unrolled Generative Adversarial Networks

C24: A Simple but Tough-to-Beat Baseline for Sentence Embeddings (blog entry)

C25: Query-Reduction Networks for Question Answering (code)

C26: Machine Comprehension Using Match-LSTM and Answer Pointer (code)

C27: Words or Characters? Fine-grained Gating for Reading Comprehension

C28: Dynamic Coattention Networks For Question Answering (code)

C29: Multi-view Recurrent Neural Acoustic Word Embeddings

C30: Episodic Exploration for Deep Deterministic Policies for StarCraft Micromanagement

C31: Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning

C32: Generalizing Skills with Semi-Supervised Reinforcement Learning

C33: Improving Policy Gradient by Exploring Under-appreciated Rewards

3rd Floor

W1: Programming With a Differentiable Forth Interpreter

W2: Unsupervised Feature Learning for Audio Analysis

W3: Neural Functional Programming

W4: A Smooth Optimisation Perspective on Training Feedforward Neural Networks

W5: Synthetic Gradient Methods with Virtual Forward-Backward Networks

W6: Explaining the Learning Dynamics of Direct Feedback Alignment

W7: Training a Subsampling Mechanism in Expectation

W8: Deep Kernel Machines via the Kernel Reparametrization Trick

W9: Encoding and Decoding Representations with Sum- and Max-Product Networks

W10: Embracing Data Abundance

W11: Variational Intrinsic Control

W12: Fast Adaptation in Generative Models with Generative Matching Networks

W13: Efficient variational Bayesian neural network ensembles for outlier detection

W14: Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols

W15: Adaptive Feature Abstraction for Translating Video to Language

W16: Delving into adversarial attacks on deep policies

W17: Tuning Recurrent Neural Networks with Reinforcement Learning

W18: DeepMask: Masking DNN Models for robustness against adversarial samples

W19: Restricted Boltzmann Machines provide an accurate metric for retinal responses to visual stimuli

W2: Unsupervised Feature Learning for Audio Analysis

W3: Neural Functional Programming

W4: A Smooth Optimisation Perspective on Training Feedforward Neural Networks

W5: Synthetic Gradient Methods with Virtual Forward-Backward Networks

W6: Explaining the Learning Dynamics of Direct Feedback Alignment

W7: Training a Subsampling Mechanism in Expectation

W8: Deep Kernel Machines via the Kernel Reparametrization Trick

W9: Encoding and Decoding Representations with Sum- and Max-Product Networks

W10: Embracing Data Abundance

W11: Variational Intrinsic Control

W12: Fast Adaptation in Generative Models with Generative Matching Networks

W13: Efficient variational Bayesian neural network ensembles for outlier detection

W14: Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols

W15: Adaptive Feature Abstraction for Translating Video to Language

W16: Delving into adversarial attacks on deep policies

W17: Tuning Recurrent Neural Networks with Reinforcement Learning

W18: DeepMask: Masking DNN Models for robustness against adversarial samples

W19: Restricted Boltzmann Machines provide an accurate metric for retinal responses to visual stimuli

**Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !**

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

## 2 comments:

Re: https://openreview.net/pdf?id=HkXKUTVFl

I'm trying dropout in relation to the back error projection. Anyway there are tons of ideas to explore, especially if you start using fast random projection algorithms for both the back error projection and the forward aspects of a network.

I think what is happening is that you are getting unsupervised feature learning in the deeper layers and then one final readout layer. That may give a boost in performance in some circumstances. There probably are better ways to do unsupervised feature learning prior to a readout layer. There are also some aspects to do with noise and maybe some cooling effect over time as the system adapts. Thumbs up or thumbs down, I don't know. You be the judge.

Post a Comment