Nuit Blanche in Review (January 2017)

A few things happened since the last Nuit Blanche in Review. We had a few implementations, a thesis, few jobs announcements and the NIPS videos....Enjoy !

Also, Laurent is current at Photonics West this week !

Implementation

• Edward: Deep Probabilistic Programming - implementation -
• PhaseMax: Convex Phase Retrieval via Basis Pursuit - implementation -

Thesis

• Thesis: Privacy-aware and Scalable Recommender Systems using Sketching Techniques by Raghavendran Balu

Sunday Morning Insight

• Sunday Morning Insight: D'avions et d'Intelligence Artificielle en France [in French]

Other in-depth insights

• Wasserstein GAN / Towards Principled Methods for Training Generative Adversarial Networks
• Learning the structure of learning
• Continuing To Learn the Structure of Learning
• Controlling Light Transmission Through Highly Scattering Media Using Semi-Definite Programming as a Phase Retrieval Computation Method
• Understanding deep learning requires rethinking generalization
• In Texas, we just call them Neural Networks.
• An NlogN Parallel Fast Direct Solver for Kernel Matrices
• NIPS 2016 Tutorial: Generative Adversarial Networks / Learning in Implicit Generative Models
• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference / Scaling Binarized Neural Networks on Reconfigurable Logic
• A Matrix Factorization Approach for Learning Semidefinite-Representable Regularizers
• Randomized Clustered Nystrom for Large-Scale Kernel Machines

Book

• Book: A Course in Machine Learning, Hal Daumé III

Job:

• Job: Postdoc, Research Associate in Data Science, UCL, London
• Jobs: Data Science Postdoctoral Fellows, Harvard University
• Job: Summer 2017 IBM Social Good Fellowship (Undergraduates, Graduates, NGOs)
• CSjobs: two postdoctoral researchers, "C-SENSE: Exploiting low dimensional signal models for sensing, computation and processing" and "CS for Radar and Electronic Surveillance", Edinburgh, Scotland
• CSJob: Postdoctoral Position Computational Optics and Optical Computing, LKB, ENS, Paris, France

Paris Machine Learning meetup

• Ce soir: Paris Machine Learning Hors serie # 5 Season 4, Workshop Amazon Web Service
• Paris Machine Learning Meetup #5 Season 4: LIME 'Why should I trust you', Apache SAMOA, GANs for Cracks, Opps, NIPS2016

Videos:

• The #NIPS2016 Videos are out
• Saturday Morning Videos: Foundations of Machine Learning Boot Camp, Simons Institute, Berkeley
• Saturday Morning Videos: Fairness, Accountability, and Transparency in Machine Learning @NYU School of Law
• Saturday Morning Videos: Deep Learning Symposium, #NIPS2016
• Saturday Morning Video: Stan Conference 2017 video streaming
• Saturday Morning Video: Head, Torso and Tail - Performance for modeling real data, Alex Smola
• Videos and Slides: 6th MMDS Workshop on Algorithms for Modern Massive Data Sets

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Wasserstein GAN / Towards Principled Methods for Training Generative Adversarial Networks

We mentioned GANs before. Here are contributions on how to use Earth Mover's distances to improve their training (Cedric Villani is mentioned in the references of the second paper and points to a newer version of the Optimal transport, old and new manuscript)

Wasserstein GAN by Martin Arjovsky, Soumith Chintala, Léon Bottou

We introduce a new algorithm named WGAN, an alternative to traditional GAN training. In this new model, we show that we can improve the stability of learning, get rid of problems like mode collapse, and provide meaningful learning curves useful for debugging and hyperparameter searches. Furthermore, we show that the corresponding optimization problem is sound, and provide extensive theoretical work highlighting the deep connections to other distances between distributions.

Towards Principled Methods for Training Generative Adversarial Networks by Martin Arjovsky, Léon Bottou

The goal of this paper is not to introduce a single algorithm or method, but to make theoretical steps towards fully understanding the training dynamics of generative adversarial networks. In order to substantiate our theoretical analysis, we perform targeted experiments to verify our assumptions, illustrate our claims, and quantify the phenomena. This paper is divided into three sections. The first section introduces the problem at hand. The second section is dedicated to studying and proving rigorously the problems including instability and saturation that arize when training generative adversarial networks. The third section examines a practical and theoretically grounded direction towards solving these problems, while introducing new tools to study them.

 

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Saturday Morning Videos: Fairness, Accountability, and Transparency in Machine Learning @NYU School of Law

There was a videostream while it happened, here are the four videos covering the 3rd Workshop on Fairness, Accountability, and Transparency in Machine Learning is being held in New York,

 The program is here.

  Fairness, Accountability, and Transparency in Machine Learning: Part One
  Fairness, Accountability, and Transparency in Machine Learning: Part Two
  Fairness, Accountability, and Transparency in Machine Learning: Part Three

Fairness, Accountability, and Transparency: Part Four

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Saturday Morning Videos: Foundations of Machine Learning Boot Camp, Simons Institute, Berkeley

This past week saw a series of tutorial presentations in a "bootcamp" on the Foundations of Machine Learning at the Simons Institute at Berkeley organized by Sanjoy Dasgupta, Sanjeev Arora , Nina Balcan, Peter Bartlett, Sham Kakade, Santosh Vempala. Links to the videos can be found directly from the page:

The Boot Camp is intended to acquaint program participants with the key themes of the program. It will consist of five days of tutorial presentations, each with ample time for questions and discussion, as follows:
Monday, January 23rd

• Elad Hazan (Princeton University): Optimization of Machine Learning
• Lecture 1: Optimization for Machine Learning I
• Lecture 2: Optimization for Machine Learning II 

• Andreas Krause (ETH Zürich) and Stefanie Jegelka (MIT): Submodularity: Theory and Applications
• Lecture 1: Submodularity: Theory and Applications I
• Lecture 2: Submodularity: Theory and Applications II

  

Tuesday, January 24th

• Emma Brunskill (Carnegie Mellon University): A Tutorial on Reinforcement Learning
• Lecture 1: A Tutorial on Reinforcement Learning I
• Lecture 2: A Tutorial on Reinforcement Learning II 

• Sanjoy Dasgupta (UC San Diego) and Rob Nowak (University of Wisconsin-Madison): Interactive Learning of Classifiers and Other Structures
• Lecture 1: Interactive Learning of Classifiers and Other Structures 

• Sergey Levine (UC Berkeley): Deep Robotic Learning
• Lecture 1: Deep Robotic Learning 

Wednesday, January 25th

• Tamara Broderick (MIT) and Michael Jordan (UC Berkeley): Nonparametric Bayesian Methods: Models, Algorithms, and Applications
• Lecture 1: Nonparametric Bayesian Methods: Models, Algorithms, and Applications I
• Lecture 2: Nonparametric Bayesian Methods: Models, Algorithms, and Applications II
• Lecture 3: Nonparametric Bayesian Methods: Models, Algorithms, and Applications III
• Lecture 4: Nonparametric Bayesian Methods: Models, Algorithms, and Applications IV 

Thursday, January 26th

• Ruslan Salakhutdinov (Carnegie Mellon University): Tutorial on Deep Learning
• Lecture 1: Tutorial on Deep Learning I
• Lecture 2: Tutorial on Deep Learning II 
• Lecture 3: Tutorial on Deep Learning III
• Lecture 4: Tutorial on Deep Learning IV  

Friday, January 27th

• Daniel Hsu (Columbia University): Tensor Decompositions for Learning Latent Variable Models
• Lecture 1: Tensor Decompositions for Learning Latent Variable Models I
• Lecture 2: Tensor Decompositions for Learning Latent Variable Models II  

• Percy Liang (Stanford University): Natural Language Understanding: Foundations and State-of-the-Art
• Lecture 1: Natural Language Understanding: Foundations and State-of-the-Art

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Saturday Morning Videos: Deep Learning Symposium, #NIPS2016

Here are the three sessions of the Deep Learning Symposium  at NIPS2016, enjoy !

Deep Learning Symposium Session 1

01:54:39

Deep Learning Symposium Session 201:05:19

Neural Information Processing Systems Conference - NIPS 2016

Deep Learning Symposium Session 301:21:49

Neural Information Processing Systems Conference - NIPS 2016

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Job: Postdoc, Research Associate in Data Science, UCL, London

Jason just sent me the following:

Hi Igor,

I just wanted to let you know about a related vacancy for a postdoctoral researcher at UCL.
Would you consider posting on Nuit Blanche if you think it suitable?
Many thanks in advance for your help ....
Best wishes,
Jason 

Here is the announcement:

Research Associate in Data Science


We are seeking an excellent postdoctoral researcher in signal processing, applied mathematics, physics, statistics, computer science, or a related field, to develop novel signal analysis techniques for extracting scientific information from large observational data-sets. Research aims will include a focus on signal analysis on the sphere but can be extended to incorporate the interests and expertise of the successful applicant.

Signals defined on the sphere are prevalent in a diverse range of fields, including cosmology, geophysics, acoustics, and computer graphics, for example. In cosmology, observations made by the European Space Agency's Planck and Euclid satellites live on the celestial sphere, leading to very large and precise spherical data-sets, the robust analysis of which can reveal a great deal about the nature of our Universe. The successful applicant will focus on the theoretical and methodological foundations of the analysis of signals defined on the sphere and will work closely with another postdoctoral researcher who will fill a complementary post focused on application to observational data.

The successful applicant will be based in the Astrophysics Group at the Mullard Space Science Laboratory (MSSL) of University College London (UCL) and will join the multi-disciplinary astroinformatics team, which comprises researchers with expertise in astrophysics, applied mathematics, signal processing, statistics and software engineering. The goal of the astroinformatics team is to develop and apply novel analysis techniques to extract scientific information from very large data sets, in order to develop a deeper understanding of the fundamental physics underlying the evolution of our Universe. The successful applicant will work closely with Dr Jason McEwen, while also interacting with other members of the astroinformatics team and the Astrophysics Group.

Further information is available at http://www.jasonmcewen.org/opportunities.html.
--
www.jasonmcewen.org

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Controlling Light Transmission Through Highly Scattering Media Using Semi-Definite Programming as a Phase Retrieval Computation Method

Controlling Light Transmission Through Highly Scattering Media Using Semi-Definite Programming as a Phase Retrieval Computation Method by Moussa N'Gom, Miao-Bin Lien, Theodore B. Norris, Eric Michielssen, Raj Rao Nadakuditi

Complex Semi-Definite Programming (SDP) is introduced as a novel approach to phase retrieval enabled control of monochromatic light transmission through highly scattering media. In a simple optical setup, a spatial light modulator is used to generate a random sequence of phase-modulated wavefronts, and the resulting intensity speckle patterns in the transmitted light are acquired on a camera. The SDP algorithm allows computation of the complex transmission matrix of the system from this sequence of intensity-only measurements, without need for a reference beam. Once the transmission matrix is determined, optimal wavefronts are computed that focus the incident beam to any position or sequence of positions on the far side of the scattering medium, without the need for any subsequent measurements or wavefront shaping iterations. The number of measurements required and the degree of enhancement of the intensity at focus is determined by the number of pixels controlled by the spatial light modulator.

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

The #NIPS2016 Videos are out

The videos for the Neural Information Processing Systems Conference - NIPS 2016 are out (first three days), enjoy !

Deep Reinforcement Learning Through Policy Optimization19:19

Neural Information Processing Systems Conference - NIPS 2016

Theory and Algorithms for Forecasting Non-Stationary Time Series01:45:04

Neural Information Processing Systems Conference - NIPS 2016

Crowdsourcing: Beyond Label Generation01:50:18

Neural Information Processing Systems Conference - NIPS 2016

ML Foundations and Methods for Precision Medicine and Healthcare02:08:17

Neural Information Processing Systems Conference - NIPS 2016

Variational Inference: Foundations and Modern Methods01:53:04

Neural Information Processing Systems Conference - NIPS 2016

Generative Adversarial Networks01:55:53

Neural Information Processing Systems Conference - NIPS 2016

Predictive Learning

56:52

Neural Information Processing Systems Conference - NIPS 2016

Intelligent Biosphere

49:42

Neural Information Processing Systems Conference - NIPS 2016

Value Iteration Networks19:40

Neural Information Processing Systems Conference - NIPS 2016

Tractable Operations for Arithmetic Circuits of Probabilistic Models19:15

Neural Information Processing Systems Conference - NIPS 2016

Testing for Differences in Gaussian Graphical Models: Applications to Brain Connectivity16:51

Neural Information Processing Systems Conference - NIPS 2016

NIPS 2016

SDP Relaxation with Randomized Rounding for Energy Disaggregation

Neural Information Processing Systems Conference - NIPS 2016

SDP Relaxation with Randomized Rounding for Energy Disaggregation

List of Episodes

Neural Information Processing Systems Conference - NIPS 2016

Bayesian Intermittent Demand Forecasting for Large Inventories18:01

Neural Information Processing Systems Conference - NIPS 2016

Synthesis of MCMC and Belief Propagation17:26

Neural Information Processing Systems Conference - NIPS 2016

Deep Learning for Predicting Human Strategic Behavior

Neural Information Processing Systems Conference - NIPS 2016

Using Fast Weights to Attend to the Recent Past21:02

Neural Information Processing Systems Conference - NIPS 2016

Sequential Neural Models with Stochastic Layers20:18

Neural Information Processing Systems Conference - NIPS 2016

Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences20:55

Neural Information Processing Systems Conference - NIPS 2016

Graphons, mergeons, and so on!17:10

Neural Information Processing Systems Conference - NIPS 2016

Hierarchical Clustering via Spreading Metrics17:40

Neural Information Processing Systems Conference - NIPS 2016

Clustering with Same-Cluster Queries18:05

Neural Information Processing Systems Conference - NIPS 2016

Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA22:50

Neural Information Processing Systems Conference - NIPS 2016

Fast and Provably Good Seedings for k-Means19:44

Neural Information Processing Systems Conference - NIPS 2016

Supervised learning through the lens of compression16:52

Neural Information Processing Systems Conference - NIPS 2016

Supervised learning through the lens of compression

Neural Information Processing Systems Conference - NIPS 2016

MetaGrad: Multiple Learning Rates in Online Learning16:36

Neural Information Processing Systems Conference - NIPS 2016

Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning15:52

Neural Information Processing Systems Conference - NIPS 2016

Global Analysis of Expectation Maximization for Mixtures of Two Gaussians18:58

Neural Information Processing Systems Conference - NIPS 2016

Machine Learning and Likelihood-Free Inference in Particle Physics50:37

Neural Information Processing Systems Conference - NIPS 2016

Matrix Completion has No Spurious Local Minimum18:26

Neural Information Processing Systems Conference - NIPS 2016

Large-Scale Price Optimization via Network Flow18:59

Neural Information Processing Systems Conference - NIPS 2016

Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks18:17

Neural Information Processing Systems Conference - NIPS 2016

Supervised Word Mover's Distance21:30

Neural Information Processing Systems Conference - NIPS 2016

Beyond Exchangeability: The Chinese Voting Process19:34

Neural Information Processing Systems Conference - NIPS 2016

Protein contact prediction from amino acid co-evolution using convolutional networks for…20:12

Neural Information Processing Systems Conference - NIPS 2016

Deep Learning without Poor Local Minima19:19

Neural Information Processing Systems Conference - NIPS 2016

Learning to Poke by Poking: Experiential Learning of Intuitive Physics

Neural Information Processing Systems Conference - NIPS 2016

Learning to Poke by Poking: Experiential Learning of Intuitive Physics

Neural Information Processing Systems Conference - NIPS 2016

Learning What and Where to Draw21:37

Neural Information Processing Systems Conference - NIPS 2016

Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural…22:55

Neural Information Processing Systems Conference - NIPS 2016

Achieving the KS threshold in the general stochastic block model with linearized acyclic…15:04

Neural Information Processing Systems Conference - NIPS 2016

Orthogonal Random Features18:42

Neural Information Processing Systems Conference - NIPS 2016

Poisson-Gamma dynamical systems16:40

Neural Information Processing Systems Conference - NIPS 2016

The Multiscale Laplacian Graph Kernel20:34

Neural Information Processing Systems Conference - NIPS 2016

Stochastic Online AUC Maximization14:40

Neural Information Processing Systems Conference - NIPS 2016

Without-Replacement Sampling for Stochastic Gradient Methods19:55

Neural Information Processing Systems Conference - NIPS 2016

Regularized Nonlinear Acceleration18:52

Neural Information Processing Systems Conference - NIPS 2016

Generalization of ERM in Stochastic Convex Optimization: The Dimension Strikes Back16:21

Neural Information Processing Systems Conference - NIPS 2016

Bayesian Optimization with Robust Bayesian Neural Networks14:50

Neural Information Processing Systems Conference - NIPS 2016

Learning About the Brain: Neuroimaging and Beyond51:37

Neural Information Processing Systems Conference - NIPS 2016

Learning About the Brain: Neuroimaging and Beyond

Neural Information Processing Systems Conference - NIPS 2016

Reproducible Research: the Case of the Human Microbiome55:53

Neural Information Processing Systems Conference - NIPS 2016

Interpretable Distribution Features with Maximum Testing Power22:45

Neural Information Processing Systems Conference - NIPS 2016

Examples are not enough, learn to criticize! Criticism for Interpretability20:39

Neural Information Processing Systems Conference - NIPS 2016

Showing versus doing: Teaching by demonstration19:03

Neural Information Processing Systems Conference - NIPS 2016

Relevant sparse codes with variational information bottleneck17:53

Neural Information Processing Systems Conference - NIPS 2016

Dense Associative Memory for Pattern Recognition24:11

Neural Information Processing Systems Conference - NIPS 2016

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.