Asymmetric Random Projections

** Nuit Blanche is now on Twitter: @NuitBlog **

It is interesting that much of the literature about random projections is to make them less data oblivious. That goal is in large part driven by our stinginess on the complexity of computing these Random Projections. What if the number of Random Projections were not capped? What if, instead of metering the number of random projections to get the most out of it, one were given plenty of them in one fell swoop? This is what we are trying to do at LightOn: use Light to provide plenty of random projections. How much is plenty? Let's put it this way, having a Random Projection of size 1 or 1 million using our OPU requires the same effort. We are running a Cloud service where you can try our technology, it's here: https://www.lighton.ai/lighton-cloud/ 

In the meantime, let us take a look at these interesting asymmetric RPs:

Asymmetric Random Projections by Nick Ryder, Zohar Karnin, Edo Liberty

Random projections (RP) are a popular tool for reducing dimensionality while preserving local geometry. In many applications the data set to be projected is given to us in advance, yet the current RP techniques do not make use of information about the data. In this paper, we provide a computationally light way to extract statistics from the data that allows designing a data dependent RP with superior performance compared to data-oblivious RP. We tackle scenarios such as matrix multiplication and linear regression/classification in which we wish to estimate inner products between pairs of vectors from two possibly different sources. Our technique takes advantage of the difference between the sources and is provably superior to oblivious RPs. Additionally, we provide extensive experiments comparing RPs with our approach showing significant performance lifts in fast matrix multiplication, regression and classification problems.

Follow @NuitBlog or join the CompressiveSensing Reddit, the Facebook page, the Compressive Sensing group on LinkedIn  or the Advanced Matrix Factorization group on LinkedIn

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email.

Other links:

Paris Machine Learning: Meetup.com||@Archives||LinkedIn||Facebook|| @ParisMLGroup< br/> About LightOn: Newsletter ||@LightOnIO|| on LinkedIn || on CrunchBase || our Blog

About myself: LightOn || Google Scholar || LinkedIn ||@IgorCarron ||Homepage||ArXiv

Courses: Marc Lelarge's hands-on tour to deep learning with PyTorch

** Nuit Blanche is now on Twitter: @NuitBlog **

From Marc's twitter account:

Just finished teaching a 5 days summer school: Hands-on tour to deep learning with @PyTorch (https://t.co/LjFjnTGa5j), in collaboration with @abursuc, @tlacroix6 and Stéphane d'Ascoli. All slides and notebooks available online (@GoogleColab compatible). pic.twitter.com/SEC5DmhFv8

— marc lelarge (@marc_lelarge) 1 juillet 2019

Here is Marc Lelarge's hands-on tour to deep learning with PyTorch

Day 1:

• (slides) introductory slides
• (code) a first example on Colab: dogs and cats with VGG
• (code) making a regression with autograd: intro to pytorch

Day 2:

• (slides) refresher: linear/logistic regressions, classification and PyTorch module.
• (code) understanding convolutions and your first neural network for a digit recognizer.
• (slides) embeddings and dataloader
• (code) Collaborative filtering: matrix factorization and recommender system
• (slides) Variational Autoencoder by Stéphane
• (code) AE and VAE

Day 3:

• (slides) Towards deep learning for the real world by Andrei
• (code) softmax temperature; Mixture Density Networks; Bayes by backpropagation
• (slides) Generative Adversarial Networks
• (code) Conditional and Info GANs

Day 4:

• Reccurrent Neural Networks: slides and associated code
• (code) PyTorch tutorial on char-RNN
• (code) Word2vec
• (code) Playing with word embedding
• Structured Self-attentive Sentence Embedding paper code to obtain Glove NLP mini-project

Day 5:

• (slides) Opening the black box
• (code) CAM
• (code) Adversarial examples
• Graph Neural Networks by Timothée

Follow @NuitBlog or join the CompressiveSensing Reddit, the Facebook page, the Compressive Sensing group on LinkedIn  or the Advanced Matrix Factorization group on LinkedIn

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email.

Other links:
Paris Machine Learning: Meetup.com||@Archives||LinkedIn||Facebook|| @ParisMLGroup< br/> About LightOn: Newsletter ||@LightOnIO|| on LinkedIn || on CrunchBase || our Blog
About myself: LightOn || Google Scholar || LinkedIn ||@IgorCarron ||Homepage||ArXiv

SPARS 2019, HDPA-2019 and papers from the Theoretical Physics for Deep Learning ICML 2019 Workshop

** Nuit Blanche is now on Twitter: @NuitBlog **

With the summer comes the conference and schools. This time around is no exception:

SPARS 2019 is currently going on in Toulouse, here is the list of presentations there. Some folks are tweeting about the event with the #SPARS2019 hashtag.

HDPA-2019 : High dimensional probability and algorithms, is currently taking place here in Paris.

The Theoretical Physics for Deep Learning ICML 2019 Workshop took place June 14th 2019 in Long Beach, CA. Here are all the accepted papers:

• Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes Roman Novak (Google Brain)*; Lechao Xiao (Google Brain); Jaehoon Lee (Google Brain); Yasaman Bahri (Google Brain); Greg Yang (Microsoft Research AI); Jiri Hron (University of Cambridge); Daniel Abolafia (Google Brain); Jeffrey Pennington (Google Brain); Jascha Sohl-Dickstein (Google Brain)
• Topology of Learning in Artificial Neural Networks Maxime Gabella (Magma Learning)*
• Jet grooming through reinforcement learning Frederic Dreyer (University of Oxford)*; Stefano Carrazza (University of Milan)
• Inferring the quantum density matrix with machine learning Kyle Cranmer (New York University); Siavash Golkar (NYU)*; Duccio Pappadopulo (Bloomberg)
• Backdrop: Stochastic Backpropagation Siavash Golkar (NYU)*; Kyle Cranmer (New York University)
• Explain pathology in Deep Gaussian Process using Chaos Theory Anh Tong (UNIST)*; Jaesik Choi (Ulsan National Institute of Science and Technology)
• Towards a Definition of Disentangled Representations Irina Higgins (DeepMind)*; David Amos (DeepMind); Sebastien Racaniere (DeepMind); David Pfau (DeepMind); Loic Matthey (DeepMind); Danilo Jimenez Rezende (DeepMind); Alexander Lerchner (DeepMind)
• Towards Understanding Regularization in Batch Normalization Ping Luo (The Chinese University of Hong Kong); Xinjiang Wang (); Wenqi Shao (The Chinese University of HongKong)*; Zhanglin Peng (SenseTime)
• Covariance in Physics and Convolutional Neural Networks Miranda Cheng (University of Amsterdam)*; Vassilis Anagiannis (University of Amsterdam); Maurice Weiler (University of Amsterdam); Pim de Haan (University of Amsterdam); Taco S. Cohen (Qualcomm AI Research); Max Welling (University of Amsterdam)
• Meanfield theory of activation functions in Deep Neural Networks Mirco Milletari (Microsoft)*; Thiparat Chotibut (SUTD) ; Paolo E. Trevisanutto (National University of Singapore)
• Finite size corrections for neural network Gaussian processes Joseph M Antognini (Whisper AI)*
• Analysing the dynamics of online learning in over-parameterised two-layer neural networks Sebastian Goldt (Institut de Physique théorique, Paris)*; Madhu Advani (Harvard University); Andrew Saxe (University of Oxford); Florent Krzakala (École Normale Supérieure); Lenka Zdeborova (CEA Saclay)
• A Halo Merger Tree Generation and Evaluation Framework Sandra Robles (Universidad Autónoma de Madrid); Jonathan Gómez (Pontificia Universidad Católica de Chile); Adín Ramírez Rivera (University of Campinas)*; Jenny Gonzáles (Pontificia Universidad Católica de Chile); Nelson Padilla (Pontificia Universidad Católica de Chile); Diego Dujovne (Universidad Diego Portales)
• Learning Symmetries of Classical Integrable Systems Roberto Bondesan (Qualcomm AI Research)*, Austen Lamacraft (Cavendish Laboratory, University of Cambridge, UK)
• Pathological Spectrum of the Fisher Information Matrix in Deep Neural Networks Ryo Karakida (National Institute of Advanced Industrial Science and Technology)*; Shotaro Akaho (AIST); Shun-ichi Amari (RIKEN)
• How Noise during Training Affects the Hessian Spectrum Mingwei Wei (Northwestern University); David Schwab (Facebook AI Research)*
• A Quantum Field Theory of Representation Learning Robert Bamler (University of California at Irvine)*; Stephan Mandt (University of California, Irivine)
• Convergence Properties of Neural Networks on Separable Data Remi Tachet des Combes (Microsoft Research Montreal)*; Mohammad Pezeshki (Mila & University of Montreal); Samira Shabanian (Microsoft, Canada); Aaron Courville (MILA, Université de Montréal); Yoshua Bengio (Mila)
• Universality and Capacity Metrics in Deep Neural Networks Michael Mahoney (University of California, Berkeley)*; Charles Martin (Calculation Consulting)
• Asymptotics of Wide Networks from Feynman Diagrams Guy Gur-Ari (Google)*; Ethan Dyer (Google)
• Deep Learning on the 2-Dimensional Ising Model to Extract the Crossover Region Nicholas Walker (Louisiana State Univ - Baton Rouge)*
• Large Scale Structure of the Loss Landscape of Neural Networks Stanislav Fort (Stanford University)*; Stanislaw Jastrzebski (New York University)
• Momentum Enables Large Batch Training Samuel L Smith (DeepMind)*; Erich Elsen (Google); Soham De (DeepMind)
• Learning the Arrow of Time Nasim Rahaman (University of Heidelberg)*; Steffen Wolf (Heidelberg University); Anirudh Goyal (University of Montreal); Roman Remme (Heidelberg University); Yoshua Bengio (Mila)
• Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural Networks Rohan Ghosh (National University of Singapore)*; Anupam Gupta (National University of Singapore)
• A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off Yaniv Blumenfeld (Technion)*; Dar Gilboa (Columbia University); Daniel Soudry (Technion)
• Rethinking Complexity in Deep Learning: A View from Function Space Aristide Baratin (Mila, Université de Montréal)*; Thomas George (MILA, Université de Montréal); César Laurent (Mila, Université de Montréal); Valentin Thomas (MILA); Guillaume Lajoie (Université de Montréal, Mila); Simon Lacoste-Julien (Mila, Université de Montréal)
• The Deep Learning Limit: Negative Neural Network eigenvalues just noise? Diego Granziol (Oxford)*; Stefan Zohren (University of Oxford); Stephen Roberts (Oxford); Dmitry P Vetrov (Higher School of Economics); Andrew Gordon Wilson (Cornell University); Timur Garipov (Samsung AI Center in Moscow)
• Gradient descent in Gaussian random fields as a toy model for high-dimensional optimisation Mariano Chouza (Tower Research Capital); Stephen Roberts (Oxford); Stefan Zohren (University of Oxford)*
• Deep Learning for Inverse Problems Abhejit Rajagopal (University of California, Santa Barbara)*; Vincent R Radzicki (University of California, Santa Barbara)

Follow @NuitBlog or join the CompressiveSensing Reddit, the Facebook page, the Compressive Sensing group on LinkedIn  or the Advanced Matrix Factorization group on LinkedIn

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email.

Other links:
Paris Machine Learning: Meetup.com||@Archives||LinkedIn||Facebook|| @ParisMLGroup< br/> About LightOn: Newsletter ||@LightOnIO|| on LinkedIn || on CrunchBase || our Blog
About myself: LightOn || Google Scholar || LinkedIn ||@IgorCarron ||Homepage||ArXiv

Nuit Blanche in Review (June 2019)

** Nuit Blanche is now on Twitter: @NuitBlog **

Credit: JAXA / JMA / Seán Doran, 

Planet Earth Music by Schubert/Budapest Strings, Seán Doran, 

Much has happened since the last Nuit Blanche in Review (May 2019). In particular, we featured a number of architecture search implementation approaches and had our last general Paris Machine Learning meeting of season 6 as well as several in-depth videos. Enjoy !

Implementations:

• Improving Neural Architecture Search Image Classifiers via Ensemble Learning 
• NAS-Bench-101: Towards Reproducible Neural Architecture Search 
• Meta-learning of textual representations 
• An Open Source AutoML Benchmark 
• Probabilistic Rollouts for Learning Curve Extrapolation Across Hyperparameter Settings 
• Improving Automated Variational Inference with Normalizing Flows 
• Efficient Forward Architecture Search 
• Bayesian Optimization over Sets
• Graduated Optimisation of Black-Box Functions
• Accelerating the Nelder - Mead Method with Predictive Parallel Evaluation
• Toward Instance-aware Neural Architecture Search
• A simple dynamic bandit algorithm for hyper-parameter tuning
• Towards Learning of Filter-Level Heterogeneous Compression of Convolutional Neural Networks 
• Random Search and Reproducibility for Neural Architecture Search
• Deep Learning based compressive sensing - Highly Technical Reference Page/Aggregator, implementation -
• Efficient Principal Subspace Projection of Streaming Data Through Fast Similarity Matching 

In-depth:

• One-Shot Neural Architecture Search via Compressive Sensing
• Degrees of Freedom Analysis of Unrolled Neural Networks
• Distributed Learning with Random Features
• Inverse Scattering via Transmission Matrices: Broadband Illumination and Fast Phase Retrieval Algorithms
• Random Projections for Quadratic Programs over a Euclidean Ball
• The low-rank eigenvalue problem

Videos:

• Saturday Morning Videos: AutoML Workshop at ICML 2019
• Saturday Morning Videos: L4DC: Learning for Dynamics and Control, May 30 & 31, 2019, MIT
• Fast.ai courses: Part 2: Deep Learning from the Foundations
• Ce soir: Paris Machine Learning Meetup, Season 6 Finale. Data4Good, Hybridation, Sequence Predictions, Unreliable Data

Meetings:

• CfP: The 2019 Conference on Mathematical Theory of Deep Neural Networks (DeepMath 2019) Princeton Club, New York City, Oct 31-Nov 1 2019.

Follow @NuitBlog or join the CompressiveSensing Reddit, the Facebook page, the Compressive Sensing group on LinkedIn  or the Advanced Matrix Factorization group on LinkedIn

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email.

Other links:
Paris Machine Learning: Meetup.com||@Archives||LinkedIn||Facebook|| @ParisMLGroup< br/> About LightOn: Newsletter ||@LightOnIO|| on LinkedIn || on CrunchBase || our Blog
About myself: LightOn || Google Scholar || LinkedIn ||@IgorCarron ||Homepage||ArXiv

Fast.ai courses: Part 2: Deep Learning from the Foundations

** Nuit Blanche is now on Twitter: @NuitBlog **

One of the very appealing aspects of Machine learning these days is the ability to learn from very good people. Jeremy Howard at fast.ai does that very well and he just announced a new course entitled Part 2: Deep Learning from the Foundations.

Here are the list of courses so far:

• Part 1: Practical Deep Learning for Coders 
• Part 2: Deep Learning from the Foundations
• 8 - Backprop from the foundations
• 9 - The training in depth
• 10 - Looking inside the model
• 11 - Data Block API; generic optimizer
• 12 - Advanced training; ULMFiT
• 13 - Swift: Deep Learning Basics
• 14 - Swift: Putting it all together

also

• Computational Linear Algebra for Coders
• Introduction to Machine Learning for Coders!

• Getting started
• Lessons (Part 1)
• Lessons (Part 2)
  • Part 2 overview
  • 8 - Backprop from the foundations
  • 9 - The training in depth
  • 10 - Looking inside the model
  • 11 - Data Block API; generic optimizer
  • 12 - Advanced training; ULMFiT
  • 13 - Swift: Deep Learning Basics
  • 14 - Swift: Putting it all toget

Follow @NuitBlog or join the CompressiveSensing Reddit, the Facebook page, the Compressive Sensing group on LinkedIn  or the Advanced Matrix Factorization group on LinkedIn

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email.

Other links:

Paris Machine Learning: Meetup.com||@Archives||LinkedIn||Facebook|| @ParisMLGroup< br/> About LightOn: Newsletter ||@LightOnIO|| on LinkedIn || on CrunchBase || our Blog

About myself: LightOn || Google Scholar || LinkedIn ||@IgorCarron ||Homepage||ArXiv

Improving Neural Architecture Search Image Classifiers via Ensemble Learning - implementation -

** Nuit Blanche is now on Twitter: @NuitBlog **

Improving Neural Architecture Search Image Classifiers via Ensemble Learning by Vladimir Macko, Charles Weill, Hanna Mazzawi and Javier Gonzalvo‎

AdaNAS is an algorithm for learning an ensemble that improves the performance of neural architecture search models while having a similar parameter count as single large model. Our experiments demonstrate that these ensembles improve accuracy upon a single neural network of the same size. Our models achieve comparable results with the state-of-the-art on CIFAR-10 and set a new state-of-the-art on CIFAR-100.

An implementation is ehre: https://github.com/tensorflow/adanet/tree/master/research/improve_nas

Follow @NuitBlog or join the CompressiveSensing Reddit, the Facebook page, the Compressive Sensing group on LinkedIn  or the Advanced Matrix Factorization group on LinkedIn

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email.

Other links:
Paris Machine Learning: Meetup.com||@Archives||LinkedIn||Facebook|| @ParisMLGroup< br/> About LightOn: Newsletter ||@LightOnIO|| on LinkedIn || on CrunchBase || our Blog
About myself: LightOn || Google Scholar || LinkedIn ||@IgorCarron ||Homepage||ArXiv

Degrees of Freedom Analysis of Unrolled Neural Networks

** Nuit Blanche is now on Twitter: @NuitBlog **

Studying the great convergence !

Degrees of Freedom Analysis of Unrolled Neural Networks by Morteza Mardani, Qingyun Sun, Vardan Papyan, Shreyas Vasanawala, John Pauly, David Donoho

Unrolled neural networks emerged recently as an effective model for learning inverse maps appearing in image restoration tasks. However, their generalization risk (i.e., test mean-squared-error) and its link to network design and train sample size remains mysterious. Leveraging the Stein's Unbiased Risk Estimator (SURE), this paper analyzes the generalization risk with its bias and variance components for recurrent unrolled networks. We particularly investigate the degrees-of-freedom (DOF) component of SURE, trace of the end-to-end network Jacobian, to quantify the prediction variance. We prove that DOF is well-approximated by the weighted \textit{path sparsity} of the network under incoherence conditions on the trained weights. Empirically, we examine the SURE components as a function of train sample size for both recurrent and non-recurrent (with many more parameters) unrolled networks. Our key observations indicate that: 1) DOF increases with train sample size and converges to the generalization risk for both recurrent and non-recurrent schemes; 2) recurrent network converges significantly faster (with less train samples) compared with non-recurrent scheme, hence recurrence serves as a regularization for low sample size regimes.

Follow @NuitBlog or join the CompressiveSensing Reddit, the Facebook page, the Compressive Sensing group on LinkedIn  or the Advanced Matrix Factorization group on LinkedIn

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email.

Other links:
Paris Machine Learning: Meetup.com||@Archives||LinkedIn||Facebook|| @ParisMLGroup< br/> About LightOn: Newsletter ||@LightOnIO|| on LinkedIn || on CrunchBase || our Blog
About myself: LightOn || Google Scholar || LinkedIn ||@IgorCarron ||Homepage||ArXiv

Distributed Learning with Random Features

** Nuit Blanche is now on Twitter: @NuitBlog **

Distributed Learning with Random Features by Jian Li, Yong Liu, Weiping Wang

Distributed learning and random projections are the most common techniques in large scale nonparametric statistical learning. In this paper, we study the generalization properties of kernel ridge regression using both distributed methods and random features. Theoretical analysis shows the combination remarkably reduces computational cost while preserving the optimal generalization accuracy under standard assumptions. In a benign case, O(N−−√)partitions and O(N−−√) random features are sufficient to achieve O(1/N) learning rate, where N is the labeled sample size. Further, we derive more refined results by using additional unlabeled data to enlarge the number of partitions and by generating features in a data-dependent way to reduce the number of random features.

Follow @NuitBlog or join the CompressiveSensing Reddit, the Facebook page, the Compressive Sensing group on LinkedIn  or the Advanced Matrix Factorization group on LinkedIn

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email.

Other links:
Paris Machine Learning: Meetup.com||@Archives||LinkedIn||Facebook|| @ParisMLGroup< br/> About LightOn: Newsletter ||@LightOnIO|| on LinkedIn || on CrunchBase || our Blog
About myself: LightOn || Google Scholar || LinkedIn ||@IgorCarron ||Homepage||ArXiv

NAS-Bench-101: Towards Reproducible Neural Architecture Search - implementation -

** Nuit Blanche is now on Twitter: @NuitBlog **

NAS-Bench-101: Towards Reproducible Neural Architecture Search by Chris Ying, Aaron Klein, Esteban Real, Eric Christiansen, Kevin Murphy, Frank Hutter

Recent advances in neural architecture search (NAS) demand tremendous computational resources, which makes it difficult to reproduce experiments and imposes a barrier-to-entry to researchers without access to large-scale computation. We aim to ameliorate these problems by introducing NAS-Bench-101, the first public architecture dataset for NAS research. To build NAS-Bench-101, we carefully constructed a compact, yet expressive, search space, exploiting graph isomorphisms to identify 423k unique convolutional architectures. We trained and evaluated all of these architectures multiple times on CIFAR-10 and compiled the results into a large dataset of over 5 million trained models. This allows researchers to evaluate the quality of a diverse range of models in milliseconds by querying the pre-computed dataset. We demonstrate its utility by analyzing the dataset as a whole and by benchmarking a range of architecture optimization algorithms.

Data and code for NAS-Bench-101 is here: https://github.com/google-research/nasbench

Follow @NuitBlog or join the CompressiveSensing Reddit, the Facebook page, the Compressive Sensing group on LinkedIn  or the Advanced Matrix Factorization group on LinkedIn

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email.

Other links:

Paris Machine Learning: Meetup.com||@Archives||LinkedIn||Facebook|| @ParisMLGroup< br/> About LightOn: Newsletter ||@LightOnIO|| on LinkedIn || on CrunchBase || our Blog

About myself: LightOn || Google Scholar || LinkedIn ||@IgorCarron ||Homepage||ArXiv

CfP: The 2019 Conference on Mathematical Theory of Deep Neural Networks (DeepMath 2019) Princeton Club, New York City, Oct 31-Nov 1 2019.

** Nuit Blanche is now on Twitter: @NuitBlog **

Adam let me know of the following:

Dear Igor
I hope this email finds you well. Thank you for posing our conference on your blog earlier this year. As the deadline for submission is fast approaching (June 28th), I was hoping you would post once again. We have an amazing lineup of speakers (below) and I expect the subject matter is very much in line with your readership!
Cheers!

-Adam

Here is the announcement:

ANNOUNCEMENT
The 2019 Conference on Mathematical Theory of Deep Neural Networks (DeepMath 2019)
Princeton Club, New York City, Oct 31-Nov 1 2019.
Web: https://www.deepmath-conference.com/
======= Important Dates =======
Submission deadline for 1-page abstracts: June 28, 2019Notification: TBA.
Conference: Oct 31-Nov 1 2019.
======= Speakers =======
Sanjeev Arora (Princeton University, Keynote Speaker), Anima Anandkumar (CalTech), Yasaman Bahri (Google), Minmin Chen (Google), Michael Elad (Technion), Surya Ganguli (Stanford), Tomaso Poggio (MIT), David Schwab (CUNY), Shai Shalev-Shwartz (Hebrew University), Haim Sompolinsky (Hebrew University and Harvard), and Naftali Tishby (Hebrew University).
======= Call for Abstracts =======
In addition to these high-profile invited speakers, we invite 1-page non-archival abstract submissions. Abstracts will be reviewed double-blind and presented as posters.
To complement the wealth of conferences focused on applications, all submissions for DeepMath 2019 must target theoretical and mechanistic understanding of the underlying properties of neural networks.
Insights may come from any discipline and we encourage submissions from researchers working in computer science, engineering, mathematics, neuroscience, physics, psychology, statistics, or related fields.
Topics may address any area of deep learning theory, including architectures, computation, expressivity, generalization, optimization, representations, and may apply to any or all network types including fully connected, recurrent, convolutional, randomly connected, or other network topologies.
======= Conference Topic =======
Recent advances in deep neural networks (DNNs), combined with open, easily-accessible implementations, have made DNNs a powerful, versatile method used widely in both machine learning and neuroscience. These advances in practical results, however, have far outpaced a formal understanding of these networks and their training. Recently, long-past-due theoretical results have begun to emerge, shedding light on the properties of large, adaptive, distributed learning architectures.
Following the success of the 2018 IAS-Princeton joint symposium on the same topic (https://sites.google.com/site/princetondeepmath/home), the 2019 meeting is more centrally located and broader in scope, but remains focused on rigorous theoretical understanding of deep neural networks.

Follow @NuitBlog or join the CompressiveSensing Reddit, the Facebook page, the Compressive Sensing group on LinkedIn  or the Advanced Matrix Factorization group on LinkedIn

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email.

Other links:

Paris Machine Learning: Meetup.com||@Archives||LinkedIn||Facebook|| @ParisMLGroup< br/> About LightOn: Newsletter ||@LightOnIO|| on LinkedIn || on CrunchBase || our Blog

About myself: LightOn || Google Scholar || LinkedIn ||@IgorCarron ||Homepage||ArXiv