Video: Deep Learning by Geoff Hinton, University of Cambridge, June 29th, 2015

Back in 2007, many people felt that vision ought to be a feedforward process with little or no feedbacks. And while this might be true for the process of scene understanding, Geoff Hinton makes the case that learning that understanding on the other hand, probably requires backpropagation which in turn will require us to look more deeply onto how the cortex really work. Without further ado:

the abstract:

I will describe an efficient, unsupervised learning procedure for a simple type of two-layer neural network called a Restricted Boltzmann Machine. I will then show how this algorithm can be used recursively to learn multiple layers of features without requiring any supervision. After this unsupervised “pre-training”, the features in all layers can be fine-tuned to be better at discriminating between classes by using the standard backpropagation procedure from the 1980s. Unsupervised pre-training greatly improves generalization to new data, especially when the number of labelled examples is small. Ten years ago, the pre-training approach initiated a revival of research on deep, feedforward neural networks. I will describe some of the major successes of deep networks for speech recognition, object recognition and machine translation and I will speculate about where this research is headed. The fact that backpropagation learning is now the method of choice for a wide variety of really difficult tasks means that neuroscientists may need to reconsider their well-worn arguments about why it cannot possibly be occurring in cortex. I shall conclude by undermining two of the commonest objections to the idea that cortex is actually backpropagating error derivatives through a hierarchy of cortical areas and I shall show that spike-time dependent plasticity is a signature of backpropagation.

other formats are available from Cambridge:

MPEG-4 Video	1280x720	2.99 Mbits/sec	1,45 GB	View
MPEG-4 Video	640x360	1.94 Mbits/sec	961,61 MB	View
WebM	640x360	876.95 kbits/sec	423,92 MB	View
iPod Video	480x270	522.46 kbits/sec	252,56 MB	View
MP3	44100 Hz	250.09 kbits/sec	120,90 MB	Listen

  Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.