If you recall some of the ideas expressed in this Sunday Morning Insight entry on Matrix Factorizations and the Grammar of Life, you will like the next items as it pertains to this path for extracting structures from datasets. Some of the authors featured in the Sunday Morning Insight entry have come up with a new paper entitled: Structure Discovery in Nonparametric Regression through Compositional Kernel Search by David Duvenaud, James Robert Lloyd, Roger Grosse, Joshua B. Tenenbaum, Zoubin Ghahramani. The abstract reads:
Despite its importance, choosing the structural form of the kernel in nonparametric regression remains a black art. We define a space of kernel structures which are built compositionally by adding and multiplying a small number of base kernels. We present a method for searching over this space of structures which mirrors the scientific discovery process. The learned structures can often decompose functions into interpretable components and enable long-range extrapolation on time-series datasets. Our structure search method outperforms many widely used kernels and kernel combination methods on a variety of prediction tasks.
poster and presentation Automated Structure Discovery in Nonparametric Regression through Compositional Grammars and a related webpage The Kernel Cookbook: Advice on Covariance functions all by David Duvenaud. The most important part of this paper and attendant document is the availability of an implementation on Github at:
Also from David Duvenaud's series of talks:
Introduction to Probabilistic Programming and Automated InferenceComputational and Biological Learning Lab, University of Cambridge, March 2013
Meta-reasoning and Bounded RationalityTea talk, Feb 2013
Of related interest for background: Gaussian Processes for Machine Learning by Carl Edward Rasmussen and Christopher K. I. Williams
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.