Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases

 We just had a meetup on Stan where Eric mentioned Michael Betancourt's video presentation on Hamiltonian Monte Carlo (see below) and I note the Random Features expansion can speed some of these computations:

Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases by Cheng Zhang, Babak Shahbaba, Hongkai Zhao

For big data analysis, high computational cost for Bayesian methods often limits their applications in practice. In recent years, there have been many attempts to improve computational efficiency of Bayesian inference. Here we propose an efficient and scalable computational technique for a state-of-the-art Markov Chain Monte Carlo (MCMC) methods, namely, Hamiltonian Monte Carlo (HMC). The key idea is to explore and exploit the structure and regularity in parameter space for the underlying probabilistic model to construct an effective approximation of its geometric properties. To this end, we build a surrogate function to approximate the target distribution using properly chosen random bases and an efficient optimization process. The resulting method provides a flexible, scalable, and efficient sampling algorithm, which converges to the correct target distribution. We show that by choosing the basis functions and optimization process differently, our method can be related to other approaches for the construction of surrogate functions such as generalized additive models or Gaussian process models. Experiments based on simulated and real data show that our approach leads to substantially more efficient sampling algorithms compared to existing state-of-the art methods.

The Geometric Foundations of Hamiltonian Monte Carlo
M. J. Betancourt, Simon Byrne, Samuel Livingstone, Mark Girolami

Although Hamiltonian Monte Carlo has proven an empirical success, the lack of a rigorous theoretical understanding of the algorithm has in many ways impeded both principled developments of the method and use of the algorithm in practice. In this paper we develop the formal foundations of the algorithm through the construction of measures on smooth manifolds, and demonstrate how the theory naturally identifies efficient implementations and motivates promising generalizations.

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Paris Machine Learning Meetup, Hors série #2: Scalable Machine Learning with H2O

 

Tonight, we will have Paris Machine Learning Meetup, Hors série #2 on Scalable Machine Learning with H2O. Jiqiong Qiu is a co-organizer of this meetup with Franck Bardol and Igor Carron. This "Hors Série 2" will be focused on a few presentations of H2O -H20.ai- by Jo-fai Chow and Jakub Háva. The meetup will be hosted and sponsored by Murex.

If you want to try the H2o, feel free to follow this installation guide. We provide not only the stand alone installation guide but also a dockerfile with H2o+Tensorflow+Spark.

All the presentations are at: http://bit.ly/h2o_paris_1


1. Introduction to Machine Learning with H2O (Joe - 30 mins)
In this talk, I will walk you through our company (H2O.ai), our open-source machine learning platform (H2O) and use cases from some of our users. This will be useful for attendees who are not familiar with H2O.

2. Project “Deep Water” (H2O integration with TensorFlow and other deep learning libraries) (Joe - 30 mins)
The “Deep Water" project is about integrating our H2O platform with other open-source deep learning libraries such as TensorFlow from Google. We are also working GPU implementation for H2O.In this talk about the motivation and potential benefits of our recent project named “Deep Water”. 

3. Sparkling Water 2.0 (Jakub - 45 mins)
Sparkling Water integrates the H2O open source distributed machine learning platform with the capabilities of Apache Spark. It allows users to leverage H2O’s machine learning algorithms with Apache Spark applications via Scala, Python, R or H2O’s Flow GUI which makes Sparkling Water a great enterprise solution. Sparkling Water 2.0 was built to coincide with the release of Apache Spark 2.0 and introduces several new features. These include the ability to use H2O frames as Apache Spark’s SQL datasource, transparent integration into Apache Spark machine learning pipelines, the power to use Apache Spark algorithms via the Flow GUI and easier deployment of Sparkling Water in a Python environment. In this talk we will introduce the basic architecture of Sparkling Water and provide an overview of the new features available in Sparkling Water 2.0. The talk will also include a live demo showing how to integrate H2O algorithms into Apache Spark pipelines – no terminal needed!

About Joe :https://uk.linkedin.com/in/jofaichow 
Jo-fai (or Joe) is a data scientist at H2O.ai. Before joining H2O, he was in the business intelligence team at Virgin Media where he developed data products to enable quick and smart business decisions. He also worked (part-time) for Domino Data Lab as a data science evangelist promoting products via blogging and giving talks at meetups. Joe has a background in water engineering. Before his data science journey, he was an EngD researcher at STREAM Industrial Doctorate Centre working on machine learning techniques for drainage design optimization. Prior to that, he was an asset management consultant specialized in data mining and constrained optimization for the utilities sector in UK and abroad. He also holds a MSc in Environmental Management and a BEng in Civil Engineering.

About Jakub :https://cz.linkedin.com/in/havaj
Jakub (or “Kuba”) finished his bachelors degree in computer science at Charles University in Prague, and is currently finishing his master’s in software engineering as well. As a bachelors thesis, Kuba wrote a small platform for distributed computing of tasks of any type. On his current masters studies he’s developing a cluster monitoring tool for JVM based languages which should make debugging and reasoning about performance of distributed systems easier using a concept called distributed stack traces. At H2O, Kuba mostly works on Sparkling Water project.

 

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Paris Machine Learning Meetup, Hors série #1: Introduction to Bayesian Inference with Stan and R

For this first Hors série Paris Machine Learning meetup of the season, Dataiku is co-organizing this meetup. To register for the even you need to go there. Streaming video is here and will start at around 7:00PM Paris time.

 

Presentation slides should be available before the meetup:

Eric Novik, Introduction to Bayesian Inference with Stan and R

Stan is a modern, high-performance probabilistic programming language that interfaces with R, Python, Julie, Matlab, and Stata. Eric Novik, one of the organizers of the NYC Bayesian Data Analysis meetup and a founder of Stan Group, will be in town to give a brief talk on Stan.
We will build up a simple Stan model from scratch to demonstrate various parts of the Stan program and will also present a multi-level, overdispersed Poison model that can be used for pricing products in a retail setting. We fit the latter model with the rstanarm package, which can be thought of as a fully Bayesian equivalent to lme4.
After the talk, Eric and Stan core developers Daniel Lee and Michael Betancourt will answer questions from the audience and offer some thoughts on the future of statistical inference and Bayesian computing.

 
Eric Kramer, DSS + Stan

  Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

CSjobs: Two Ph.D. positions at TU Wien (Austria) and Aalto University (Finland)

Alex just sent me the following:

Hi Igor, 

We have two Phd positions on the application of Compressed Sensing to Complex Networks available at TU Vienna and Aalto University. It would be great if you could circulate the ad on your blog. 

Thanks, Alex 

 

Sure Alex !

----- 

The Institute of Telecommunications (Institute of Telecommunications) at TU Wien (Vienna, Austria) has two openings for full-time Ph.D. positions to conduct cutting-edge research in the general area of signal processing for big data in close collaboration with the Machine Learning for Big Data Group (http://research.cs.aalto.fi/MLBigDat) at Aalto University (Finland). 

In the context of the WWTF project Co3-iGraB, we develop foundations for extracting useful information from massive data sets. The key idea is to tackle big data problems by a combination of graph signal processing for complex networks and compressed sensing. The project is anticipated to lead to novel and versatile inference tools that are applicable to numerous real-world problems. 

DESIRED SKILLS AND EXPERIENCE We look for talented and passionate researchers to work on problems in statistical/sparse graph signal processing, graph learning, and distributed/non-convex optimization. Interested candidates should hold a master degree (or equivalent) in Electrical Engineering, Mathematics, Computer Science, or a related discipline, and show a strong background in mathematics and analytical thinking. Expertise in signal processing, statistics, optimization, information theory, or distributed computing is advantageous. Excellent written and oral communication skills in English are a must.

 Applications including a cover letter, a detailed curriculum vitae, course transcripts, and the names of two referees should be sent by email to Prof. Gerald Matz (gerald.matz@tuwien.ac.at) with cc to Prof. Alex Jung (alexander.jung@aalto.fi) no later than Sept. 30, 2016. 

ABOUT THE EMPLOYER The selected candidates will be offered a three-year Ph.D. contract with competitive salary and will have the opportunity to be part of a dynamic and internationally recognized team in an inspiring research environment. Multiple long-term stays with our collaboration partner at Aalto University (Finland) are envisaged. TU Wien is a leading academic institution, located in Austria's capital Vienna, offering superior quality of living in the heart of Europe

 More information can be found here.

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

CfP: SPARS 2017 Lisbon, Portugal, June 5-8, 2017

The SPARS 2017 workshop will take place in Lisbon and they have a call for papers:

The Signal Processing with Adaptive Sparse Structured Representations (SPARS) workshop aims at bringing together people from statistics, engineering, mathematics, and computer science, fostering the exchange and dissemination of new ideas and results, both applied and theoretical, on the general area of sparsity-related techniques and computational methods, for high dimensional data analysis, signal processing, and related applications. 

SPARS 2017 will be held at Instituto Superior Técnico (IST), the engineering school of the University of Lisbon, on June 5-8, 2017.

In addition to 8 plenary lectures, the workshop will feature a single track format with approximately 30 standard (20min) talks, and 3 poster/demo sessions.

         Main topics: 

• Sparse coding and representations, and dictionary learning.
• Sparse and low-rank approximation algorithms.
• Compressive sensing and learning.
• Dimensionality reduction and feature extraction.
• Sparsity measures in approximation theory, information theory, and statistics.
• Low-complexity/low-dimensional regularization.
• Statistical and Bayesian models and algorithms for sparsity.
• Sparse network theory and analysis.
• Sparsity and low-rank regularization.
• Applications.

Invited  speakers:

• Yoram Bresler, Beckman Institute, University of Illinois, USA.
• Volkan Cevher, École Polytechnique Fédérale de Lausanne, Switzerland.
• Jalal Fadili, École Nationale Supérieure d'Ingénieurs de Caen, France.
• Anders Hansen, University of Cambridge, UK.
• Gitta Kutyniok, Technische Universität Berlin, Germany.
  
• Philip Schniter, Ohio State University, USA.
• Eero Simoncelli, Howard Hughes Medical Institute, New York University, USA.
• Rebecca Willett, University of Wisconsin, USA.

A SpaRTan/MacSeNet Summer School on Sparse Representations will be organized during the week before SPARS 2017. More information will follow shortly.





Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Sunday Morning Insight: "More F$*%(g Data"

On Friday, we had a, too short, panel discussion on the AI ecosystem here in France. At the end, Paul asked us what would be needed to have an even more dynamic ecosystem:

#franceisai @IgorCarron : "we need to have more access to data from the public system" to develop AI ecosystem pic.twitter.com/AHitWxSn3z

— Marion Cassiau (@MarionCassiau) 16 septembre 2016

After thinking about it a little while, it became clear to me that the answer was "More Data". Later that day, I met a person who wanted to organize a Hackaton in the area of Data Science. Again, it was clear that to attract local data scientists, you needed to have unheard-of and fascinating datasets. As a result of these interactions, I have been thinking of the type of datasets or access to datasets that could have large impacts in the start-up area and on our local community:
Health: If the British NHS and Deepmind did a deal in Great Britain, it would seem a shrewd move by the French state to open the health datasets owned by the wide variety of the health stakeholders. Yes, privacy issues are central and yes they can be solved efficiently, Most importantly, if we do not use them, we are not making our health-AI ecosystem dynamic. It is important to remember that by default it is a better move by the state to not move in that direction. Until people realize that you will get better diagnostics and treatment in the UK than in France, there will not be a move to exploit these data. By that time, the game will really be over for French start-ups to really make an impact. Let me make the additional comment that current start-ups that are doing well in that realm are companies that have been able to get their data from outside french soil. It is not just a shame and it's tragic.
Movie industry: The FranceisAI meeting took place at BPI France, a short 200 meters away from where the first public showing of a movie, at which admission was charged, took place.  The French created the movie industry and started many of the business practices and narrative used in scripts that stand to this day. Nowadays, under the reasoning that some things need to be protected, entire collections are seldom reachable by start-ups and artists types.  The database of INA is one such example but I am sure it is not the only one. What I notice is that a company like Netflix, that did not exist 15 years ago, is becoming a giant in the TV industry because of its awesome data driven work. It looks as if it is even becoming better now because of its ability to create and analyse "More F$*%(g Data". I am sure start-ups here in Paris could fast track their analytical tools based on local databases such as that of INA.
Environment: At the last Paris Machine Learning meetup this past Wednesday, we had David Klein, a data scientist in California who helps entities like conservation metrics in quantifying conservation actions throughout the world (his presentation is here). His presentation is exciting and listed in the meetup's archives here. Go read it, I'll wait....During the Q&A, one of our audience participant asked if, like the turtles David could detect, we could do the same for bull sharks. Why bull sharks you say ?  Well these sharks have had a tremendous impact on the local economy of the Reunion island, a French "département" in the Indian Ocean, Currently the French fisheries ministry staff goes through the lengthy process of tagging the sharks which can then be detected by sparsely located offline sensors. After watching David's presentation, we were wondering about the possibility of using acoustic sensors or drones with cameras and Deep Learning to detect in time (not after the fact) if any bull sharks were getting too close to the beach/surf areas. Because of the non-responsiveness of the current detection system, the state conservation department has had to remove some sharks. While the solution might appease the contentious relations with the population, it is simply neither a guarantee for humans nor it is ideal for the shark population. Using AI/ML capabilities with an offline detection system would still detect the non tagged sharks: a capability that currently simply does not exist or would cost a lot of money if it required the tagging of every sharks in the region. I am sure that there are other wildlife issues that could be solved using some of these techniques but since the environment has mostly been the realm of the state, it is high time it opens up and provides "More F$*%(g Data".
I could go on and on about different subject areas where the state owns more data than it can make sense of. Let me point one dataset out that does not seem to have a direct economic impact because it is looks too sciency: Earthquake detection. The Institut de physique du globe de Paris has some of the very large datasets that would be ideal for a beautiful hackaton or for start-ups that want to try their algorithms. These dataset should not be seen as just for the Science community. If the story of Kaggle is any indication, even Science can change as a result of releasing "More F$*%(g Data".
Current models such as those used in Deep Learning /AI require large amounts of data for their training. Policymakers need to understand that if homegrown start-ups and the attendant ecosystems in general are to strive and have an edge, it is because and only because they can have access to large amounts of F$*%(g data. 

Image Credit: NASA/JPL-Caltech This image was taken by Rear Hazcam: Left B (RHAZ_LEFT_B) onboard NASA's Mars rover Curiosity on Sol 1463 (2016-09-17 11:56:14 UTC). Full Resolution

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

CSjobs: Several Post-Doctoral Research Associate Signal and Image Processing Institute, University of Southern California

Mahdi just sent me the following:

 

Hi Igor,

We have several postdoc positions available at USC signal and image processing institute. We would appreciate it if you can circulate the attached description on your blog.

Thanks,

Mahdi

Sure Mahdi ! here is the announcement:

Post-Doctoral Research Associate Signal and Image Processing Institute, University of Southern California

Several Postdoctoral Research Associate positions are available immediately for an exciting new government funded project with the goal of 3D coherent x-ray imaging of silicon integrated circuits at less than less than 10nm resolution. Successful candidates will work as part of an interdisciplinary multi-institution team (including USC, Northwestern University, Argonne National Labs and the Paul Scherrer Institute), with a focus on system modeling, simulation, image analysis and computational image reconstruction from sparsely-sampled data. The position will involve algorithm development and analysis, software implementations, evaluation on experimental data, and preparation of research articles. Required Qualifications: PhD in Electrical Engineering, Statistics, Computer Science, or Physics. Programming experience, preferably including Matlab, Python, C++. Experience and publications in at least one of the following areas: diffractive optics, optical simulation, coherent diffraction imaging, phaseless imaging, ptychography, computational imaging, 3D tomographic image reconstruction, inverse problems, low-dimensional signal representations (sparsity, low-rank, etc.), and numerical optimization.

Successful applicants will join the Signal and Image Processing Institute in the Department of Electrical Engineering and work with a team of faculty including Richard Leahy, Anthony Levi, Justin Haldar and Mahdi Soltanolkotabi.

The University of Southern California strongly values diversity and is committed to equal opportunity in employment. Women and men, and members of all racial and ethnic groups, are encouraged to apply.

Send applications to:

Richard M. Leahy, Ph.D.
Professor and Director
Signal and Image Processing Institute
3740 McClintock Ave, EEB400
University of Southern California
Los Angeles, CA 90089 2564
http://neuroimage.usc.edu
leahy@sipi.usc.edu

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

A Computationally Efficient Projection-Based Approach for Spatial Generalized Linear Mixed Models

A Computationally Efficient Projection-Based Approach for Spatial Generalized Linear Mixed Models by Yawen Guan, Murali Haran

Inference for spatial generalized linear mixed models (SGLMMs) for high-dimensional non-Gaussian spatial data is computationally intensive. The computational challenge is due to the high-dimensional random effects and because Markov chain Monte Carlo (MCMC) algorithms for these models tend to be slow mixing. Moreover, spatial confounding inflates the variance of fixed effect (regression coefficient) estimates. Our approach addresses both the computational and confounding issues by replacing the high-dimensional spatial random effects with a reduced-dimensional representation based on random projections. Standard MCMC algorithms mix well and the reduced-dimensional setting speeds up computations per iteration. We show, via simulated examples, that Bayesian inference for this reduced-dimensional approach works well both in terms of inference as well as prediction; our methods also compare favorably to existing "reduced-rank" approaches. We also apply our methods to two real world data examples, one on bird count data and the other classifying rock types.

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Paris Machine Learning #1 Season 4: AlphaGo, Deep Learning & Global Biodiversity, DataScience Game

Tonight, we will have the first meetup of season 4. We just passed 4000 members, woohoo ! 

We should have three presentations, if you are interested to do one:  it is here.

Franck Bardol and Igor Carron. Introduction

+ calendrier/calendar

• FranceisAI Sep 16-18
• Hors série 1 Introduction to Stan Sep 20,
• Hors série 2 Scalable Machine Learning with H2O Sep 21
• Paris Machine Learning Meetup #2 Season 4, October 12

Others 

• Paris NLP 1, Sep 28
• Deep Learning,
• Bordeaux Machine Learning, Sep 27

 Speakers and slides:

Brief presentations

• Louis Dorard, Le Machine Learning Canvas 
• Antoine, le Paris NLP meetup

Longer presentations 
 
David Klein - 'Deep Learning for Global Biodiversity Monitoring'

Abstract: Healthy ecosystems with intact biodiversity provide human societies with valuable services such as clean air and water, storm protection, tourism, medicine, food, and cultural resources. Protecting this natural capital is one of the great challenges of our era. Species extinction and ecological degradation steadily continues despite conservation funding of roughly U.S. $20 billion per year worldwide. Measurements of conservation outcomes are often uninformative, hindering iterative improvements and innovation in the field. There is cause for optimism, however, as recent technological advances in sensor networks, data management, and machine intelligence can provide affordable and effective measures of conservation outcomes. I will present several working case studies using our system, which employs deep learning to empower biologists to analyze petabytes of sensor data from a network of remote microphones and cameras. This system, which is being used to monitor endangered species and ecosystems around the globe, has enabled an order of magnitude improvement in the cost effectiveness of such projects. This approach can be expanded to encompass a greater variety of sensor sources, such as drones, to monitor animal populations, habitat quality, and to actively deter wildlife from hazardous structures. I present a strategic vision for how data-driven approaches to conservation can drive iterative improvements through better information and outcomes-based funding mechanisms, ultimately enabling increasing returns on biodiversity investments. 

Stephane Senecal, Les techniques de Machine Learning au cœur du succès d’AlphaGo

The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and of selecting moves. After a brief review of the game of Go itself, we will focus on how researchers at Google DeepMind introduced a new approach to computer Go that uses “value networks” to evaluate board positions for predicting game issue and “policy networks” to select moves.
These models are in fact deep neural networks and are trained or learnt by a novel combination of supervised learning from human expert games, and reinforcement learning techniques from games of self-play. Basic concepts of deep neural networks models and of reinforcement learning framework will be explained.
Google DeepMind researchers also introduce a new search algorithm that combines Monte Carlo simulation techniques with value and policy networks. Using this search algorithm, and without any lookahead search (as for classical computer Chess), their program AlphaGo defeated the human European Go champion by 5 games to 0 and a world-class human Go champion by 4 games to 1. This is the first time that a computer program has defeated human professional players in the full-sized game of Go, a feat previously thought to be at least a decade away.

Antoine Ly + équipes UPMC & Polytechnique, DataScience Game
Presentation slides. 
Presentation slides UPMC

Le championnat du monde de data science étudiant 
L'histoire du Data Science Game, sa spécificité, son organisation, l'explication de ce succès.
Présence des équipes UPMC et X : questions réponses avec audience sur méthodes / algorithmes / stratégies etc

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.

Speeding up Bayesian Optimizations with Random Projections - implementations -

Random projections and features used in Bayesian optimization to go faster ! Here are two implementations.

COMBO: An Efficient Bayesian Optimization Library for Materials Science by Tsuyoshi Uenoa, Trevor David Rhoneb, Zhufeng Houc, Teruyasu Mizoguchid, Koji Tsuda

In many subfields of chemistry and physics, numerous attempts have been made to accelerate scientific discovery using data-driven experimental design algorithms. Among them, Bayesian optimization has been proven to be an effective tool. A standard implementation (e.g., scikit-learn), however, can accommodate only small training data. We designed an efficient protocol for Bayesian optimization that employs Thompson sampling, random feature maps, one-rank Cholesky update and automatic hyperparameter tuning, and implemented it as an open-source python library called COMBO (COMmon Bayesian Optimization library). Promising results using COMBO to determine the atomic structure of a crystalline interface are presented. COMBO is available at https://github.com/tsudalab/combo.

Bayesian Optimization in a Billion Dimensions via Random Embeddings by Ziyu Wang, Frank Hutter, Masrour Zoghi, David Matheson, Nando de Freitas

Bayesian optimization techniques have been successfully applied to robotics, planning, sensor placement, recommendation, advertising, intelligent user interfaces and automatic algorithm configuration. Despite these successes, the approach is restricted to problems of moderate dimension, and several workshops on Bayesian optimization have identified its scaling to high-dimensions as one of the holy grails of the field. In this paper, we introduce a novel random embedding idea to attack this problem. The resulting Random EMbedding Bayesian Optimization (REMBO) algorithm is very simple, has important invariance properties, and applies to domains with both categorical and continuous variables. We present a thorough theoretical analysis of REMBO. Empirical results confirm that REMBO can effectively solve problems with billions of dimensions, provided the intrinsic dimensionality is low. They also show that REMBO achieves state-of-the-art performance in optimizing the 47 discrete parameters of a popular mixed integer linear programming solver.

The Rembo repository is here: https://github.com/ziyuw/rembo

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.