AutoML Challenge: Python Notebooks for Round 1 and more...

In  the Sunday Morning Insight entry entitled The Hardest Challenges We Should be Unwilling to Postpone,  I mentioned a challenge set up by Isabelle Guyon entitled the AutoML challenge ( http://codalab.org/AutoML, her presentation is here). In short, the idea is to have a Kaggle like challenge that features several datasets of increasing difficulty and see how algorithm entries fare with these different datasets. Deep down, the algorithm needs to pay attention to its own running time and have a nice way of automatically select relevant features.

With Franck, we decided to use the mighty power of the large membership of the Paris Machine Learning meetup (Top 5 in the world) to help out in the setting up of a day long hackaton so that local teams could participate in the challenge. Currently round 1 of the challenge is over we are currently in the Tweakathon1 stage where you can submit codes that will eventually be run automatically on May 15 for AutoML2. From here:

Tweakathon1 

Continue practicing on the same data (the phase 1 data are now available for download from the 'Get Data' page). In preparation for phase 2, submit code capable of producing predictions on both VALIDATION AND TEST DATA. The leaderboard shows scores on phase 1 validation data only.

AutoML2


Start: May 15, 2015, 11:59 p.m.

Description: INTERMEDIATE phase on multiclass classification problems. Blind test of the code on NEW DATA: There is NO NEW SUBMISSION. The last code submitted in phase 1 is run automatically on the new phase 2 datasets. [+] Prize winning phase.


Tweakathon2  

Start: May 16, 2015, 11:59 p.m.

Description: Continue practicing on the same data (the data are now available for download from the 'Get Data' page). In preparation for phase 3, submit code capable of producing predictions on both VALIDATION AND TEST DATA. The leaderboard shows scores on phase 2 validation data only. 

Here are some of the presentations made during the hackaton and some of the attendant python notebooks released for tweakaton 1:

• Isabelle Guyon and Lukasz Romaszko (ChaLearn): Presentation of the AutoML challenge. Tips to solve it and win! 
• Olivier Grisel (INRIA): How to use Scikit-Learn to solve machine learning problems. 
• iPython notebook example given in talk
• iPython notebook to solve round 1of the AutoML challenge
• Julien Demouth (NVIDIA): Deep Neural Networks and GPUs.

The page for the hackaton is here. A big thank you to Pierre Roussel for hosting us at ESPCI ParisTech and to the coaches

• Delphine Le, Ecole Centrale Paris
• Djalel Benbouzid, Univ. Paris Saclay
• Bogdan-Ionut Cirstea, ENST, Paris
• Olivier Grisel, INRIA, Paris
• Lovro Ilijasic, Univ. Paris Saclay
• Lukasz Romazko, CVC Barcelona, ChaLearn

Other links:

• NVIDIA machine learning
• Introduction paper (accepted to IJCNN 2015)
• Sample code for the hackathon (do not submit as is, run the code on your local computer, then submit results).
• GPU track instructions
• Sklearn Kaggle examples

  Join the CompressiveSensing subreddit or the Google+ Community and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.