Manitest: Are classifiers really invariant? - implementation -

 As I was reading about IGO and Emmy Noether's theorem, Hussein just sent me the following:

We recently published with Pascal Frossard a paper that introduces a method to quantify the invariance of arbitrary classifiers to geometric transformations. Our method is based on viewing the set of transformed images as a smooth manifold and define our invariance measure as a well-chosen geodesic distance on that manifold. The paper is on the arXiv http://arxiv.org/abs/1507.06535 and the code is available on the project website https://sites.google.com/site/invmanitest/

If you have comments or questions, please let us know! Thanks,

Hussein.

Thanks Hussein! 

 

Manitest: Are classifiers really invariant?  by Alhussein Fawzi, Pascal Frossard

Invariance to geometric transformations is a highly desirable property of automatic classifiers in many image recognition tasks. Nevertheless, it is unclear to which extent state-of-the-art classifiers are invariant to basic transformations such as rotations and translations. This is mainly due to the lack of general methods that properly measure such an invariance. In this paper, we propose a rigorous and systematic approach for quantifying the invariance to geometric transformations of any classifier. Our key idea is to cast the problem of assessing a classifier's invariance as the computation of geodesics along the manifold of transformed images. We propose the Manitest method, built on the efficient Fast Marching algorithm to compute the invariance of classifiers. Our new method quantifies in particular the importance of data augmentation for learning invariance from data, and the increased invariance of convolutional neural networks with depth. We foresee that the proposed generic tool for measuring invariance to a large class of geometric transformations and arbitrary classifiers will have many applications for evaluating and comparing classifiers based on their invariance, and help improving the invariance of existing classifiers.