Do Androids Recall Dreams of Electric Sheeps ?

This gripping scene from Blade Runner asks the simple question of how one could possibly store and recall all the memories of an android. It turns out that one of the process by which machine vision is enabled these days is through the computation of  keypoint descriptors  such as SIFT and others. The idea is to use the keypoint descriptors to perform comparison, permit classification, enable detection and other similar degraded operations. 

It turns out that our friends at EPFL, who brought is the FREAK descriptors seem to think that a reconstruction is possible from these degraded information. The FREAK descriptors were mentioned earlier and 

are inspired  by the Fovea,. The image reconstruction is described in the following paper:.  

Beyond Bits: Reconstructing Images from Local Binary Descriptors by Emmanuel D'Angelo. Alexandre Alahi, Pierre Vandergheynst.The abstract

Local Binary Descriptors (LBDs) are good at matching image parts, but how much information is actually carried? Surprisingly, this question is usually ignored and replaced by a comparison of matching performances. In this paper, we directly address it by trying to reconstruct plausible images from different LBDs such as BRIEF and FREAK. Using an inverse problem framework, we show that this task is achievable with only the information in the descriptors, excluding the need of additional data. Hence, our results represent a novel justification for the performance of LBDs. Furthermore, since plausible images can be inferred using only these simple measurements, this emphasizes the concerns about privacy and secrecy of image keypoints raised by [Weinzaepfel 2011], that could have an important impact on public applications of image matching.

Wow! The Binarized FREAK seem to produce understandable images. I am waiting to see an implementation of this.

Credit: Blade Runner, Warner Bros. The title of this entry is based Philip K. Dick's Do Androids Dream of Electric Sheep? who's plot inspired Blade Runner..

Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.