tag:blogger.com,1999:blog-6141980.post2446694736711248232..comments2024-03-20T12:28:35.004-05:00Comments on Nuit Blanche: Sunday Morning Insight: Randomization is not a dirty wordIgorhttp://www.blogger.com/profile/17474880327699002140noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-6141980.post-50996028737712852752013-12-17T15:03:44.506-06:002013-12-17T15:03:44.506-06:00Yes, diversity of approaches is at the same time s...Yes, diversity of approaches is at the same time stimulating and bewildering.<br />As an example I found truly fascinating that the same idea of "randomizing" a recurrent neural network came up three times in the span of a few years (Echo State Networks, Liquid State Machines, Backpropagation-Decorrelation). Researchers are starting to appreciate the correlation with ELMs, as an example the following paper uses an architecture composed of a combination of a ESN and a ELM:<br />http://www.sciencedirect.com/science/article/pii/S0893608012003085<br />There is also another connection between ELMs and a neuroscientific framework called Neuroengineering, that I first learned about here:<br />http://www.sciencedirect.com/science/article/pii/S089360801300049X<br />By going back in time one could also connect these architectures to the earliest works on NNs (eg the Perceptron) or to the works on Random Projections in the Eighties...<br />Sorry for the long post and congratulations for the blog.<br />SimoneSimone Scardapanehttps://www.blogger.com/profile/00174003090612832725noreply@blogger.comtag:blogger.com,1999:blog-6141980.post-17545432811439002072013-12-17T05:29:00.503-06:002013-12-17T05:29:00.503-06:00Simone,
Thanks for the heads-up.
The dearth of p...Simone,<br /><br />Thanks for the heads-up.<br /><br />The dearth of papers and the fact that these communities barely know each other is a sure sign we are at the edge of our collective knowledge. If you do make the connection between these and the ESN, then please let me know, that new community needs to know about this.<br /><br />Cheers,<br /><br />Igor.Igorhttps://www.blogger.com/profile/17474880327699002140noreply@blogger.comtag:blogger.com,1999:blog-6141980.post-87999976369218668672013-12-16T07:15:34.851-06:002013-12-16T07:15:34.851-06:00Very nice article, the explosion of randomized tec...Very nice article, the explosion of randomized techniques in ML is one of the things that interest me most. I would add to the discussion the work on Reservoir Computing approaches, e.g. Echo State Networks, that can be seen as the recurrent counterpart to ELM methods. I would love to see the connections between all these subfields analyzed in detail.Simone Scardapanehttps://www.blogger.com/profile/00174003090612832725noreply@blogger.com