q-Neurons: Neuron Activations based on Stochastic Jackson's Derivative Operators

Ke just sent me the following:

Hello Igor,

I am doing machine learning and information geometry. I have been following your blog and enjoyed the many interesting stuff.


We have just built a new type of stochastic neuron:
https://arxiv.org/abs/1806.00149which is super simple to implement while shows consistently better performance.


If possible, could you kindly make a small announcement about it on nuit-blanche? Thanks in advance!

Best,
Ke

https://courbure.com/k

Thanks Ke !

q-Neurons: Neuron Activations based on Stochastic Jackson's Derivative Operators by Frank Nielsen, Ke Sun

We propose a new generic type of stochastic neurons, called 

q

-neurons, that considers activation functions based on Jackson's 

q

-derivatives with stochastic parameters 

q

. Our generalization of neural network architectures with 

q

-neurons is shown to be both scalable and very easy to implement. We demonstrate experimentally consistently improved performances over state-of-the-art standard activation functions, both on training and testing loss functions.

Join the CompressiveSensing subreddit or the Google+ Community or the Facebook page and post there !
Liked this entry ? subscribe to Nuit Blanche's feed, there's more where that came from. You can also subscribe to Nuit Blanche by Email, explore the Big Picture in Compressive Sensing or the Matrix Factorization Jungle and join the conversations on compressive sensing, advanced matrix factorization and calibration issues on Linkedin.