Databases
StackOverflow2013
Postsdbo
POContinuous vs Discrete artificial neural networks
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POContinuous vs Discrete artificial neural networks
    text
    copied!<p>I realize that this is probably a very niche question, but has anyone had experience with working with continuous neural networks? I'm specifically interested in what a continuous neural network may be useful for vs what you normally use discrete neural networks for.</p> <p>For clarity I will clear up what I mean by continuous neural network as I suppose it can be interpreted to mean different things. I do <strong>not</strong> mean that the activation function is continuous. Rather I allude to the idea of a increasing the number of neurons in the hidden layer to an infinite amount. </p> <p>So for clarity, here is the architecture of your typical discreet NN: <img src="https://sites.google.com/site/garamatt/nn.png" alt="alt text"> The <code>x</code> are the input, the <code>g</code> is the activation of the hidden layer, the <code>v</code> are the weights of the hidden layer, the <code>w</code> are the weights of the output layer, the <code>b</code> is the bias and apparently the output layer has a linear activation (namely none.) </p> <p>The difference between a discrete NN and a continuous NN is depicted by this figure: <img src="https://sites.google.com/site/garamatt/nn2.png" alt="alt text"> That is you let the number of hidden neurons become infinite so that your final output is an integral. In practice this means that instead of computing a deterministic sum you instead must approximate the corresponding integral with quadrature. </p> <p>Apparently its a common misconception with neural networks that too many hidden neurons produces over-fitting.</p> <p>My question is specifically, given this definition of discrete and continuous neural networks, I was wondering if anyone had experience working with the latter and what sort of things they used them for.</p> <p>Further description on the topic can be found here: <a href="http://www.iro.umontreal.ca/~lisa/seminaires/18-04-2006.pdf" rel="nofollow noreferrer">http://www.iro.umontreal.ca/~lisa/seminaires/18-04-2006.pdf</a></p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload