Databases
StackOverflow2013
Postsdbo
POScipy optimization algorithms? (for minimizing neural network cost function) - python
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POScipy optimization algorithms? (for minimizing neural network cost function) - python
    text
    copied!<p>I wrote a neural network object in python that has a cost function and determines its gradients via back-propogation. I see a bunch of optimization functions <a href="http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_cg.html#scipy.optimize.fmin_cg" rel="nofollow">here</a>, but I have no idea how to implement them. I'm also having a hard time finding any example code to learn from.</p> <p>Clearly I need to somehow tell it what parameters I'm trying to change, the cost function I'm trying to minimize, and then the gradient calculated by backprop. How do I tell, say, fmin_cg what's what?</p> <p>bonus question: where can I learn about the differences in uses of the various algorithms?</p> <p>===== OK, update =====</p> <p>This is what I have atm:</p> <pre><code>def train(self, x, y_vals, iters = 400): t0 = concatenate((self.hid_t.reshape(-1), self.out_t.reshape(-1)), 1) self.forward_prop(x, t0) c = lambda v: self.cost(x, y_vals, v) g = lambda v: self.back_prop(y_vals, v) t_best = fmin_cg(c, t0, g, disp=True, maxiter=iters) self.hid_t = reshape(t_best[:,:(hid_n * (in_n+1))], (hid_n, in_n+1)) self.out_t = reshape(t_best[:,(hid_n * (in_n+1)):], (out_n, hid_n+1)) </code></pre> <p>And, this is the error it's throwing:</p> <pre><code>Traceback (most recent call last): File "&lt;stdin&gt;", line 1, in &lt;module&gt; File "netset.py", line 27, in &lt;module&gt; net.train(x,y) File "neuralnet.py", line 60, in train t_best = fmin_cg(c, t0, g, disp=True, maxiter=iters) File "/usr/local/lib/python2.7/dist-packages/scipy/optimize/optimize.py", line 952, in fmin_cg res = _minimize_cg(f, x0, args, fprime, callback=callback, **opts) File "/usr/local/lib/python2.7/dist-packages/scipy/optimize/optimize.py", line 1017, in _minimize_cg deltak = numpy.dot(gfk, gfk) ValueError: matrices are not aligned </code></pre> <p>...Halp!</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload