Databases
StackOverflow2013
Postsdbo
POautomatically stop scipy.optimize.fmin_bfgs after n function calls (not BFGS iterations!)
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POautomatically stop scipy.optimize.fmin_bfgs after n function calls (not BFGS iterations!)
    primarykey
    data
    text
    <p>I use logistic regression with <strong>scipy.optimize.fmin_bfgs</strong> for minimizing the cost function. The cost function stays constant for my particular data set and BFGS does not converge, so I want to apply lasso+ridge regularization.</p> <p>Now, I want to try out optimizing the cost for various values of the regularization parameters lambda1/2 in order to find the best combination:</p> <pre><code>for lambda1 in range(...): for lambda2 in range(..): scipy.optimize.fmin_bfgs(...) # Optimize cost with lambda1 and lambda2 </code></pre> <p>The problem is that, because BFGS is not converging, it stays "forever" in the call for the first values of lambda1/2.</p> <p>Is there a way to automatically stop fmin_bfgs after a while? The <strong>maxiter</strong> parameter does not help me, because I have 1000s of samples and a large number of features/sample, so it doesn't even finish one such iteration in acceptable time.</p> <p>In scipy 0.11, fmin_bfgs has a <strong>maxfun</strong> parameter -- can one emulate this somehow in scipy 0.10?</p> <p><strong>EDIT: By popular demand, here are some relevant snippets of code:</strong></p> <p>The function computing the cost (the usual notations apply):</p> <pre><code>def computeCost(theta, X, y): h = sigmoid(X.dot(theta.T)) J = y.T.dot(log(h)) + (1.0 - y.T).dot(log(1.0 - h)) J_reg2 = theta[1:]**2 J_reg1 = theta[1:] cost = (-1.0 / m) * (J.sum() + LAMBDA2 * J_reg2.sum() + LAMBDA1 * J_reg1.sum()) return cost </code></pre> <p>Invoking the fmin_bfgs function:</p> <pre><code>initial_thetas = numpy.zeros((len(train_X[0]), 1)) myargs = (train_X, train_y) theta = scipy.optimize.fmin_bfgs(computeCost, x0=initial_thetas, args=myargs) </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCristina
    1. USCristina
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POautomatically stop scipy.optimize.fmin_bfgs after n function calls (not BFGS iterations!)
      1. This table or related slice is empty.
      1. VTUpMod
    1. COA workaround way would be to add a sort of number of calls counter to the function itself and `raise` a special exception when the desired number of calls is reached.
      singulars
      1. POautomatically stop scipy.optimize.fmin_bfgs after n function calls (not BFGS iterations!)
      1. USev-br
    2. CO@Zhenya Thanks! I was hoping there might be an easier way, but I might end up doing that.
      singulars
      1. POautomatically stop scipy.optimize.fmin_bfgs after n function calls (not BFGS iterations!)
      1. USCristina
    3. CO@Denis What do you mean by "delta X"? I have my training samples in X, no modifications are done to them.
      singulars
      1. POautomatically stop scipy.optimize.fmin_bfgs after n function calls (not BFGS iterations!)
      1. USCristina
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload