StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PODifference on performance between numpy and matlab
text
Body
copied!<p>I am computing the <code>backpropagation</code> algorithm for a sparse autoencoder. I have implemented it in python using <code>numpy</code> and in <code>matlab</code>. The code is almost the same, but the performance is very different. The time matlab takes to complete the task is 0.252454 seconds while numpy 0.973672151566, that is almost four times more. I will call this code several times later in a minimization problem so this difference leads to several minutes of delay between the implementations. Is this a normal behaviour? How could I improve the performance in numpy?</p> <p>Numpy implementation:</p> <p>Sparse.rho is a tuning parameter, sparse.nodes are the number of nodes in the hidden layer (25), sparse.input (64) the number of nodes in the input layer, theta1 and theta2 are the weight matrices for the first and second layer respectively with dimensions 25x64 and 64x25, m is equal to 10000, rhoest has a dimension of (25,), x has a dimension of 10000x64, a3 10000x64 and a2 10000x25.</p> <p><code>UPDATE</code>: I have introduced changes in the code following some of the ideas of the responses. The performance is now numpy: 0.65 vs matlab: 0.25.</p> <pre><code>partial_j1 = np.zeros(sparse.theta1.shape) partial_j2 = np.zeros(sparse.theta2.shape) partial_b1 = np.zeros(sparse.b1.shape) partial_b2 = np.zeros(sparse.b2.shape) t = time.time() delta3t = (-(x-a3)*a3*(1-a3)).T for i in range(m): delta3 = delta3t[:,i:(i+1)] sum1 = np.dot(sparse.theta2.T,delta3) delta2 = ( sum1 + sum2 ) * a2[i:(i+1),:].T* (1 - a2[i:(i+1),:].T) partial_j1 += np.dot(delta2, a1[i:(i+1),:]) partial_j2 += np.dot(delta3, a2[i:(i+1),:]) partial_b1 += delta2 partial_b2 += delta3 print "Backprop time:", time.time() -t </code></pre> <p>Matlab implementation:</p> <pre><code>tic for i = 1:m delta3 = -(data(i,:)-a3(i,:)).*a3(i,:).*(1 - a3(i,:)); delta3 = delta3.'; sum1 = W2.'*delta3; sum2 = beta*(-sparsityParam./rhoest + (1 - sparsityParam) ./ (1.0 - rhoest) ); delta2 = ( sum1 + sum2 ) .* a2(i,:).' .* (1 - a2(i,:).'); W1grad = W1grad + delta2* a1(i,:); W2grad = W2grad + delta3* a2(i,:); b1grad = b1grad + delta2; b2grad = b2grad + delta3; end toc </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload