Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <h2>Results</h2> <p>I sped up your <em>original</em> version, since your edit 3 was actually not working (and also does something different).</p> <p>So, on my PC: </p> <p>Your (original) version: <strong>8.428473 seconds</strong>.<br> My obfuscated one-liner given below: <strong>0.964589 seconds</strong>.</p> <p>First, for no other reason than to impress, I'll give it as I wrote it:</p> <pre><code>%%// Some example data xdim = 500; ydim = 500; n_timepoints = 10; % for example estimate = zeros(xdim,ydim); %// initialization with explicit size picture = randn(xdim,ydim,n_timepoints); %%// Your original solution %// (slightly altered to make my version's results agree with yours) tic Y = randn(n_timepoints,xdim*ydim); ii = 1; for x = 1:xdim for y = 1:ydim X = squeeze(picture(x,y,:)); %// or similar creation of X matrix B = (X'*X)^(-1)*X' * Y(:,ii); ii = ii+1; %// sometimes you keep everything and do %// estimate(x,y,:) = B(:); %// sometimes just the first element is important and you do estimate(x,y) = B(1); end end toc %%// My version tic %// UNLEASH THE FURY!! estimate2 = reshape(sparse(1:xdim*ydim*n_timepoints, ... builtin('_paren', ones(n_timepoints,1)*(1:xdim*ydim),:), ... builtin('_paren', permute(picture, [3 2 1]),:))\Y(:), ydim,xdim).'; %' toc %%// Check for equality max(abs(estimate(:)-estimate2(:))) % (always less than ~1e-14) </code></pre> <h2>Breakdown</h2> <p>First, here's the version that you should <em>actually</em> use:</p> <pre><code>%// Construct sparse block-diagonal matrix %// (Type "help sparse" for more information) N = xdim*ydim; %// number of columns M = N*n_timepoints; %// number of rows ii = 1:N; jj = ones(n_timepoints,1)*(1:N); s = permute(picture, [3 2 1]); X = sparse(ii,jj(:), s(:)); %// Compute ALL the estimates at once estimates = X\Y(:); %// You loop through the *second* dimension first, so to make everything %// agree, we have to extract elements in the "wrong" order, and transpose: estimate2 = reshape(estimates, ydim,xdim).'; %' </code></pre> <p>Here's an example of what <code>picture</code> and the corresponding matrix <code>X</code> looks like for <code>xdim = ydim = n_timepoints = 2</code>:</p> <pre><code>&gt;&gt; clc, picture, full(X) picture(:,:,1) = -0.5643 -2.0504 -0.1656 0.4497 picture(:,:,2) = 0.6397 0.7782 0.5830 -0.3138 ans = -0.5643 0 0 0 0.6397 0 0 0 0 -2.0504 0 0 0 0.7782 0 0 0 0 -0.1656 0 0 0 0.5830 0 0 0 0 0.4497 0 0 0 -0.3138 </code></pre> <p>You can see why <code>sparse</code> is necessary -- it's mostly zeros, but will grow large quickly. The full matrix would quickly consume all your RAM, while the <code>sparse</code> one will not consume much more than the original <code>picture</code> matrix does. </p> <p>With this matrix <code>X</code>, the new problem </p> <pre><code>X·b = Y </code></pre> <p>now contains <em>all</em> the problems</p> <pre><code>X1 · b1 = Y1 X2 · b2 = Y2 ... </code></pre> <p>where </p> <pre><code>b = [b1; b2; b3; ...] Y = [Y1; Y2; Y3; ...] </code></pre> <p>so, the single command </p> <pre><code>X\Y </code></pre> <p>will solve all your systems at once. </p> <p>This offloads all the hard work to a set of highly specialized, compiled to machine-specific code, optimized-in-every-way algorithms, rather than the interpreted, generic, always-two-steps-away from the hardware loops in MATLAB. </p> <p>It should be straightforward to convert this to a version where <code>X</code> is a matrix; you'll end up with something like what <code>blkdiag</code> does, which can also be used by <code>mldivide</code> in exactly the same way as above.</p>
    singulars
    1. This table or related slice is empty.
    1. POOptimizing repetitive estimation (currently a loop) in MATLAB
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USRody Oldenhuis
    1. USRody Oldenhuis
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POOptimizing repetitive estimation (currently a loop) in MATLAB
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COWhy did you change the order of the first two dimensions when using `permute`? This creates an `X` matrix that goes across columns first, then rows, which works, of course, but is sort of against the matlab preference towards columns.
      singulars
      1. PO
      1. USFrederick
    2. CO@Frederick to make the results agree with yours :) of course, with some minor tweaking this is easily corrected. Did you try this with X being a matrix yet?
      singulars
      1. PO
      1. USRody Oldenhuis
    3. COI'm working on it now. I'm overhauling the example because it's a mess . This all-matrix was the the kind of solution I had in mind, though.
      singulars
      1. PO
      1. USFrederick
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload