Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>The performance difference is not related to the sizes of the matrices, but to the order in which you fill the data:</p> <pre><code>b[i,:,:] = np.random.rand(2000,2000) b[:,:,i] = np.random.rand(2000,2000) </code></pre> <p>In the first case, you are filling cells that are contiguous in memory. In the second case, cells are scattered across memory.</p> <p>When items are in contiguous memory, all neighbour cells will probably be cached when the first of them is fetched. In the other case, when one is fetched, most of its neighboring cells that will be kept in cache will not be used.</p> <p>For ilustration purposes, let's consider the two-dimensional case, and let's assume that two items fit in cache. The following matrix:</p> <pre><code>numpy.array('[[10, 20, 30], [40, 50, 60]]') </code></pre> <p>is stored like this in memory:</p> <pre><code>10 20 30 40 50 60 </code></pre> <p>Let's see what happens when we iterate over it in row order:</p> <pre><code>a[0][0] → fetch 10 from memory (cached: 10 20) a[0][1] → read 20 from cache a[0][2] → fetch 30 from memory (cached: 30 40) a[1][0] → read 40 from cache a[1][1] → fetch 50 from memory (cached: 50 60) a[1][2] → read 60 from cache </code></pre> <p>Now, let's iterate in column order:</p> <pre><code>a[0][0] → fetch 10 from memory (cached: 10 20) a[1][0] → fetch 40 from memory (cached: 30 40) a[2][1] → fetch 20 from memory (cached: 10 20) a[0][1] → fetch 50 from memory (cached: 50 60) a[1][2] → fetch 30 from memory (cached: 30 40) a[1][2] → fetch 60 from memory (cached: 50 60) </code></pre> <p>So, in the first case, you can iterate over the whole matrix using only three memory accesses, and in the second case, you need six. As a rule of thumb, reading a value from memory is ~200 times slower than reading it from cache.</p>
    singulars
    1. This table or related slice is empty.
    1. POThree dimensional datasets
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USRoberto Bonvallet
    1. USRoberto Bonvallet
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POThree dimensional datasets
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COThanks for your reply! Do you have any idea of where I can find information about cells position in memory?
      singulars
      1. PO
      1. USbserra
    2. CO@bserra: In Numpy, the cells are stored row-wise by default, i.e. in this order: a[0,0,0], a[0,0,1], a[0,0,2]..., but you can change it to column-wise: `numpy.array('[[1, 2, 3], [4, 5, 6]]', order='F')`.
      singulars
      1. PO
      1. USRoberto Bonvallet
    3. COWikipedia discusses it at [Row-major_order](http://en.wikipedia.org/wiki/Row-major_order).
      singulars
      1. PO
      1. USJonathan Leffler
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload