Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>You can speed things up a bit from what mtrw posted above just by doing what you initially described (generating a bunch of random numbers and multiplying and dividing accordingly)... </p> <p>Also, you probably already know this, but be sure to do the operations in-place (*=, /=, +=, etc) when working with large-ish numpy arrays. It makes a huge difference in memory usage with large arrays, and will give a considerable speed increase, too.</p> <pre><code>In [53]: def rand_row_doubles(row_limits, num): ....: ncols = len(row_limits) ....: x = np.random.random((num, ncols)) ....: x *= row_limits ....: return x ....: In [59]: %timeit rand_row_doubles(np.arange(7) + 1, 1000000) 10 loops, best of 3: 187 ms per loop </code></pre> <p>As compared to:</p> <pre><code>In [66]: %timeit ManyRandDoubles(np.arange(7) + 1, 1000000) 1 loops, best of 3: 222 ms per loop </code></pre> <p>It's not a huge difference, but if you're <em>really</em> worried about speed, it's something.</p> <p>Just to show that it's correct:</p> <pre><code>In [68]: x.max(0) Out[68]: array([ 0.99999991, 1.99999971, 2.99999737, 3.99999569, 4.99999836, 5.99999114, 6.99999738]) In [69]: x.min(0) Out[69]: array([ 4.02099599e-07, 4.41729377e-07, 4.33480302e-08, 7.43497138e-06, 1.28446819e-05, 4.27614385e-07, 1.34106753e-05]) </code></pre> <p>Likewise, for your "rows sum to one" part...</p> <pre><code>In [70]: def rand_rows_sum_to_one(nrows, ncols): ....: x = np.random.random((ncols, nrows)) ....: y = x.sum(axis=0) ....: x /= y ....: return x.T ....: In [71]: %timeit rand_rows_sum_to_one(1000000, 13) 1 loops, best of 3: 455 ms per loop In [72]: x = rand_rows_sum_to_one(1000000, 13) In [73]: x.sum(axis=1) Out[73]: array([ 1., 1., 1., ..., 1., 1., 1.]) </code></pre> <p>Honestly, even if you re-implement things in C, I'm not sure you'll be able to beat numpy by much on this one... I could be very wrong, though!</p>
    singulars
    1. This table or related slice is empty.
    1. POFastest Way to generate 1,000,000+ random numbers in python
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USJoe Kington
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POFastest Way to generate 1,000,000+ random numbers in python
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. CO@Joe - I tried your method for the limited numbers and found it to be slower on my machine. I'm curious if you could try mine and compare? I also stole your method for the sum-to-1 numbers; it was way faster than what I was trying before. Thanks!
      singulars
      1. PO
      1. USmtrw
    2. CO@mtrw- Your updated functions are faster than mine by a fair bit, now. (166ms vs 184ms) Yours don't require that the entire chunk of memory be contiguous, just the memory for each column, which I think is what's mostly causing the difference. The downside is in accessing the data after it's created. You'll have to use list comprehensions (or similar) for yours, whereas mine returns a single 2D numpy array (slightly faster and much more flexible indexing). It doesn't matter much if you just need to access one row at a time, though. Cheers!
      singulars
      1. PO
      1. USJoe Kington
    3. COThanks for the hard work! Trying to piece the code together now...
      singulars
      1. PO
      1. USSandro
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload