Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Use <code>narray</code> for efficient handling of large numeric arrays in Ruby. </p> <p>It is faster, and uses less memory. <a href="http://narray.rubyforge.org/" rel="nofollow">http://narray.rubyforge.org/</a></p> <p>As for the DB storage part. It is possible to add marshalling to <code>narray</code> (which doesn't have it by default):</p> <pre><code># This adds support for Marshal to NArray - found it at: http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/194510 class NArray def _dump *ignored Marshal.dump :typecode =&gt; typecode, :shape =&gt; shape, :data =&gt; to_s end def self._load buf h = Marshal.load buf typecode = h[:typecode] shape = h[:shape] data = h[:data] to_na data, typecode, *shape end end </code></pre> <p>. . . which will be quite efficient without you needing to write specific code. </p> <p>If the numbers are completely random, there will be limits to what you can achieve with compression. Truly random data is pretty much incompressible.</p> <p>If you have a good constraining model for how the numbers are arranged, then you can use knowledge of that to create a better compression ratio. The post suggesting to use a <code>Range</code> is an extreme example of that, but of course only works if you have a very simple structure.</p> <p>If the structure is more or less arbitrary, just marshall the data and apply an of-the-shelf compression algorithm to it, like <code>zlib</code>.</p> <p>Edit: The fact that the items are sorted lends itself to better compression. It also ticks a box for <code>narray</code> - it will sort numbers quicker than anything you can do manually with a Ruby <code>Array</code></p> <p>Sample code:</p> <pre><code>require 'narray' require 'zlib' # Ten thousand integers . . . n = NArray.int(10000).random(100000).sort # Compressed . . . stored = Zlib::Deflate.deflate( Marshal.dump(n), 9 ) stored.length =&gt; 14297 </code></pre> <p>That's not bad, 1.5 bytes per number (actual compression ratios will vary a lot, but you'll generally see less than 4 bytes per number using this technique)</p>
    singulars
    1. This table or related slice is empty.
    1. POEfficiently store / compress an array of integers in Ruby?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USNeil Slater
    1. USNeil Slater
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POEfficiently store / compress an array of integers in Ruby?
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    1. CONice! But you example assumes a non-sparse array. NArray doesn't support sparse arrays, so it will take a lot of space to store a two element array with indicies of 1 and 900,000,000.
      singulars
      1. PO
      1. USFred
    2. COThe OP hasn't asked for a sparse array, AFAICT. In fact, given that the array is sorted, it seems quite unlikely to me. However, it isn't 100% clear just what is being asked for, yet.
      singulars
      1. PO
      1. USNeil Slater
    3. COYes. This sounds like one of those questions you toss out to a class to get them to realize that every solution has tradeoffs. Every time a proposal is put forward, the teacher adds a condition that blows up the solution. It can be fun!
      singulars
      1. PO
      1. USFred
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload