Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <h3>Re-use Other People's Algorithms</h3> <p>There are a ton of <a href="http://graphics.stanford.edu/~seander/bithacks.html" rel="nofollow">bit-calculation optimizations out there</a> including many that are non-obvious, like <a href="http://en.wikipedia.org/wiki/Hamming_weight" rel="nofollow">Hamming Weights</a> and specialized algorithms for finding the next true or false bit, that are rather independent of how you structure your data.</p> <p>Reusing algorithms that other people have written can really speed up computation and lookups, not to mention development time. Some algorithms are so specialized and use computational magic that will have you scratching your head: in that case, you can take the author's word for it (<strong>after you confirm their correctness with unit tests</strong>).</p> <h3>Take Advantage of CPU Caching and Multithreading</h3> <p>I personally reduce my multidimensional bit arrays to one dimension, optimized for expected traversal. </p> <p><em>This way, there is a greater chance of hitting the CPU cache.</em></p> <p>In your case, I would also think deeply about the mutability of the data and whether you want to put locks on blocks of bits. With 100MBs of data, you have the potential of running your algorithms in parallel using many threads, if you can structure your data and algorithms to avoid contention.</p> <p>You may even have a <strong>lockless</strong> model if you divide up ownership of the blocks of data by thread so no two threads can read or write to the same block. It all depends on your requirements.</p> <p>Now is a good time to think about these issues. But since no one knows your data and usage better than you do, you must consider design options in the context of your data and usage patterns.</p>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    3. VO
      singulars
      1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload