Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    3. VO
      singulars
      1. This table or related slice is empty.
    1. COA Huffman tree is the first thing I wrote after I saw the chunks in the trie - I was thinking of trying to encode lines as bit strings, one string for each chunk, concatenated - but while I was writing the bit-packing logic, I thought about using flat value-typed arrays for the trie encoding instead. Implementing Huffman encoding correctly and efficiently, and decoding in particular, gets pretty tedious pretty quickly. I may pick it back up and perhaps encode based on character frequency instead.
      singulars
    2. COYes, indexing using less bits than 32 is something I've thought about. Other things: 16M character data is cutting close for 24 bits, but if I aligned character data to word boundaries, costing on average 0.5 bytes per chunk, I could use 24 bits to index up to 32M position, for half the saving. And that bit-packing logic I was writing for the Huffman tree encoding, may come in useful for using less than a whole number of bytes for storing indexes. My next step will probably be writing a "bitfield array" class.
      singulars
    3. COI'll award this one the win. I wrote up a bit-packed array class that can index signed or unsigned integers of constant bit width, and I determine the maximum width required when converting from my mutable loading-time StringTrie to my immutable SlimTrie. Storing the SlimTrie on disk and reloading later saves time and memory, avoiding stale GC garbage hanging around. Now down to 75M!
      singulars
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload