StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POJava persistent key-value store
text
Body
copied!<p>I know this may have been asked a zillion times before but I cannot seem to find the golden solution for my exact use case.</p> <p>I only have one data structure, a map where the key is a string. The objects of the map are maps themselves but this time the values are simple objects/primitives such as string, int, double, etc. So a map of maps. The keys of the innermost map is constant, i.e. no entries are ever added/removed from the innermost map except when created. So it is kind of like a traditional table, albeit each row may have arbitrary columns.</p> <p>I need this data structure to be persistent and replicated.</p> <p>Here are my requirements:</p> <ol> <li>Pure Java solution </li> <li>The disk map is only used in case of re-start. Hence there are never any reads from disk and all the writing is only done by one application)</li> <li>Embedded.</li> <li>Performance. It is the UPDATE performance of existing records that is important. UPDATEs will happen potentially 100k times per second (but more likely 20-50k per second). As for INSERTs/DELETEs they do of course happen but probably only a few times per day. Hence I do not worry too much about INSERT/DELETE performance. </li> <li>Replicated. For resilience I need the disk copy of the map to be replicated. The replication from master to slave does not need to be part of the original transaction, i.e. I can sacrifice some ACIDness for performance.</li> <li>Number of records is expected to be 100k-200k, but not much higher. The size of each record is probably 100-200 KBytes so really not that much data in total. I'm guessing the total size of the data file will be below 100 MBytes and that is probably an estimate on the high side.</li> <li>The total amount of data is not more than it can always fit in memory. (this is why I can guarantee that there will be no disk reads, except during startup)</li> <li>My application is not distributed. At any given point in time there's only one active process that writes to disk.</li> <li>Liberal open source license. (Apache, BSD, LGPL, should be fine)</li> </ol> <p>The application in question never needs to store anything but the above data structure, i.e. it will not have a future uncovered need for other persistent data structures. Hence it sounds fair to optimize based on this particular data structure.</p> <p>I've looked at Berkeley DB Java edition but it fails on requirement #6. I've looked at TokyoCabinet/KoyotoCabinet but it fails on requirement #1.</p> <p>So what would you recommend? </p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload