Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>From experimenting further with this and reading about <a href="http://redis.io/topics/persistence" rel="noreferrer">redis persistence</a>, I think the following observations can be made:</p> <ul> <li>When using RDB (the default settings), redis will fork each time a <code>save</code> operation is triggered, which (by default) is set to once every 15 minutes <em>as a minimum</em>. When more writes to Redis are performed, then RDB writes are as frequent as once every <em>60 seconds</em>.</li> <li>Each fork will use a "copy-on-write" memory allocation, which means that whilst memory won't actually double - it will appear so on tools like <code>ps</code>, <code>htop</code> and the like.</li> <li>The fork itself can be quite a cpu-intensive operation, <a href="http://redis.io/topics/latency" rel="noreferrer">particularly on xen-based virtual hosts</a> (which is what we're using currently).</li> <li>The write operation seems to <em>completely overwrite</em> the existing RDB file. It does not write only the changes, but rather dumps the <em>entire</em> dataset to disk.</li> </ul> <p>So on a modest virtual host with 4Gb RAM and data set of around 750Mb (at the time I posted the question), this starts to become rather "expensive". We observed those CPU/Memory spikes, as well as increased IO, even under fairly moderate load / redis usage.</p> <p>So to answer my own question - this does seem to be the "expected" behaviour.</p> <p>As for improving the situation, we opted to switch our configuration to use a combination of RDB and AOF. AOF (Append Only File), does appear to only write <em>changes</em> to disk. You can (and should) still configure the AOF file to rewrite (using <code>auto-aof-rewrite-percentage</code> and <code>auto-aof-rewrite-min-size</code> settings). It is also advisable to still use RDB for snapshots. In this configuration however, you can probably do full rewrites / snapshots less frequently and still maintain pretty-good performance and even-better durability.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload