Databases
StackOverflow2013
Postsdbo
POHow should I implement simple caches with concurrency on Redis?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow should I implement simple caches with concurrency on Redis?
    primarykey
    data
    text
    <h1>Background</h1> <p>I have a 2-tier web service - just my app server and an RDBMS. I want to move to a pool of identical app servers behind a load balancer. I currently cache a bunch of objects in-process. I hope to move them to a shared Redis. </p> <p>I have a dozen or so caches of simple, small-sized business objects. For example, I have a set of <code>Foos</code>. Each <code>Foo</code> has a unique <code>FooId</code> and an <code>OwnerId</code>. One "owner" may own multiple <code>Foos</code>.</p> <p>In a traditional RDBMS this is just a table with an index on the PK FooId and one on OwnerId. I'm caching this in one process simply:</p> <pre><code>Dictionary&lt;int,Foo&gt; _cacheFooById; Dictionary&lt;int,HashSet&lt;int&gt;&gt; _indexFooIdsByOwnerId; </code></pre> <p>Reads come straight from here, and writes go here and to the RDBMS. I usually have this invariant:</p> <p>"For a given group [say by OwnerId], the whole group is in cache or none of it is."</p> <p>So when I cache miss on a Foo, I pull that Foo <em>and</em> all the owner's other Foos from the RDBMS. Updates make sure to keep the index up to date and respect the invariant. When an owner calls GetMyFoos I never have to worry that some are cached and some aren't.</p> <h1>What I did already</h1> <p>The first/simplest answer seems to be to use plain ol' <code>SET</code> and <code>GET</code> with a composite key and json value:</p> <pre><code>SET( "ServiceCache:Foo:" + theFoo.Id, JsonSerialize(theFoo)); </code></pre> <p>I later decided I liked:</p> <pre><code>HSET( "ServiceCache:Foo", theFoo.FooId, JsonSerialize(theFoo)); </code></pre> <p>That lets me get all the values in one cache as HVALS. It also felt right - I'm literally moving hashtables to Redis, so perhaps my top-level items should be hashes.</p> <p>This works to first order. If my high-level code is like:</p> <pre><code>UpdateCache(myFoo); AddToIndex(myFoo); </code></pre> <p>That translates into:</p> <pre><code>HSET ("ServiceCache:Foo", theFoo.FooId, JsonSerialize(theFoo)); var myFoos = JsonDeserialize( HGET ("ServiceCache:FooIndex", theFoo.OwnerId) ); myFoos.Add(theFoo.OwnerId); HSET ("ServiceCache:FooIndex", theFoo.OwnerId, JsonSerialize(myFoos)); </code></pre> <p>However, this is broken in two ways.</p> <ol> <li>Two concurrent operations can read/modify/write at the same time. The latter "wins" the final <code>HSET</code> and the former's index update is lost.</li> <li>Another operation could read the index in between the first and second lines. It would miss a Foo that it should find.</li> </ol> <h2>So how do I index properly?</h2> <p>I think I could use a Redis set instead of a json-encoded value for the index. That would solve part of the problem since the "add-to-index-if-not-already-present" would be atomic.</p> <p>I also read about using <code>MULTI</code> as a "transaction" but it doesn't seem like it does what I want. Am I right that I can't really <code>MULTI; HGET; {update}; HSET; EXEC</code> since it doesn't even do the <code>HGET</code> before I issue the <code>EXEC</code>?</p> <p>I also read about using WATCH <em>and</em> MULTI for optimistic concurrency, then retrying on failure. But WATCH only works on top-level keys. So it's back to <code>SET/GET</code> instead of <code>HSET/HGET</code>. And now I need a new index-like-thing to support getting all the values in a given cache.</p> <p>If I understand it right, I can combine all these things to do the job. Something like:</p> <pre><code>while(!succeeded) { WATCH( "ServiceCache:Foo:" + theFoo.FooId ); WATCH( "ServiceCache:FooIndexByOwner:" + theFoo.OwnerId ); WATCH( "ServiceCache:FooIndexAll" ); MULTI(); SET ("ServiceCache:Foo:" + theFoo.FooId, JsonSerialize(theFoo)); SADD ("ServiceCache:FooIndexByOwner:" + theFoo.OwnerId, theFoo.FooId); SADD ("ServiceCache:FooIndexAll", theFoo.FooId); EXEC(); //TODO somehow set succeeded properly } </code></pre> <p>Finally I'd have to translate this pseudocode into real code depending how my client library uses <code>WATCH/MULTI/EXEC</code>; it looks like they need some sort of context to hook them together.</p> <p><strong>All in all this seems like a lot of complexity for what has to be a very common case; I can't help but think there's a better, smarter, Redis-ish way to do things that I'm just not seeing.</strong></p> <h2>How do I lock properly?</h2> <p>Even if I had no indexes, there's still a (probably rare) race condition.</p> <pre><code>A: HGET - cache miss B: HGET - cache miss A: SELECT B: SELECT A: HSET C: HGET - cache hit C: UPDATE C: HSET B: HSET ** this is stale data that's clobbering C's update. </code></pre> <p>Note that C could just be a really-fast A.</p> <p>Again I think <code>WATCH</code>, <code>MULTI</code>, retry would work, but... ick.</p> <p>I know in some places people use special Redis keys as locks for other objects. Is that a reasonable approach here?</p> <p>Should those be top-level keys like <code>ServiceCache:FooLocks:{Id}</code> or <code>ServiceCache:Locks:Foo:{Id}</code>? Or make a separate hash for them - <code>ServiceCache:Locks</code> with <code>subkeys Foo:{Id}</code>, or <code>ServiceCache:Locks:Foo</code> with subkeys <code>{Id}</code> ?</p> <p>How would I work around abandoned locks, say if a transaction (or a whole server) crashes while "holding" the lock?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USsolublefish
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POHow should I implement simple caches with concurrency on Redis?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POHow should I implement simple caches with concurrency on Redis?
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload