StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POPaging in a Rest Collection
text
Body
copied!<p>I'm interested in exposing a direct REST interface to collections of JSON documents (think <a href="http://couchdb.apache.org/" rel="noreferrer">CouchDB</a> or <a href="http://persvr.org/" rel="noreferrer">Persevere</a>). The problem I'm running into is how to handle the <code>GET</code> operation on the collection root if the collection is large.</p> <p>As an example pretend I'm exposing StackOverflow's <code>Questions</code> table where each row is exposed as a document (not that there necessarily is such a table, just a concrete example of a sizable collection of 'documents'). The collection would be made available at <code>/db/questions</code> with the usual CRUD api <code>GET /db/questions/XXX</code>, <code>PUT /db/questions/XXX</code>, <code>POST /db/questions</code> is in play. The standard way to get the entire collection is to <code>GET /db/questions</code> but if that naively dumps each row as a JSON object, you'll get a rather sizeable download and a lot of work on the part of the server. </p> <p>The solution is, of course, paging. Dojo has solved this problem in its <a href="http://docs.dojocampus.org/dojox/data/JsonRestStore#id7" rel="noreferrer">JsonRestStore</a> via a clever RFC2616-compliant extension of using the <code>Range</code> header with a custom range unit <code>items</code>. The result is a <code>206 Partial Content</code> that returns only the requested range. The advantage of this approach over a query parameter is that it leaves the query string for...queries (e.g. <code>GET /db/questions/?score>200</code> or somesuch, and yes that'd be encoded <code>%3E</code>).</p> <p>This approach completely covers the behavior I want. The problem is that <a href="http://tools.ietf.org/html/rfc2616" rel="noreferrer">RFC 2616</a> specifies that on a 206 response (emphasis mine):</p> <blockquote> <p>The <strong>request</strong> MUST have included a Range header field (<a href="http://tools.ietf.org/html/rfc2616#section-14.35" rel="noreferrer">section 14.35</a>) indicating the desired range, and MAY have included an If-Range header field (<a href="http://tools.ietf.org/html/rfc2616#section-14.27" rel="noreferrer">section 14.27</a>) to make the request conditional.</p> </blockquote> <p>This makes sense in the context of the standard usage of the header but is a problem because I'd like the 206 response to be the default to handle naive clients/random people exploring.</p> <p>I've gone over the RFC in detail looking for a solution but have been unhappy with my solutions and am interested in SO's take on the problem. </p> <p>Ideas I've had:</p> <ul> <li><em>Return <code>200</code> with a <code>Content-Range</code> header!</em> - I don't think that this is wrong, but I'd prefer if a more obvious indicator that the response is only Partial Content.</li> <li><em>Return <code>400 Range Required</code></em> - There is not a special 400 response code for required headers, so the default error has to be used and read by hand. This also makes exploration via web browser (or some other client like Resty) more difficult.</li> <li><em>Use a query parameter</em> - The standard approach, but I'm hoping to allow queries a la Persevere and this cuts into the query namespace.</li> <li><em>Just return <code>206</code>!</em> - I think most clients wouldn't freak out, but I'd rather not go against a MUST in the RFC</li> <li><em>Extend the spec! Return <code>266 Partial Content</code></em> - Behaves exactly like 206 but is in response to a request that MUST NOT contain the <code>Range</code> header. I figure that 266 is high enough that I shouldn't run into collision issues and it makes sense to me but I'm not clear on whether this is considered taboo or not.</li> </ul> <p>I'd think this is a fairly common problem and I'd like to see this done in a sort of de facto fashion so I or someone else isn't reinventing the wheel.</p> <p>What's the best way to expose a full collection via HTTP when the collection is large?</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload