StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>I'd send a GET request that contains a <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35" rel="nofollow noreferrer">RANGE header</a> to limit the actual data transfer where possible (the remote server might not honour the RANGE request but it's still worth a try). It probably doesn't make much difference whether you use sockets (directly) or curl to make the requests. But... you never know without benchmarks. For curl take a look at the "CURLOPT_RANGE" option at <a href="http://docs.php.net/function.curl-setopt" rel="nofollow noreferrer">http://docs.php.net/function.curl-setopt</a></p> <p>It probably doesn't fit your profile ("several an hour, on a server with only slim CPU power available.") but you might want to try handling multiple urls at a time, i.e. having multiple active connections and only handle those that won't block on a read operation. If the limiting factor is mostly/only cpu power ...forget this part. <em>sockets</em>: Take a look at <a href="http://docs.php.net/stream_select" rel="nofollow noreferrer">stream_select</a> <em>curl</em>: see <a href="http://docs.php.net/function.curl-multi-exec" rel="nofollow noreferrer">curl_multi_exec()</a></p> <p>If the curl module is unavailable you can also use the http url wrapper in combination with <a href="http://docs.php.net/function.stream-context-create" rel="nofollow noreferrer">stream_context_create()</a> to send a request containing a RANGE header.</p> <p>Looks like you've already figured out what to do with the data once you've received it.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload