StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>If you have access to a server that can run server-side code, you could do something along the lines of this:</p> <h2>Requirements</h2> <ul> <li>jQuery script to call scraper located on same server</li> <li>Server-side language to handle url scraping (in this case, PHP w/ curl is used)</li> </ul> <h2>PHP Scraper</h2> <pre><code><?php /** * Receives a url and optional callback, scrapes the url, and returns the results * @author Jason Featheringham * @link http://thejase.com */ /** * Retrieves a web page, including content, error and header information * @param string $url A web address to fetch * @return array The results of the screen scrape attempt */ function get_web_page( $url ) { $options = array( CURLOPT_RETURNTRANSFER => true, // return web page CURLOPT_HEADER => false, // don't return headers CURLOPT_FOLLOWLOCATION => true, // follow redirects CURLOPT_ENCODING => "", // handle all encodings CURLOPT_USERAGENT => "spider", // who am i CURLOPT_AUTOREFERER => true, // set referer on redirect CURLOPT_CONNECTTIMEOUT => 120, // timeout on connect CURLOPT_TIMEOUT => 120, // timeout on response CURLOPT_MAXREDIRS => 10, // stop after 10 redirects ); $ch = curl_init( $url ); curl_setopt_array( $ch, $options ); $result = curl_getinfo( $ch ); $result['content'] = curl_exec( $ch ); $result['error']['number'] = curl_errno( $ch ); $result['error']['message'] = curl_error( $ch ); curl_close( $ch ); return $result; } // either fetch web page or generate error message for result $result = json_encode( ( $url = $_GET['url'] ) ? get_web_page( $url ) : Array( "error" => Array( "message" => "You must specify a url parameter." ) ) ); // if callback is specified, return JSONP result, otherwise just JSON echo ( $callback = $_GET['callback'] ) ? header("text/javascript") ?: "$callback($result);" : header("application/json") ?: $result; ?> </code></pre> <h2>jQuery Code</h2> <pre><code>$.getJSON( "scraper.php?url=http://www.yahoo.com&callback=?", function(result) { if( result.error ) { // handle error } // otherwise, use the result object (usually result.content) as you see fit // ... }); </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload