StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>Try downloading the page as a string first, then passing it to HtmlAgilityPack. This will let you isolate errors that happen during the download process from those that happen during the html parsing process. If you have an issue with proxybonanza (see end of post) you will be able to isolate that issue from a HtmlAgilityPack configuration issue.</p> <p>Download page using WebClient:</p> <pre><code>// Download page System.Net.WebClient client = new System.Net.WebClient(); client.Proxy = new System.Net.WebProxy("{proxy address and port}"); string html = client.DownloadString("http://example.com"); // Process result HtmlAgilityPack.HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument(); htmlDoc.LoadHtml(html); </code></pre> <p>If you want more control over the request, use <a href="http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.aspx" rel="nofollow">System.Net.HttpWebRequest</a>:</p> <pre><code>// Create request HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://example.com/"); // Apply settings (including proxy) request.Proxy = new WebProxy("{proxy address and port}"); request.KeepAlive = false; request.Timeout = 100000; request.ReadWriteTimeout = 1000000; request.ProtocolVersion = HttpVersion.Version10; // Get response try { HttpWebResponse response = (HttpWebResponse)request.GetResponse(); Stream stream = response.GetResponseStream(); StreamReader reader = new StreamReader(stream); string html = reader.ReadToEnd(); } catch (WebException) { // Handle web exceptions } catch (Exception) { // Handle other exceptions } // Process result HtmlAgilityPack.HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument(); htmlDoc.LoadHtml(html); </code></pre> <p>Also, ensure that your proxy provider (proxybonanza) allows access from your production environment to your proxies. Most providers will limit access to the proxies to certain IP addresses. They may have allowed access to the external IP of the network where you are running locally but NOT the external IP address of your production environment.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload