Databases
StackOverflow2013
Postsdbo
POscraping website with javascript cookie with c#
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POscraping website with javascript cookie with c#
    text
    copied!<p>I want to scrape some things from the following site: <a href="http://www.conrad.nl/modelspoor" rel="nofollow noreferrer">http://www.conrad.nl/modelspoor</a>.</p> <p>This is my function:</p> <pre><code>public string SreenScrape(string urlBase, string urlPath) { CookieContainer cookieContainer = new CookieContainer(); HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(urlBase + urlPath); httpWebRequest.CookieContainer = cookieContainer; httpWebRequest.UserAgent = "Mozilla/6.0 (Windows; U; Windows NT 7.0; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.9 (.NET CLR 3.5.30729)"; WebResponse webResponse = httpWebRequest.GetResponse(); string result = new System.IO.StreamReader(webResponse.GetResponseStream(), Encoding.Default).ReadToEnd(); webResponse.Close(); if (result.Contains("&lt;frame src=")) { Regex metaregex = new Regex("http:[a-z:/._0-9!?=A-Z&amp;]*",RegexOptions.Multiline); result = result.Replace("\r\n", ""); Match m = metaregex.Match(result); string key = m.Groups[0].Value; foreach (Match match in metaregex.Matches(result)) { HttpWebRequest redirectHttpWebRequest = (HttpWebRequest)WebRequest.Create(key); redirectHttpWebRequest.CookieContainer = cookieContainer; webResponse = redirectHttpWebRequest.GetResponse(); string redirectResponse = new System.IO.StreamReader(webResponse.GetResponseStream(), Encoding.Default).ReadToEnd(); webResponse.Close(); return redirectResponse; } } return result; } </code></pre> <p>But when i do this i get a string with an error from the website that it use javascript.</p> <p>Does anybody know how to fix this?</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload