Databases
StackOverflow2013
Postsdbo
POWhy is this PowerShell code (Invoke-WebRequest / getElementsByTagName) so incredibly slow on my machines, but not others?
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhy is this PowerShell code (Invoke-WebRequest / getElementsByTagName) so incredibly slow on my machines, but not others?
    text
    copied!<p>I wrote some screen-scraping code in PowerShell and was surprised that it took around 30 seconds to parse a few HTML tables. I stripped it down to try and figure out where all the time was being spent, and it seems to be in the <code>getElementsByTagName</code> calls.</p> <p>I've included a script below which on both my home desktop, my work desktop and my home slate, takes around 1-2 seconds for each iteration (full results pasted below). However, other people in the PowerShell community are reporting far shorter times (only several milliseconds for each iteration).</p> <p>I'm struggling to find any way of narrowing down the problem, and there doesn't seem to be a pattern to the OS/PS/.NET/IE versions.</p> <p>The desktop I'm currently running it on is a brand new Windows 8 install with only PS3 and .NET 4.5 installed (and all Windows Update patches). No Visual Studio. No PowerShell profile.</p> <pre><code>$url = "http://www.icy-veins.com/restoration-shaman-wow-pve-healing-gear-loot-best-in-slot" $response = (iwr $url).ParsedHtml # Loop through the h2 tags $response.body.getElementsByTagName("h2") | foreach { # Get the table that comes after the heading $slotTable = $_.nextSibling # Grab the rows from the table, skipping the first row (column headers) measure-command { $rows = $slotTable.getElementsByTagName("tr") | select -Skip 1 } | select TotalMilliseconds } </code></pre> <p>Results from my desktop (the work PC and slate give near identical results):</p> <pre><code>TotalMilliseconds ----------------- 1575.7633 2371.5566 1073.7552 2307.8844 1779.5518 1063.9977 1588.5112 1372.4927 1248.7245 1718.3555 3283.843 2931.1616 2557.8595 1230.5093 995.2934 </code></pre> <p>However, some people in the <a href="https://plus.google.com/communities/114336958783305019912" rel="nofollow">Google+ PowerShell community reported results like this</a>:</p> <pre><code> TotalMilliseconds ----------------- 76.9098 112.6745 56.6522 140.5845 84.9599 48.6669 79.9283 73.4511 94.0683 81.4443 147.809 139.2805 111.4078 56.3881 41.3386 </code></pre> <p>I've tried both PowerShell ISE and a standard console, no difference. For the work being done, these times seem kinda excessive, and judging by the posts in the <a href="https://plus.google.com/communities/114336958783305019912" rel="nofollow">Google+ community</a>, it can go quicker!</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload