StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>You want the <a href="http://nokogiri.org/Nokogiri/XML/Node.html#method-i-inner_text" rel="noreferrer"><code>Nokogiri::XML::Node#inner_text</code></a> method:</p> <pre><code>require 'nokogiri' require 'open-uri' html = Nokogiri::HTML(open 'http://stackoverflow.com/questions/6129357') # Alternatively html = Nokogiri::HTML(IO.read 'myfile.html') text = html.at('body').inner_text # Pretend that all words we care about contain only a-z, 0-9, or underscores words = text.scan(/\w+/) p words.length, words.uniq.length, words.uniq.sort[0..8] #=> 907 #=> 428 #=> ["0", "1", "100", "15px", "2", "20", "2011", "220px", "24158nokogiri"] # How about words that are only letters? words = text.scan(/[a-z]+/i) p words.length, words.uniq.length, words.uniq.sort[0..5] #=> 872 #=> 406 #=> ["Answer", "Ask", "Badges", "Browse", "DocumentFragment", "Email"] </code></pre> <pre class="lang-rb prettyprint-override"><code># Find the most frequent words require 'pp' def frequencies(words) Hash[ words.group_by(&:downcase).map{ |word,instances| [word,instances.length] }.sort_by(&:last).reverse ] end pp frequencies(words) #=> {"nokogiri"=>34, #=> "a"=>27, #=> "html"=>18, #=> "function"=>17, #=> "s"=>13, #=> "var"=>13, #=> "b"=>12, #=> "c"=>11, #=> ... # Hrm...let's drop the javascript code out of our words html.css('script').remove words = html.at('body').inner_text.scan(/\w+/) pp frequencies(words) #=> {"nokogiri"=>36, #=> "words"=>18, #=> "html"=>17, #=> "text"=>13, #=> "with"=>12, #=> "a"=>12, #=> "the"=>11, #=> "and"=>11, #=> ... </code></pre>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload