Databases
StackOverflow2013
Postsdbo
POXpath query to grab text between different html tags
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POXpath query to grab text between different html tags
    text
    copied!<p>I am using R to screen scrape. I've grabbed a page and I've managed to find all the links on the page that found in a certain place on the page (anchor tags within anchor tags with a name attribute) using:</p> <pre><code>links &lt;- xpathSApply(doc, "//a[@name]//a/@href") </code></pre> <p>Now I have grabbed got the documents from the links with Curl and I want to scrape a certain amount of text. The text seems to always be between an <code>&lt;p&gt;</code> tag (although there are other <code>&lt;p&gt;</code> tags in the text and end before the following text </p> <pre><code>&lt;/pre&gt;&lt;hr&gt;Back to: &lt;a href="#TOP"&gt; </code></pre> <p>I decided to grab all the text between <code>&lt;p&gt;</code> and <code>&lt;a href="#TOP"&gt;</code> and I cant seem to nail the xpath query. So far I have got:</p> <pre><code>text &lt;- xpathSApply(doc, '"/ //text()[preceding:://a/@href="#TOP"] and following::*//p') </code></pre> <p>Could anyone point me in the right direction? There are quite a few xpath answers on stackoverflow but they don't always explain the answer which makes it hard to edit them for my own use.</p> <p>Sample HTML:</p> <pre><code>&lt;span ID="MSGHDR-CONTENT-TYPE-H-PRE"&gt;Content-type:&lt;/b&gt;&lt;/span&gt; &lt;span ID="MSGHDR-CONTENT- TYPE-PRE"&gt;text/plain; charset=us-ascii&lt;/span&gt; &lt;/span&gt;&lt;p&gt; lots and lots of text here that I want &lt;/pre&gt;&lt;hr&gt;Back to: &lt;a href="#TOP"&gt;Top of message&lt;/a&gt; &amp;#124; &lt;a href="/cgi-bin/wa?A1=ind9709&amp;L=cybcom&amp;D=0"&gt;Previous page&lt;/a&gt; &amp;#124; &lt;a href="/cgi-bin/wa?A0=cybcom&amp;D=0"&gt;Main CYBCOM page&lt;/a&gt;&lt;p&gt; </code></pre>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload