StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POCan rapidminer extract xpaths from a list of URLS, instead of first saving the HTML pages?
text
Body
copied!<p>I've recently discovered RapidMiner, and I'm very excited about it's capabilities. However I'm still unsure if the program can help me with my specific needs. I want the program to scrape xpath matches from an URL list I've generated with another program. (it has more options then the 'crawl web' operator in RapidMiner) </p> <p>I've seen the following tutorials from Neil Mcguigan: <a href="http://vancouverdata.blogspot.com/2011/04/web-scraping-rapidminer-xpath-web.html" rel="nofollow">http://vancouverdata.blogspot.com/2011/04/web-scraping-rapidminer-xpath-web.html</a>. But the websites I try to scrape have thousands of pages, and I don't want to store them all on my pc. And the web crawler simply lacks critical features so I'm unable to use it for my purposes. Is there a way I can just make it read the URLS, and scrape the xpath's from each of those URLS?</p> <p>I've also looked at other tools for extracting html from pages, but I've been unable to figure out how they work (or even install) since I'm not a programmer. Rapidminer on the other hand is easy to install, the operator descriptions make sense but I've been unable to connect them in the right order. </p> <p>I need to have some input to keep the motivation going. I would like to know what operator I could use instead of 'process documents from files.' I've looked at 'process documents from web' but it doesn't have an input, and it still needs to crawl. Any help is much appreciated.</p> <p>Looking forward to your replies.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload