Note that there are some explanatory texts on larger screens.

plurals
  1. UScryo28
    primarykey
    data
    text
    plurals
    1. This table or related slice is empty.
    1. COI don't think it makes sense to retry more frequently than once every 3 seconds. I'd even increase it. Only you can decide how long you can wait for the result. This code is synchronous and it blocks until timeout specified. If you run this code from a ruby script from some background task - you can wait for quite a long time. Like 3-5 minutes. But if the code is run as part of processing of real user http requests you usually don't want to wait at all. Even 1 second is too much. So it is correct to implement some asynchronous checking with server side push or ajax long-poll.
      singulars
    2. COYou surely can build some kind of inheritance hierarchy for individual scrapers. And you might be good with just keeping different xpath selectors. However, in my experience, usually scraping of multiple sites is not that simple as specifying different xpath selectors. Usually one have to implement some kind of a custom non-typical scraping logic for some of the sites. So, being able to hook into different parts of the scraping logic is a must-have feature. And this is exactly the place where object hierarchy comes to the stage.
      singulars
    3. COYes. I know that will work. Thank you. But it is not really what I am looking for as I will have to run the script with correct arguments for every new projects. And I am sure I will forget to do it sooner or later. So I am more into the solution I provided in the link above. I'd be happy to have it modified to choose not the email only, but author/email pair.
      singulars
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload