Databases
StackOverflow2013
Postsdbo
POWhat is the state of the art in HTML content extraction?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhat is the state of the art in HTML content extraction?
    primarykey
    data
    text
    <p>There's a lot of scholarly work on HTML content extraction, e.g., Gupta &amp; Kaiser (2005) <a href="http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.60.357" rel="nofollow noreferrer">Extracting Content from Accessible Web Pages</a>, and some signs of interest here, e.g., <a href="https://stackoverflow.com/questions/435547/html-downloading-and-text-extraction">one</a>, <a href="https://stackoverflow.com/questions/1386107/text-extraction-from-html-java">two</a>, and <a href="https://stackoverflow.com/questions/1696914/extracting-pure-content-text-from-html-pages-by-excluding-navigation-and-chrome">three</a>, but I'm not really clear about how well the practice of the latter reflects the ideas of the former. What is the best practice?</p> <p>Pointers to good (in particular, open source) implementations and good scholarly surveys of implementations would be the kind of thing I'm looking for.</p> <p><strong>Postscript the first</strong>: To be precise, the kind of survey I'm after would be a paper (published, unpublished, whatever) that discusses both criteria from the scholarly literature, and a number of existing implementations, and analyses how unsuccessful the implementations are from the viewpoint of the criteria. And, really, a post to a mailing list would work for me too.</p> <p><strong>Postscript the second</strong> To be clear, after Peter Rowell's answer, which I have accepted, we can see that this question leads to two subquestions: (i) the solved problem of cleaning up non-conformant HTML, for which Beautiful Soup is the most recommended solution, and (ii) the unsolved problem or separating cruft (mostly site-added boilerplate and promotional material) from meat (the contentthat the kind of people who think the page might be interesting in fact find relevant. To address the state of the art, new answers need to address the cruft-from-meat peoblem explicitly.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCommunity
    1. USCharles Stewart
    plurals
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POWhat is the state of the art in HTML content extraction?
      1. USfastcodejava
      1. VTFavorite
    2. VO
      singulars
      1. POWhat is the state of the art in HTML content extraction?
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POWhat is the state of the art in HTML content extraction?
      1. USAndrew Harry
      1. VTFavorite
    1. CO"Scholarly Surveys"? Like in scholarly journals? Any particular journals? What's wrong with citeseer? Not scholarly enough? Seriously -- what searches have you used?
      singulars
      1. POWhat is the state of the art in HTML content extraction?
      1. USS.Lott
    2. COHow goes the research? any new insights?
      singulars
      1. POWhat is the state of the art in HTML content extraction?
      1. USAndrew Harry
    3. CO@Harry: Nothing very neat, I'm afraid.
      singulars
      1. POWhat is the state of the art in HTML content extraction?
      1. USCharles Stewart
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload