Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>No doubt, <em>Google News</em> may use other tricks (or even a combination thereof), but one relatively cheap trick, computationally, to infer topics from free-text would exploit the NLP notion that <strong>a word gets its meaning only when connected to other words</strong>.<br> An algorithm susceptible of discovering new topic categories from multiple documents could be outlined as follow:</p> <ul> <li>POS (part-of-speech) tag the text<br> We probably want to focus more on nouns and maybe even more so on named entities (such as <em>Obama</em> or <em>New England</em>)</li> <li>Normalize the text<br> In particular replace inflected words by their common stem. Maybe even replace some adjectives by a corresponding Named Entity (ex: Parisian ==> Paris, legal ==> law)<br> Also, remove noise words and noise expressions.</li> <li>identify some words from a list of manually maintained "current / recurring hot words" (Superbowl, Elections, scandal...)<br> This can be used in subsequent steps to provide more weight to some N-grams</li> <li>Enumerate all N-grams found in each documents (where N is 1 to say 4 or 5)<br> Be sure to count, separately, the number of occurrences of each N-gram within a given document and the number of documents which cite a given N-gram</li> <li>The most frequently cited N-grams (i.e. the ones cited in the most documents) are probably the Topics.</li> <li>Identify the existing topics (from a list of known topics)</li> <li>[optionally] Manually review the new topics</li> </ul> <p>This general recipe can also be altered to leverage other attributes of the documents and the text therein. For example the document origin (say cnn/sports vs. cnn/politics ...) can be used to select domain specific lexicons. Another example the process can more or less heavily emphasize the words/expressions from the document title (or other areas of the text with a particular mark-up).</p>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    3. VO
      singulars
      1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload