Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Here's a really basic description; for more details, you can read this textbook (free online): <a href="http://informationretrieval.org/" rel="noreferrer">http://informationretrieval.org/</a>¹</p> <p>1). For all files, create an index. The index consists of all unique words that occur in your dataset (called a "corpus"). With each word, a list of document ids is associated; each document id refers to a document that contains the word.</p> <p>Variations: sometimes when you generate the index you want to ignore stop words ("a", "the", etc). You have to be careful, though ("to be or not to be" is a real query composed of stopwords).</p> <p>Sometimes you also stem the words. This has more impact on search quality in non-English languages that use suffixes and prefixes to a greater extent. </p> <p>2) When a user enters a query, look up the corresponding lists, and merge them. If it's a strict boolean query, the process is pretty straightforward -- for AND, a docid has to occur in all the word lists, for OR, in at least one wordlist, etc.</p> <p>3) If you want to rank your results, there are a number of ways to do that, but the basic idea is to use the frequency with which a word occurs in a document, as compared to the frequency you expect it to occur in any document in the corpus, as a signal that the document is more or less relevant. See textbook.</p> <p>4) You can also store word positions to infer phrases, etc.</p> <p>Most of that is irrelevant for desktop search, as you are more interested in recall (all documents that include the term) than ranking.</p> <hr> <p>¹ previously on <a href="http://www-csli.stanford.edu/~hinrich/information-retrieval-book.html" rel="noreferrer">http://www-csli.stanford.edu/~hinrich/information-retrieval-book.html</a>, accessible via wayback machine</p>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    3. VO
      singulars
      1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload