Databases
StackOverflow2013
Postsdbo
POSolr query: stop words, OR and AND weirdness
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POSolr query: stop words, OR and AND weirdness
    primarykey
    data
    text
    <p>We are using Solr 3.5 with schema that has the following field declaration:</p> <pre><code>&lt;fieldType name="fieldN" class="solr.TextField" positionIncrementGap="100"&gt; &lt;analyzer type="index"&gt; &lt;tokenizer class="solr.WhitespaceTokenizerFactory"/&gt; &lt;filter class="solr.WordDelimiterFilterFactory" generateWordParts="0" generateNumberParts="0" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1" splitOnNumerics="0" preserveOriginal="1"/&gt; &lt;filter class="solr.LengthFilterFactory" min="2" max="256"/&gt; &lt;filter class="solr.LowerCaseFilterFactory"/&gt; &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /&gt; &lt;filter class="solr.PorterStemFilterFactory"/&gt; &lt;/analyzer&gt; &lt;analyzer type="query"&gt; &lt;tokenizer class="solr.WhitespaceTokenizerFactory"/&gt; &lt;filter class="solr.LengthFilterFactory" min="2" max="256"/&gt; &lt;filter class="solr.LowerCaseFilterFactory"/&gt; &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /&gt; &lt;filter class="solr.PorterStemFilterFactory"/&gt; &lt;/analyzer&gt; &lt;/fieldType&gt; </code></pre> <p>When we send a query like this:</p> <pre><code>field1:"term1" </code></pre> <p>Solr returns results.</p> <p>When we run this query we still get results:</p> <pre><code>field1:"term1" AND (field2:term2 OR field3:term2) </code></pre> <p>While <em>term2</em> is a stop word and <em>term1</em> is a regular word.</p> <p>But when we send a query like this:</p> <pre><code>field1:"term1" AND (field2:term2 OR field3:term2 OR field4:term2) </code></pre> <p>Nothing returns.</p> <p>We also noticed that when we do something like:</p> <pre><code>(field1:"term1" AND (field2:term2 OR field3:term2)) OR (field1:"term1" AND field4:term2) </code></pre> <p>works too, but as the real query should search for one term in about 200 fields, this option is less preferred.</p> <p>Thanks.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USNoam
    1. USNoam
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POSolr query: stop words, OR and AND weirdness
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POSolr query: stop words, OR and AND weirdness
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POSolr query: stop words, OR and AND weirdness
      1. This table or related slice is empty.
      1. VTUpMod
    1. COAnd what is your expected result? Since term2 is a stopword, shouldn't you expect no results for both the 2nd and 3rd queries? Anyway, the first step you should take is to inspect your index with Luke, just to be sure what exactly you are querying against.
      singulars
      1. POSolr query: stop words, OR and AND weirdness
      1. USMarko Topolnik
    2. COI was expecting to that the stop word part would not affect the results, query like _term1_ _term2_ should return all the documents that match _term1_ when _term2_ is a stop word. I will try the tests, thanks.
      singulars
      1. POSolr query: stop words, OR and AND weirdness
      1. USNoam
    3. COYes, analyzing a stopword produces no tokens so the entire query term should be as if not there. But I think this presents a challenge to the QueryParser. Your 2nd query is a BooleanQuery of two clauses where the right clause is an inner BooleanQuery. This inner query turns out to have no clauses if term2 is a stopword, so Lucene is left with an empty BooleanQuery. I wonder how it handles that. There was (a long time ago, but still) a JIRA [issue](https://issues.apache.org/jira/browse/LUCENE-933) about exactly this case.
      singulars
      1. POSolr query: stop words, OR and AND weirdness
      1. USMarko Topolnik
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload