Databases
StackOverflow2013
Postsdbo
POSolr returning poor and invalid spelling suggestions
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POSolr returning poor and invalid spelling suggestions
    primarykey
    data
    text
    <p>I'm trying to find a way to improve the spelling suggestions offered by Solr while still retaining as much of the good suggestions as possible. One particular search is causing me s a massive headache:</p> <p>"yellow flowers"</p> <p>When searching for this Solr offers the suggestions "yellow flowersit"</p> <p>As far as I can tell this is coming from a section in the field used to generation the suggestions that looks like "flowers. It"</p> <p>This is only one of many iterations of issues I've had with this particular query (I'm sure their are plenty of others, but I'm focusing on resolving this one). Previously I had issues with ellipses concatenating words such as:</p> <p>"flowers...a" => "yellow flowersa"</p> <p>I've tired various things like adding spellcheck.accuracy to the query, but I found this excluded valid suggestions before it excluded the invalid ones (it excluded "acet" => "acer" at 0.7, but didn't exclude the "flowerse" until 0.9). I've tried onlyMorePopular to both true and false, neither had an appreciable effect.</p> <p>I'm considering trying to use thresholdTokenFrequency, but I have a feeling I would lose a lot of valid suggestions for things like proper names that might only appear in the index once.</p> <p>Along the sames lines: searching for "flowerse" results in suggestions like "flowersse" rather than a more appropriate "flowers" suggestion.</p> <p>What can I do to improve the results of spelling suggestions, or even just list words to never be suggested or similar?</p> <p>Why is Solr suggesting such oddball word combinations that can't appear more than a couple times when the search term is one of the most common words in my index?</p> <p>text fieldType definition:</p> <pre><code>&lt;fieldType name="text" class="solr.TextField" positionIncrementGap="100"&gt; &lt;analyzer type="index"&gt; &lt;charFilter class="solr.PatternReplaceCharFilterFactory" pattern="(…|[\.]{2,})" replacement=" " /&gt; &lt;tokenizer class="solr.WhitespaceTokenizerFactory"/&gt; &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/&gt; &lt;filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0"/&gt; &lt;filter class="solr.LowerCaseFilterFactory"/&gt; &lt;filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/&gt; &lt;filter class="solr.RemoveDuplicatesTokenFilterFactory"/&gt; &lt;/analyzer&gt; &lt;analyzer type="query"&gt; &lt;tokenizer class="solr.WhitespaceTokenizerFactory"/&gt; &lt;filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/&gt; &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/&gt; &lt;filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0"/&gt; &lt;filter class="solr.LowerCaseFilterFactory"/&gt; &lt;filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/&gt; &lt;filter class="solr.RemoveDuplicatesTokenFilterFactory"/&gt; &lt;/analyzer&gt; &lt;/fieldType&gt; </code></pre> <p>Suggestions field definition:</p> <p><code>&lt;field name="suggestions" type="text" indexed="true" stored="true" multiValued="true" termVectors="true" termPositions="true" termOffsets="true" /&gt;</code></p> <p>requestHandler:</p> <pre><code>&lt;requestHandler name="standard" class="solr.SearchHandler" default="true"&gt; &lt;lst name="defaults"&gt; &lt;str name="echoParams"&gt;explicit&lt;/str&gt; &lt;str name="spellcheck.onlyMorePopular"&gt;false&lt;/str&gt; &lt;str name="spellcheck.extendedResults"&gt;false&lt;/str&gt; &lt;str name="spellcheck.count"&gt;1&lt;/str&gt; &lt;/lst&gt; &lt;arr name="last-components"&gt; &lt;str&gt;spellcheck&lt;/str&gt; &lt;/arr&gt; &lt;/requestHandler&gt; </code></pre>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USJohn
    1. USJohn
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POSolr returning poor and invalid spelling suggestions
      1. This table or related slice is empty.
      1. VTUpMod
    1. COCan you please add the fieldType definition from your schema.xml for the field that the suggestions are being derived from as well as your spelling requestHandler entry from your solrconfig.xml?
      singulars
      1. POSolr returning poor and invalid spelling suggestions
      1. USPaige Cook
    2. COThanks, done. I also added the field definition
      singulars
      1. POSolr returning poor and invalid spelling suggestions
      1. USJohn
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload