Databases
StackOverflow2013
Postsdbo
POWhat makes a good autowarming query in Solr and how do they work?
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhat makes a good autowarming query in Solr and how do they work?
    text
    copied!<p>This question is a follow up to <a href="https://stackoverflow.com/questions/5154093/solr-requests-time-out-during-index-update-perhaps-replication-a-possible-soluti">this question</a> about infrequent, isolated read timeouts in a solr installation.</p> <p>As a possible problem missing / bad autowarming queries for new searchers were found.</p> <p>Now I am confused about how good autowarming queries should "look like".</p> <p>I read up but couldnt find any good documentation on this.</p> <p>Should they hit a lot of documents in the index? Or should they have matches in all distinct fields that exist in the index?</p> <p>Wouldnt just <code>*:*</code> be the best autowarming query or why not?</p> <p>The example solr config has theese sample queries in it:</p> <pre><code>&lt;lst&gt;&lt;str name="q"&gt;solr&lt;/str&gt; &lt;str name="start"&gt;0&lt;/str&gt; &lt;str name="rows"&gt;10&lt;/str&gt;&lt;/lst&gt; &lt;lst&gt;&lt;str name="q"&gt;rocks&lt;/str&gt; &lt;str name="start"&gt;0&lt;/str&gt; &lt;str name="rows"&gt;10&lt;/str&gt;&lt;/lst&gt; </code></pre> <p>I changed them to:</p> <pre><code>&lt;lst&gt;&lt;str name="q"&gt;george&lt;/str&gt; &lt;str name="start"&gt;0&lt;/str&gt; &lt;str name="rows"&gt;10&lt;/str&gt;&lt;/lst&gt; </code></pre> <p>Why? Because the index holds film entities with fields for titles and actors. Those are the most searched ones. And george appears in titles and actors.</p> <p>I don't really know whether this makes sense. So my question is:</p> <ul> <li>What would be good autowarming queries for my index and why?</li> <li>What makes a good autowarming query? </li> </ul> <p>This is an example document from the index. The index has about 70,000 documents and they all look like this (only different values of course): example document:</p> <pre><code> &lt;doc&gt; &lt;arr name="actor"&gt;&lt;str&gt;Tommy Lee Jones&lt;/str&gt;&lt;str&gt;Will Smith&lt;/str&gt;&lt;str&gt;Rip Torn&lt;/str&gt; &lt;str&gt;Lara Flynn Boyle&lt;/str&gt;&lt;str&gt;Johnny Knoxville&lt;/str&gt;&lt;str&gt;Rosario Dawson&lt;/str&gt;&lt;str&gt;Tony Shalhoub&lt;/str&gt; &lt;str&gt;Patrick Warburton&lt;/str&gt;&lt;str&gt;Jack Kehler&lt;/str&gt;&lt;str&gt;David Cross&lt;/str&gt;&lt;str&gt;Colombe Jacobsen-Derstine&lt;/str&gt; &lt;str&gt;Peter Spellos&lt;/str&gt;&lt;str&gt;Michael Rivkin&lt;/str&gt;&lt;str&gt;Michael Bailey Smith&lt;/str&gt;&lt;str&gt;Lenny Venito&lt;/str&gt; &lt;str&gt;Howard Spiegel&lt;/str&gt;&lt;str&gt;Alpheus Merchant&lt;/str&gt;&lt;str&gt;Jay Johnston&lt;/str&gt;&lt;str&gt;Joel McKinnon Miller&lt;/str&gt; &lt;str&gt;Derek Cecil&lt;/str&gt;&lt;/arr&gt; &lt;arr name="affiliate"&gt;&lt;str&gt;amazon&lt;/str&gt;&lt;/arr&gt; &lt;arr name="aka_title"&gt;&lt;str&gt;Men in Black II&lt;/str&gt;&lt;str&gt;MIB 2&lt;/str&gt;&lt;str&gt;MIIB&lt;/str&gt; &lt;str&gt;Men in Black 2&lt;/str&gt;&lt;str&gt;Men in black II (Hombres de negro II)&lt;/str&gt;&lt;str&gt;Hombres de negro II&lt;/str&gt;&lt;str&gt;Hommes en noir II&lt;/str&gt;&lt;/arr&gt; &lt;bool name="blockbuster"&gt;false&lt;/bool&gt; &lt;arr name="country"&gt;&lt;str&gt;US&lt;/str&gt;&lt;/arr&gt; &lt;str name="description"&gt;Agent J (Will Smith) muss die Erde wieder vor einigem Abschaum bewahren, denn in Gestalt des verführerischen Dessous-Models Serleena (Lara Flynn Boyle) will ein Alien den Planeten unterjochen. Dabei benötigt J die Hilfe seines alten Partners Agent K (Tommy Lee Jones). Der wurde aber bei seiner "Entlassung" geblitzdingst, und so muß J seine Erinnerung erst mal etwas auffrischen bevor es auf die Jagd gehen kann.&lt;/str&gt; &lt;arr name="director"&gt;&lt;str&gt;Barry Sonnenfeld&lt;/str&gt;&lt;/arr&gt; &lt;int name="film_id"&gt;120912&lt;/int&gt; &lt;arr name="genre"&gt;&lt;str&gt;Action&lt;/str&gt;&lt;str&gt;Komödie&lt;/str&gt;&lt;str&gt;Science Fiction&lt;/str&gt;&lt;/arr&gt; &lt;str name="id"&gt;120912&lt;/str&gt; &lt;str name="image_url"&gt;/media/search/filmcovers/105x/kf/false/F6Q1XW.jpg&lt;/str&gt; &lt;int name="imdb_id"&gt;120912&lt;/int&gt; &lt;date name="last_modified"&gt;2011-03-01T18:51:35.903Z&lt;/date&gt; &lt;str name="locale_title"&gt;Men in Black II&lt;/str&gt; &lt;int name="malus"&gt;3238&lt;/int&gt; &lt;int name="parent_id"&gt;0&lt;/int&gt; &lt;arr name="product_dvd"&gt;&lt;str&gt;amazon&lt;/str&gt;&lt;/arr&gt; &lt;arr name="product_type"&gt;&lt;str&gt;dvd&lt;/str&gt;&lt;/arr&gt; &lt;int name="rating"&gt;49&lt;/int&gt; &lt;str name="sort_title"&gt;meninblack&lt;/str&gt; &lt;int name="type"&gt;1&lt;/int&gt; &lt;str name="url"&gt;/film/Men-in-Black-II-Barry-Sonnenfeld-Tommy-Lee-Jones-F6Q1XW/&lt;/str&gt; &lt;int name="year"&gt;2002&lt;/int&gt; &lt;/doc&gt; </code></pre> <p>Most queries are exact match queries on actor fields with some filters in place.</p> <p>Example:</p> <blockquote> <p>INFO: [] webapp=/solr path=/select/ params={facet=true&amp;sort=score+asc,+malus+asc,+year+desc&amp;hl.simple.pre=<strong>starthl</strong>&amp;hl=true&amp;version=2.2&amp;fl=*,score&amp;facet.query=year:[1900+TO+1950]&amp;facet.query=year:[1951+TO+1980]&amp;facet.query=year:[1981+TO+1990]&amp;facet.query=year:[1991+TO+2000]&amp;facet.query=year:[2001+TO+2011]&amp;bf=div(sub(10000,malus),100)^10&amp;hl.simple.post=<strong>endhl</strong>&amp;facet.field=genre&amp;facet.field=country&amp;facet.field=blockbuster&amp;facet.field=affiliate&amp;facet.field=product_type&amp;qs=5&amp;qt=dismax&amp;hl.fragsize=200&amp;mm=2&amp;facet.mincount=1&amp;qf=actor^0.1&amp;f.blockbuster.facet.mincount=0&amp;f.genre.facet.limit=20&amp;hl.fl=actor&amp;wt=json&amp;f.affiliate.facet.mincount=1&amp;f.country.facet.limit=20&amp;rows=10&amp;pf=actor^5&amp;start=0&amp;q="Josi+Kleinpeter"&amp;ps=3} hits=1 status=0 QTime=4</p> </blockquote>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload