StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>It's possible you are hitting the limits of your hardware, but there are a few things you can do to your query first to help optimize it. </p> <p><strong>Max Expansions</strong></p> <p>The first thing I would do is limit <code>max_expansions</code>. The way the prefix-queries work is by generating a list of prefixes that match the last token in your query. In your search query "some search term", the last token "term" would be expanded using "term" as the prefix seed. You may generate a list like this:</p> <ul> <li>term</li> <li>terms</li> <li>terminate</li> <li>terminator</li> <li>termite</li> </ul> <p>The prefix expansion process runs through your posting list looking for any word which matches the seed prefix. By default, this list is unbounded, which means you can generate a very large list of expansions. </p> <p>The second phase rewrites your original query into a series of <code>term</code> queries using the expansions. The bigger the expansion list, the more terms are evaluated against your index and a corresponding decrease in speed.</p> <p>If you limit the expansion process to something reasonable, you can maintain speed and still usually get good prefix matching:</p> <pre><code>{ "query" : { "multi_match" : { "query" : "some search term", "fields" : [ "title", "content" ], "type": "phrase_prefix", "max_expansions" : 100 } }, "size": 20, "fields" :["article_id", "feed_id"], } </code></pre> <p>You'll have to play with how many expansions you want. It is a tradeoff between speed and recall.</p> <p><strong>Filtering</strong></p> <p>In general, the other thing you can add is filtering. If there is some type of criteria you can filter on, you can potentially drastically improve speed. Currently, your query is executing against the entire index (250m documents), which is a lot to evaluate. If you can add filter that cuts that number down, you can see much improved latency.</p> <p>At the end of the day, the fewer documents which the query evaluates, the faster the query will run. Filters decrease the number of docs that a query will see, are cached, operate very quickly, etc etc.</p> <p>Your situation may not have any applicable filters, but if it does, they can really help!</p> <p><strong>File System Caching</strong></p> <p>This advice is entirely dependent on the rest of the system. If you aren't fully utilizing your heap (24gb) because you are doing simple search and filtering (e.g. not faceting / geo / heavy sorts / scripts) you may be able to reallocate your heap to the file system cache.</p> <p>For example, if your max heap usage peaks at 12gb, it may make sense to decrease heap size down to 15gb. The extra 10gb that you freed will go back to the OS and help cache segments, which will help boost search performance simply by the fact that more operations are diskless.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload