Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>When you index a document, several steps are performed :</p> <ul> <li>the document is analyzed,</li> <li>data is put in the RAM buffer,</li> <li>when the RAM buffer is full, data is flushed to a new segment on disk,</li> <li>if there are more than ${mergeFactor} segments, segments are merged.</li> </ul> <p>The first two steps will be run in as many threads as you have clients sending data to Solr, so if you want Solr to run three threads for these steps, all you need is to send data to Solr from three threads.</p> <p>You can configure the number of threads to use for the fourth step if you use a ConcurrentMergeScheduler (http://lucene.apache.org/java/3_0_1/api/core/org/apache/lucene/index/ConcurrentMergeScheduler.html). However, there is no mean to configure the maximum number of threads to use from Solr configuration files, so what you need is to write a custom class which call setMaxThreadCount in the constructor.</p> <p>My experience is that the main ways to improve indexing speed with Solr are :</p> <ul> <li>buying faster hardware (especially I/O),</li> <li>sending data to Solr from several threads (as many threads as cores is a good start),</li> <li>using the Javabin format,</li> <li>using faster analyzers.</li> </ul> <p>Although <a href="http://lucene.apache.org/solr/api/org/apache/solr/client/solrj/impl/StreamingUpdateSolrServer.html">StreamingUpdateSolrServer</a> looks interesting for improving indexing performance, <a href="https://issues.apache.org/jira/browse/SOLR-1565">it doesn't support the Javabin format</a>. Since Javabin parsing is <em>much</em> faster than XML parsing, I got better performance by sending bulk updates (800 in my case, but with rather small documents) using CommonsHttpSolrServer and the Javabin format.</p> <p>You can read <a href="http://wiki.apache.org/lucene-java/ImproveIndexingSpeed">http://wiki.apache.org/lucene-java/ImproveIndexingSpeed</a> for further information.</p>
    singulars
    1. This table or related slice is empty.
    1. POSpeeding up Solr Indexing
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USjpountz
    1. USjpountz
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COThanks much for the reply. I tried your first suggestion(posting documents via multiple threads) before posting this question. I did not get any big improvement. That's why I wanted to know if there was something else that I could try. Having said that there I was posting docs(actually was streaming) using multiple threads but to a single core. I was doubting if that essentially was posting sequentially and hence had no big improvement. Also I was constructing the docs being posted from a CSV file,if that makes any difference.
      singulars
      1. PO
      1. USphanips
    2. COXML deserialization can make the indexing process very slow. I updated my answer to add the advice to use the Javabin format.
      singulars
      1. PO
      1. USjpountz
    3. COAlso, StreamingUpdateSolrServer does now support Javabin as well.
      singulars
      1. PO
      1. USEric Pugh
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload