StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POFieldCache with frequently updating index
text
Body
copied!<p>Hi<br/> I have lucene index that is frequently updating with new records, I have 5,000,000 records in my index and I'm caching one of my numeric fields using FieldCache. but after updating index it takes time to reload the FieldCache again (im reloading the cache cause documentation said DocID is not reliable) so how can I minimize this overhead by adding only newly added DocIDs to the FieldCache, cause this capability turns to bottleneck in my application.</p> <pre><code> IndexReader reader = IndexReader.Open(diskDir); int[] dateArr = FieldCache_Fields.DEFAULT.GetInts(reader, "newsdate"); // This line takes 4 seconds to load the array dateArr = FieldCache_Fields.DEFAULT.GetInts(reader, "newsdate"); // this line takes 0 second as we expected // HERE we add some document to index and we need to reload the index to reflect changes reader = reader.Reopen(); dateArr = FieldCache_Fields.DEFAULT.GetInts(reader, "newsdate"); // This takes 4 second again to load the array </code></pre> <p>I want a mechanism that minimize this time by adding only newly added documents to the index in our array there is a technique like this <a href="http://invertedindex.blogspot.com/2009/04/lucene-dociduid-mapping-and-payload.html" rel="nofollow">http://invertedindex.blogspot.com/2009/04/lucene-dociduid-mapping-and-payload.html</a> to improve the performance but it still load all documents that we already have and i think there is no need to reload them all if we find a way to only adding newly added documents to the array</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload