Databases
StackOverflow2013
Postsdbo
POLucene search query using a IntField not working after document update
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POLucene search query using a IntField not working after document update
    primarykey
    data
    text
    <p>I'm trying to run a simple query on a set of two documents in Lucene using their id and the value of an <code>Intfield</code>. The query correctly returns both fields right after adding them. Now I use the retrieved document and make a change to the <code>CONTEXT_FIELD</code> (which is not used in the query) and update the document in the index. </p> <p>Interestingly, now the search does not return any results, neither the old nor the new document. If I only use the <code>METHOD_NAME</code> field in the query, everything works as expected, the problems seems to be the <code>NUMBER_OF_ARGUMENTS</code> <code>IntField</code>.</p> <p><em>Why is this happening?</em></p> <p>Sample Code:</p> <pre><code>import java.io.File; import java.io.IOException; import java.util.HashMap; import java.util.Map; import org.apache.lucene.analysis.Analyzer; import org.apache.lucene.analysis.core.KeywordAnalyzer; import org.apache.lucene.analysis.core.SimpleAnalyzer; import org.apache.lucene.analysis.miscellaneous.PerFieldAnalyzerWrapper; import org.apache.lucene.document.Document; import org.apache.lucene.document.Field; import org.apache.lucene.document.IntField; import org.apache.lucene.document.TextField; import org.apache.lucene.index.DirectoryReader; import org.apache.lucene.index.IndexWriter; import org.apache.lucene.index.IndexWriterConfig; import org.apache.lucene.index.IndexWriterConfig.OpenMode; import org.apache.lucene.index.Term; import org.apache.lucene.queryparser.classic.ParseException; import org.apache.lucene.queryparser.classic.QueryParser; import org.apache.lucene.search.BooleanClause.Occur; import org.apache.lucene.search.BooleanQuery; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.NumericRangeQuery; import org.apache.lucene.search.Query; import org.apache.lucene.search.TopDocs; import org.apache.lucene.store.FSDirectory; import org.apache.lucene.util.Version; public class LuceneDemo { private static final String ID1 = "Great#text"; private static final String ID2 = "Another#bonus"; private static final String METHOD_NAME_FIELD = "method_name"; private static final String NUMBER_OF_ARGUMENTS = "number_of_arguments"; private static final String CONTEXT_FIELD = "context"; /** Parser used to parse queries */ private static QueryParser parser = new QueryParser(Version.LUCENE_43, METHOD_NAME_FIELD, createDefaultAnalyzer()); public static void main(String[] args) throws IOException, ParseException { IndexWriter luceneIndexWriter = new IndexWriter( FSDirectory.open(new File("/tmp/test")), createWriterConfig(64)); Document doc1 = createDocument(ID1, "context1", 1); luceneIndexWriter.addDocument(doc1); Document doc2 = createDocument(ID2, "context2", 2); luceneIndexWriter.addDocument(doc2); System.out.println("Found doc1: " + findDocument(ID1, 1, luceneIndexWriter)); System.out.println("Found doc2: " + findDocument(ID2, 2, luceneIndexWriter)); doc1 = findDocument(ID1, 1, luceneIndexWriter); // Section 1 doc1.removeField(CONTEXT_FIELD); doc1.add(new TextField(CONTEXT_FIELD, "context1_changed", Field.Store.YES)); luceneIndexWriter.updateDocument(new Term(METHOD_NAME_FIELD, "text"), doc1); System.out.println("Found doc1: " + findDocument(ID1, 1, luceneIndexWriter)); System.out.println("Found doc2: " + findDocument(ID2, 2, luceneIndexWriter)); // Section 2 // doc1 = findDocument(ID1, 1, luceneIndexWriter); &lt;- null doc1.removeField(CONTEXT_FIELD); doc1.add(new TextField(CONTEXT_FIELD, "context1_changed2", Field.Store.YES)); luceneIndexWriter.updateDocument(new Term(METHOD_NAME_FIELD, "text"), doc1); System.out.println("Found doc1: " + findDocument(ID1, 1, luceneIndexWriter)); System.out.println("Found doc2: " + findDocument(ID2, 2, luceneIndexWriter)); luceneIndexWriter.close(); } private static Document createDocument(String id, String context, int value) { Document doc = new Document(); doc.add(new TextField(METHOD_NAME_FIELD, id, Field.Store.YES)); doc.add(new TextField(CONTEXT_FIELD, context, Field.Store.YES)); doc.add(new IntField(NUMBER_OF_ARGUMENTS, value, Field.Store.YES)); return doc; } private static Document findDocument(String id, int value, IndexWriter luceneIndexWriter) throws IOException, ParseException { DirectoryReader reader = DirectoryReader.open(luceneIndexWriter, true); IndexSearcher searcher = new IndexSearcher(reader); String[] split = id.split("#"); Query methodQuery = parser.parse(split[1]); Query classQuery = parser.parse(split[0]); NumericRangeQuery&lt;Integer&gt; range = NumericRangeQuery.newIntRange( NUMBER_OF_ARGUMENTS, 1, value, value, true, true); BooleanQuery query = new BooleanQuery(); query.add(methodQuery, Occur.MUST); query.add(classQuery, Occur.MUST); query.add(range, Occur.MUST); TopDocs result = searcher.search(query, 1); if (result.totalHits == 0) { System.err.println("Problem, nothing found (Method: " + id + ")"); return null; } Document document = searcher.doc(result.scoreDocs[0].doc); if (document.get(METHOD_NAME_FIELD).equals(id)) { return document; } return null; } /** create the analyzer used */ private static Analyzer createDefaultAnalyzer() { Map&lt;String, Analyzer&gt; analyzerPerField = new HashMap&lt;String, Analyzer&gt;(); analyzerPerField.put(NUMBER_OF_ARGUMENTS, new KeywordAnalyzer()); PerFieldAnalyzerWrapper analyzer = new PerFieldAnalyzerWrapper( new SimpleAnalyzer(Version.LUCENE_43), analyzerPerField); return analyzer; } /** Creates the configuration used for writing. */ public static IndexWriterConfig createWriterConfig(double ramBufferSizeMB) { IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_43, createDefaultAnalyzer()); config.setRAMBufferSizeMB(ramBufferSizeMB); config.setOpenMode(OpenMode.CREATE_OR_APPEND); config.setMaxBufferedDeleteTerms(1); // desperate try at config.setMaxBufferedDocs(2); // storing everything correctly right away // config.setInfoStream(System.out); &lt;- set this for more output return config; } } </code></pre> <p>Output:</p> <pre><code>Found doc1: Document&lt;stored,indexed,tokenized&lt;method_name:Great#text&gt; stored,indexed,tokenized&lt;context:context1&gt; stored&lt;number_of_arguments:1&gt;&gt; Found doc2: Document&lt;stored,indexed,tokenized&lt;method_name:Another#bonus&gt; stored,indexed,tokenized&lt;context:context2&gt; stored&lt;number_of_arguments:2&gt;&gt; Problem, nothing found (Method: Great#text) Found doc1: null Found doc2: Document&lt;stored,indexed,tokenized&lt;method_name:Another#bonus&gt; stored,indexed,tokenized&lt;context:context2&gt; stored&lt;number_of_arguments:2&gt;&gt; Problem, nothing found (Method: Great#text) Found doc1: null Found doc2: Document&lt;stored,indexed,tokenized&lt;method_name:Another#bonus&gt; stored,indexed,tokenized&lt;context:context2&gt; stored&lt;number_of_arguments:2&gt;&gt; </code></pre> <p>Output including config.setInfoStream(System.out): </p> <p><a href="http://bpaste.net/show/ko8kkxeFxZFE26NuecZc/" rel="nofollow">http://bpaste.net/show/ko8kkxeFxZFE26NuecZc/</a> (Too long for inclusion here, sorry)</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USNihathrael
    1. USNihathrael
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POLucene search query using a IntField not working after document update
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POLucene search query using a IntField not working after document update
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POLucene search query using a IntField not working after document update
      1. USAvner Levy
      1. VTFavorite
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload