Databases
StackOverflow2013
Postsdbo
POLong queries on very short documents
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POLong queries on very short documents
    primarykey
    data
    text
    <p>I'm fresh out of the nursery as far as Lucene/Solr are concerned, so I may be trying to utilize it completely wrong, but I hope someone can point me in the right direction.</p> <p>My documents (less than 3,000) are short statements from a taxonomy. All are single sentences, with some having no more than 4-6 words long. There is only one field for each document, so searching across multiple fields is not a route I would be looking into. What I would like to do is query the contents of a work related document and have the taxonomy statements that are relevant returned.</p> <p>Currently I am using the default example setup that came with Solr with added verb synonyms from Wordnet since performed actions are what I am trying to identify (i.e. taxonomy statement of 'Alter garments to specifications'). </p> <p>Basic word matching works as expected, but I would like to make things somewhat more sophisticated. Since the queries are so long I never end up with a high relevancy scores when searching against the tiny documents. I'm sure this can be resolved by normalizing scores in some fashion so I am not real concerned about the scores coming out, but the actual statements (documents) that are being identified.</p> <p>Would I be better off indexing the documents (currently the long queries) on the fly and querying each taxonomy statement and compiling/sorting the results or can I perform these long queries on the tiny documents effectively in some other fashion? I presume this may present it's own difficulties.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USIOInterrupt
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. COI refered the wikipedia entry on Taxonomy, and I still dont understand your question. Please include a few sample documents and queries and how they are supposed to match. What kind of analysis do you want to perform of them?
      singulars
      1. POLong queries on very short documents
      1. USaitchnyu
    2. CO3000 documents is a very small number anyway. A python script storing it all in memory would take a few MBs of RAM, take no time processing it, and store/retrieve it from disk in a blink. So you can implement any required algorithm yourself. Identifying "performed actions" seems a job for a "part of speech" algorithm (noun, verb etc). Data minng packages like Mahout can do that.
      singulars
      1. POLong queries on very short documents
      1. USaitchnyu
    3. CO@aitchnyu - Unfortunately I do not know anything about SA or LSI to be able to write my own algorithms. I was hoping I could utilize something like Solr to get me a jump start here. I need to know specifically if a certain statement(s) exists within a document. The wording could be completely different (synonyms, etc), but I need to be able to catch it. I am sure this stuff can be done with algorithms and POS tagging, but I don't know how to do it. Additionally I'm not very good with Java at all so that hinders me with some tools.
      singulars
      1. POLong queries on very short documents
      1. USIOInterrupt
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload