Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <h2>Packages</h2> <p>If you're not committed to neural networks, I've had good luck with SVM, and k-means clustering might also be helpful. Both of these are provided by <a href="http://luispedro.org/software/milk" rel="nofollow">Milk</a>. It also does Stepwise Discriminant Analysis for <a href="http://en.wikipedia.org/wiki/Feature_selection" rel="nofollow">feature selection</a>, which will definitely be useful to you if you're trying to find similar documents by topic.</p> <p>God help you if you choose this route, but the ROOT framework has a powerful machine learning package called <a href="http://tmva.sourceforge.net/#exec_summary" rel="nofollow">TMVA</a> that provides a large number of classification methods, including SVM, NN, and Boosted Decision Trees (also possibly a good option). I haven't used it, but pyROOT provides python bindings to ROOT functionality. To be fair, when I first used ROOT I had no C++ knowledge and was in over my head conceptually too, so this might actually be amazing for you. ROOT has a HUGE number of data processing tools.</p> <p>(NB: I've also written a fairly accurate document language identifier using chi-squared feature selection and cosine matching. Obviously your problem is harder, but consider that you might not need very hefty tools for it.)</p> <h2>Storage vs Processing</h2> <p>You mention in your question that:</p> <blockquote> <p>...articles would be fetched based on their tags and passed through a neural network for ranking.</p> </blockquote> <p>Just as another NB, one thing you should know about machine learning is that processes like training and evaluating tend to take a <strong>while</strong>. You should probably consider ranking all documents for each tag only once (assuming you know all the tags) and storing the results. For machine learning generally, it's much better to use more storage than more processing. </p> <p>Now to your specific case. You don't say how many tags you have, so let's assume you have 1000, for roundness. If you store the results of your ranking for each doc on each tag, that gives you 100 million floats to store. That's a lot of data, and calculating them all will take a while, but retrieving them is very fast. If instead you recalculate the ranking for each document on demand, you have to do 1000 passes of it, one for each tag. Depending on the kind of operations you're doing and the size of your docs, that could take a few seconds to a few minutes. If the process is simple enough that you can wait for your code to do several of these evaluations on demand without getting bored, then go for it, but you should time this process <em>before</em> making any design decisions / writing code you won't want to use.</p> <p>Good luck!</p>
    singulars
    1. This table or related slice is empty.
    1. PONeural Network based ranking of documents
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USandronikus
    1. USandronikus
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload