Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>What you want is called multi-label classification. Scikits-learn can do that. See here: <a href="http://scikit-learn.org/dev/modules/multiclass.html" rel="noreferrer">http://scikit-learn.org/dev/modules/multiclass.html</a>.</p> <p>I'm not sure what's going wrong in your example, my version of sklearn apparently doesn't have WordNGramAnalyzer. Perhaps it's a question of using more training examples or trying a different classifier? Though note that the multi-label classifier expects the target to be a list of tuples/lists of labels.</p> <p>The following works for me:</p> <pre><code>import numpy as np from sklearn.pipeline import Pipeline from sklearn.feature_extraction.text import CountVectorizer from sklearn.svm import LinearSVC from sklearn.feature_extraction.text import TfidfTransformer from sklearn.multiclass import OneVsRestClassifier X_train = np.array(["new york is a hell of a town", "new york was originally dutch", "the big apple is great", "new york is also called the big apple", "nyc is nice", "people abbreviate new york city as nyc", "the capital of great britain is london", "london is in the uk", "london is in england", "london is in great britain", "it rains a lot in london", "london hosts the british museum", "new york is great and so is london", "i like london better than new york"]) y_train = [[0],[0],[0],[0],[0],[0],[1],[1],[1],[1],[1],[1],[0,1],[0,1]] X_test = np.array(['nice day in nyc', 'welcome to london', 'hello welcome to new york. enjoy it here and london too']) target_names = ['New York', 'London'] classifier = Pipeline([ ('vectorizer', CountVectorizer(min_n=1,max_n=2)), ('tfidf', TfidfTransformer()), ('clf', OneVsRestClassifier(LinearSVC()))]) classifier.fit(X_train, y_train) predicted = classifier.predict(X_test) for item, labels in zip(X_test, predicted): print '%s =&gt; %s' % (item, ', '.join(target_names[x] for x in labels)) </code></pre> <p>For me, this produces the output:</p> <pre><code>nice day in nyc =&gt; New York welcome to london =&gt; London hello welcome to new york. enjoy it here and london too =&gt; New York, London </code></pre> <p>Hope this helps.</p>
    singulars
    1. This table or related slice is empty.
    1. POUse scikit-learn to classify into multiple categories
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USCerin
    1. USmwv
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POUse scikit-learn to classify into multiple categories
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COHi, I tried using the example from multiclass but I still can't get multiple labels. I have updated my question with my code sample. What am I doing wrong? Thanks!
      singulars
      1. PO
      1. USCodeMonkeyB
    2. COHi thanks for your example. The problem is that i have training data for thousands of cities (with possible spellings names etc) To make it work as in the example i would have to create training sets for combinations of all possible city names. Is it possible instead to some how get the the probabilities of labels? For example for the last text the probability its new york is X% and probability its London is Y% sorted by probabilities. And then based on some sort of threshold I can grab labels with a certain probability or higher?
      singulars
      1. PO
      1. USCodeMonkeyB
    3. COI don't think you need training data with all combinations of city names, I just added in the last two examples in `X_train` to make it more clear what `y_train` should look like. Under the hood, `OneVsRestClassifier` trains a separate classifier for each class, so you should be able to get the same results without training examples that combine city names. You can get the probability that a datapoint belongs to a class by calling `predict_proba` on a fitted classifier. This only works for appropriate classifiers.
      singulars
      1. PO
      1. USmwv
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload