StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POFaster way to count number of string occurrences in a numpy array python
text
Body
copied!<p>I have a numpy array of tuples:</p> <pre><code>trainY = np.array([('php', 'image-processing', 'file-upload', 'upload', 'mime-types'), ('firefox',), ('r', 'matlab', 'machine-learning'), ('c#', 'url', 'encoding'), ('php', 'api', 'file-get-contents'), ('proxy', 'active-directory', 'jmeter'), ('core-plot',), ('c#', 'asp.net', 'windows-phone-7'), ('.net', 'javascript', 'code-generation'), ('sql', 'variables', 'parameters', 'procedure', 'calls')], dtype=object) </code></pre> <p>I am given list of indices which subsets this np.array:</p> <pre><code>x = [0, 4] </code></pre> <p>and a string:</p> <pre><code>label = 'php' </code></pre> <p>I want to count the number of times the label <code>'php'</code> occurs in this subset of the np.array. In this case, the answer would be 2. </p> <p>Notes:</p> <p>1) A label will only appear at most ONCE in a tuple and </p> <p>2) The tuple can have length from 1 to 5. </p> <p>3) Length of the list <code>x</code> is typically 7-50.</p> <p>4) Length of <code>trainY</code> is approx 0.8mil</p> <p>My current code to do this is:</p> <pre><code>sum([1 for n in x if label in trainY[n]]) </code></pre> <p>This is currently a performance bottleneck of my program and I'm looking for a way to make it much faster. I think we can skip the loop over <code>x</code> and just do a vectorised looking up <code>trainY</code> like <code>trainY[x]</code> but I couldn't get something that worked.</p> <p>Thank you.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload