Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>While the iterator-based answers are perfectly fine, if you're working with numpy arrays (as you mention that you are) there are better and faster ways of selecting things:</p> <pre><code>import numpy as np data = np.array([ [100002, 2006, 1.1, 0.01, 6352], [100002, 2006, 1.2, 0.84, 304518], [100002, 2006, 2, 1.52, 148219], [100002, 2007, 1.1, 0.01, 6292], [10002, 2006, 1.1, 0.01, 5968], [10002, 2006, 1.2, 0.25, 104318], [10002, 2007, 1.1, 0.01, 6800], [10002, 2007, 4, 2.03, 25446], [10002, 2008, 1.1, 0.01, 6408] ]) subset1 = data[data[:,0] == 100002] subset2 = data[data[:,0] == 10002] </code></pre> <p>This yields</p> <p>subset1:</p> <pre><code>array([[ 1.00002e+05, 2.006e+03, 1.10e+00, 1.00e-02, 6.352e+03], [ 1.00002e+05, 2.006e+03, 1.20e+00, 8.40e-01, 3.04518e+05], [ 1.00002e+05, 2.006e+03, 2.00e+00, 1.52e+00, 1.48219e+05], [ 1.00002e+05, 2.007e+03, 1.10e+00, 1.00e-02, 6.292e+03]]) </code></pre> <p>subset2:</p> <pre><code>array([[ 1.0002e+04, 2.006e+03, 1.10e+00, 1.00e-02, 5.968e+03], [ 1.0002e+04, 2.006e+03, 1.20e+00, 2.50e-01, 1.04318e+05], [ 1.0002e+04, 2.007e+03, 1.10e+00, 1.00e-02, 6.800e+03], [ 1.0002e+04, 2.007e+03, 4.00e+00, 2.03e+00, 2.5446e+04], [ 1.0002e+04, 2.008e+03, 1.10e+00, 1.00e-02, 6.408e+03]]) </code></pre> <p>If you didn't know the unique values in the first column beforehand, you can use either <a href="http://docs.scipy.org/doc/numpy-1.3.x/reference/generated/numpy.unique1d.html" rel="noreferrer"><code>numpy.unique1d</code></a> or the builtin function <code>set</code> to find them. </p> <p>Edit: I just realized that you wanted to select data where you have unique combinations of two columns... In that case, you might do something like this:</p> <pre><code>col1 = data[:,0] col2 = data[:,1] subsets = {} for val1, val2 in itertools.product(np.unique(col1), np.unique(col2)): subset = data[(col1 == val1) &amp; (col2 == val2)] if np.any(subset): subsets[(val1, val2)] = subset </code></pre> <p>(I'm storing the subsets as a dict, with the key being a tuple of the combination... There are certainly other (and better, depending on what you're doing) ways to do this!)</p>
    singulars
    1. This table or related slice is empty.
    1. POSubsetting data in Python
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USJoe Kington
    1. USJoe Kington
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POSubsetting data in Python
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COThanks! In retrospect this is so obvious that I should have tried it myself, but your explanation is very complete. However, do you know how much of a speed up we can obtain using this approach compared to iterators? I thought that iterators are pretty fast too!
      singulars
      1. PO
      1. USuser308827
    2. CO@user308827 - They are, but if you're working with numpy arrays, and not lists, using the numpy way of doing things will be faster. Generally speaking, iterating through an entire numpy array is slow. Iterator-based solutions have to iterate through each element in python. When you select a subset of a numpy array using a boolean numpy array, the iteration is done behind the scenes in compiled code. (I'm oversimplifying here, but that's the gist, anyway). Basically, if you're using numpy arrays to contain your data, it's faster to operate on them with numpy functions.
      singulars
      1. PO
      1. USJoe Kington
    3. COAlso, is there a way to combine 2 conditions in the subset? E.g., subset1 = data[(data[:,0] == 100002) and (data[:,1] == 2007)] does not seem to work. thanks!
      singulars
      1. PO
      1. USuser308827
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload