Databases
StackOverflow2013
Postsdbo
POtransforming large imports of data
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POtransforming large imports of data
    primarykey
    data
    text
    <p>I have a question on the best way to import a long csv table and then transform into two tables for efficiency. This is a very scaled down version but for this purpose suits:</p> <p>each row is a unique record but the first three columns consist of very consistant data over a large amount of rows. </p> <p>I figured the best way to manage this data was to build two tables:</p> <p>The first being an auto increment id field and a group by of the first three columns.</p> <p>This gives a nice compact table of the main groupings of my data.</p> <p>The second table was to be every row but instead of holding all the repeated data I hold only the variable data columns d, e and f along with the autoincrement id field i generate when importing into the first table.</p> <p>My question is really how do I get the id from the first table - is my only way to requery that table to find the id and then do the insert into the second table?</p> <pre><code>a,b,c,d,e,f 09/02/2013,A1,1,18503112043123,11,2.1219 09/02/2013,A1,1,44102576116476,73,14.0817 09/02/2013,A1,1,66918345446536,134,25.8486 09/02/2013,A1,2,62009507978229,10,1.929 09/02/2013,A1,2,92278593945574,55,10.6095 09/02/2013,B1,1,50474606002324,90,17.361 09/02/2013,B1,1,59697581427675,7,1.3503 09/02/2013,B1,1,86298530583467,51,9.8379 09/02/2013,B1,2,34885481077847,80,15.432 09/02/2013,B1,2,25479347211047,164,31.6356 09/02/2013,B1,3,56270556524425,6,1.1574 09/02/2013,C1,1,57680166803098,24,4.6296 09/02/2013,C1,1,72778510788287,77,14.8533 09/02/2013,C1,1,26084111080146,140,27.006 09/02/2013,C1,1,31435464483578,361,65.5937 09/02/2013,C1,2,29457756254473,492,89.3964 09/02/2013,C1,2,68414218104066,293,53.2381 </code></pre> <p>EDIT</p> <p>I have two queries in mind: 1: My parent table which has an auto increment insert into parent_table select null,a,b,c from table group by a,b,c</p> <ol> <li>My child table which is all my rows of data but includes corresponding auto increment id from the parent table.</li> </ol> <p>I dont understand how to pull the id back again without doing a query back to the parent table as i input the data into the child table</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USuser1479891
    1. USuser1479891
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload