Databases
StackOverflow2013
Postsdbo
POPair-wise trimming of files based of max/min values of first/last record
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POPair-wise trimming of files based of max/min values of first/last record
    primarykey
    data
    text
    <p>I have a bit of problem with rather large csv files. I am able to write simple bash/awk scripts, but this problem is harder for my limited awk/bash programming experience.</p> <p><strong>Problem:</strong></p> <ul> <li><p>All my files are in folders. Folders have an even number of csv files that need trimming pair-wise (I will explain with this means). Files are named like: f1L, f1R, f2L, f2R, f3L, f3R, ..., fnL,fnR.</p></li> <li><p>Files need reading in pairs, ie. f1L with f1R. f2L with f2R and so on</p></li> <li><p>Files have two comma-separated fields. f1L (file start/end) and f1R, look like</p></li> </ul> <blockquote> <pre><code>f1L (START) 1349971210, -0.984375 1349971211, -1.000000 f1R (START) 1349971206, -0.015625 1349971207, 0.000000 f1L (END) 1350230398, 0.500000 1350230399, 0.515625 f1R (END) 1350230402, 0.484375 1350230403, 0.515625 </code></pre> </blockquote> <p>What I would like to do with awk is:</p> <ol> <li>Read record 1, field 1 of f1L (i.e 1349971210) then record 1, field 1 of f1R (i.e 1349971206). Then take the maximum of both values (i.e x1 = 1349971210). </li> <li>Read last record , field 1 of f1L (i.e 1350230399) then last record, field 1 of f1R (i.e 1350230403). Then take the minimum value (i.e x2 = 1350230399).</li> <li>Then extract and re-save with the same name all the lines in f1L and f1R between bigger than/equal to x1 and smaller than/equal x2.</li> <li>Repeat process for all the pairs in my directory. </li> </ol> <p>Wondered if any of you have any suggestions of a little script with bash/awk to get the job done. </p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. UStormes
    1. UStormes
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POPair-wise trimming of files based of max/min values of first/last record
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POPair-wise trimming of files based of max/min values of first/last record
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POPair-wise trimming of files based of max/min values of first/last record
      1. This table or related slice is empty.
      1. VTUpMod
    1. COLook at these examples of reading/processing the same file(s) more than once in awk: http://www.gnu.org/software/gawk/manual/html_node/Other-Arguments.html , Good luck.
      singulars
      1. POPair-wise trimming of files based of max/min values of first/last record
      1. USshellter
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload