Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>The following script should do what you want. The only thing that might not work yet is the choice of delimiters. In your original script you seem to have tabs. My solution assumes spaces. But changing that should not be a problem.</p> <p>It simply pipes all files sequentially into the <code>nawk</code> without counting the files first. I understand that this is not required. Instead of trying to keep track of positions in the file it uses arrays to store seperate statistical data for each step. In the end it iterates over all step indexes found and outputs them. Since the iteration is not sorted there is another pipe into a Unix <code>sort</code> call which handles this.</p> <pre><code>#!/bin/bash # pipe the data of all files into the nawk processor cat *.data | nawk ' BEGIN { FS=" " # set the delimiter for the columns } { step = $1 # step is in column 1 temp = $2*$2 + $3*$3 + $4*$4 # use arrays indexed by step to store data calc[step] = calc[step] + sqrt (temp) fluc[step] = fluc[step] + calc[step]*calc[step] count[step] = count[step] + 1 # count the number of samples seen for a step } END { # iterate over all existing steps (this is not sorted!) for (i in count) { stddev = sqrt((calc[i] * calc[i]) + (fluc[i] * fluc[i])) print i" "calc[i]/count[i]" "fluc[i]/count[i]" "stddev } }' | sort -n -k 1 # that' why we sort here: first column "-k 1" and numerically "-n" </code></pre> <p><strong>EDIT</strong></p> <p>As sugested by @edmorton <code>awk</code> can take care of loading the files itself. The following enhanced version removes the call to <code>cat</code> and instead passes the file pattern as parameter to <code>nawk</code>. Also, as suggested by @NictraSavios the new version introduces a special handling for the output of the statistics of the last step. Note that the gathering of the statistics is still done for all steps. It's a little difficult to suppress this during the reading of the data since at that point we don't know yet what the last step will be. Although this <em>can</em> be done with some extra effort you would probably loose a lot of robustness of your data handling since right now the script does not make any assumptions about:</p> <ul> <li>the number of files provided,</li> <li>the order of the files processed,</li> <li>the number of steps in each file,</li> <li>the order of the steps in a file,</li> <li>the completeness of steps as a range without "holes".</li> </ul> <p>Enhanced script:</p> <pre><code>#!/bin/bash nawk ' BEGIN { FS=" " # set the delimiter for the columns (not really required for space which is the default) maxstep = -1 } { step = $1 # step is in column 1 temp = $2*$2 + $3*$3 + $4*$4 # remember maximum step for selected output if (step &gt; maxstep) maxstep = step # use arrays indexed by step to store data calc[step] = calc[step] + sqrt (temp) fluc[step] = fluc[step] + calc[step]*calc[step] count[step] = count[step] + 1 # count the number of samples seen for a step } END { # iterate over all existing steps (this is not sorted!) for (i in count) { stddev = sqrt((calc[i] * calc[i]) + (fluc[i] * fluc[i])) if (i == maxstep) # handle the last step in a special way print i" "calc[i]/count[i]" "fluc[i]/count[i]" "stddev else # this is the normal handling print i" "calc[i]/count[i] } }' *.data | sort -n -k 1 # that' why we sort here: first column "-k 1" and numerically "-n" </code></pre>
    singulars
    1. This table or related slice is empty.
    1. POBash/Nawk whitespace problems
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USMarcus Rickert
    1. USMarcus Rickert
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POBash/Nawk whitespace problems
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    1. COLose the UUOC and pipe as awk is perfectly capable of opening files, and `" "` is the default FS so no need to set it explicitly.
      singulars
      1. PO
      1. USEd Morton
    2. COI think he set it explicitly to allow me to change it to tabs easily.
      singulars
      1. PO
      1. USNictraSavios
    3. COThe fluctuation seems to be gigantic, and how would I make the standard deviation only operate on the last step? (It's all thats needed)
      singulars
      1. PO
      1. USNictraSavios
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload