StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>Try this:</p> <pre><code>{ cat -<<EOS 10 7.74336e-08 7.30689e-08 0.359106 19.981796 -0.160611 0.027 10 7.74336e-08 7.30689e-08 0.363938 19.985069 0.041319 0.035 10 7.74336e-08 7.30689e-08 0.363133 19.982094 0.041319 0.068 10 7.74336e-08 7.30689e-08 0.360716 19.981796 -0.160611 0.006 10 7.74336e-08 7.30689e-08 0.361522 19.981796 0.243249 0.049 10 7.74336e-08 7.30689e-08 0.357897 19.986260 0.041319 0.035 EOS } | awk ' BEGIN { binSz=3; binSzLim=binSz ; binSzLim++ } NR==1{ # base error checking on number of cols in first record maxCols=NF maxColsLim=maxCols ; maxColsLim++ r=0 } { if (NF != maxCols) { print "Skipping record, Mismatch in data, expected " maxCols ", found " NF " recs at " NR ":" $0 next } r++ #dbg print "r="r" NR=" NR ":$0=" $0 # load data into temp arr[] by column for (c=1;c<maxColsLim;c++) { arr[r,c]+=$c #dbg printf ("arr["r","c"]=" arr[r,c] " " ) avgArr[r,c]++ #dbg print "avgArr["r","c"]="avgArr[r,c] } if(r>=binSz) { r=0 } } END { for (r=1;r<binSzLim;r++) { #dbg print "r=" r " binSzLim=" binSzLim " " (r<binSzLim) "\t" for (c=1;c<maxColsLim;c++) { #dbg printf("arr["r","c"]=" arr[r,c] "\tavg=" arr[r,c]/binSz "\t") printf( arr[r,c]/avgArr[r,c] " ") } printf "\n" } } ' </code></pre> <p>Produces output</p> <pre><code>10 7.74336e-08 7.30689e-08 0.359911 19.9818 -0.160611 0.0165 10 7.74336e-08 7.30689e-08 0.36273 19.9834 0.142284 0.042 10 7.74336e-08 7.30689e-08 0.360515 19.9842 0.041319 0.0515 </code></pre> <p>I added the first column with 10 into the data to make it easy to debug that sum and avg were working correctly.</p> <p>An interesting problem, thanks for posting and thanks for your good will in continuing to answer my questions;-) </p> <p>The <code>{ cat -<<EOS ... EOS }|</code> was just to make it easy to run the whole thing in one copy/paste and see that it is working. You can put the awk code if a file with <code>#!/bin/awk -f</code> at the top, <code>chmod 755 myScript.awk</code> and run it as <code>myScript.awk BigFile > AvgsFile</code>.</p> <p>You should only need to change to <code>binSz=1000</code> in the BEGIN block to process your files as you intended.</p> <p>I'd be interested to know what the timing is on this version.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload