Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <ol> <li><p>determine raw data size and format</p> <ul> <li>as you wrote you already know the format is stereo doubles (binary)</li> <li>size can be obtained by seek to an end of RAW data file</li> <li>on 32 bit OS beware the 2GB boundary</li> </ul></li> <li><p>read buffer size depends on what you want to do with the data</p> <ul> <li>I assume non-real-time playback</li> <li>use of some filters (like noise reduction, pause removal ...)</li> <li>so you most likely need some previous and in some cases even next samples</li> <li>for large files I use buffer size from 8KB up to 16MB</li> <li>on MCU platforms the sizes usually goes from 32B up to 2KB</li> <li>you have to try few sizes from above intervals (use #define or const)</li> <li>and choose the best compromise between speed and memory consumption</li> <li>most processing algorithms performance saturate on some size value and do not increase with size afterwards</li> </ul></li> <li><p>read loop</p> <ul> <li>the best way for me is that reading the RAW data file is done from main thread (single thread only)</li> <li>and the data is feeded to the processing threads evenly</li> <li>number of threads is usually the number of CPUs</li> <li>do not forget about sharing locks</li> <li>and all threads should have their own buffers</li> </ul></li> </ol> <p>Now the read algorithm:</p> <ol> <li>allocate/start threads buffers ...</li> <li><p>file seek to 0 from start</p> <ul> <li>set main index variable, int ix=0;</li> </ul></li> <li>find first thread with empty buffer</li> <li>lock thread/buffer for main thread<br> <ul> <li>set its actual index to main ix</li> <li>increment main ix, ix++;</li> </ul></li> <li>copy last samples to buffer if needed for processing</li> <li>if there are none yet copy safe sample values instead (0 for example)</li> <li>read rest of the buffer from RAW file</li> <li>copy safe values if end of file to fill buffer...</li> <li>unlock thread/buffer from main thread</li> <li>if not end of file goto 3</li> </ol> <p>Ok what about the processed data:</p> <ul> <li>if you have it in memory only then its not a big deal</li> <li>if you store it to file again then you need to write also write algorithm</li> <li>1 loop ix from 0 to ...</li> <li>2 wait for thread with the same ix to be done</li> <li>3 lock it for write</li> <li>4 write output data to file</li> <li>5 set thread as empty (ready for new data)</li> <li>6 unlock it.</li> <li>PS At the last write you can cut off unused data to match original file size</li> </ul> <p>PS. </p> <ul> <li>the read can be also done by dividing the entire file to N blocks where N is the number of threads but that requires some processing for merging output data and read is conflicting it self so it is slower ...</li> <li>also you can wait for all processing threads to be done and write them all at once (sometimes it is faster sometimes slower ... depends on the processing)</li> </ul> <p>Sorry for code absence but it is all platform/enviroment dependent and you did not specify any so I hope this helps anyway...</p>
    singulars
    1. This table or related slice is empty.
    1. POAlgorithm for slicing vector evenly
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USSpektre
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload