Databases
StackOverflow2013
Postsdbo
POWhats the 'best' method of writing data out to a file, to be later read in again.
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhats the 'best' method of writing data out to a file, to be later read in again.
    text
    copied!<p>What is the best way of storing data out to a file on a network, which will be later read in again programmatically. Target platform for the program is Linux (Fedora), but it will need to write out a file to a Windows (XP) machine</p> <p>This needs to be in C++, there will be a high number of write / read events so it needs to be efficient, and the data needs to be written out in such a way that it can be read back in easily.</p> <p>The whole file may not be being read back in, I'll need to search for a specific block of data in the file and read that back in.</p> <p>Will simple binary stream writer do? How should I store the data - XML?</p> <p>Anything else I need to worry about?</p> <hr> <p><strong>UPDATE :</strong> To clarify, here are some answers to <strong><em>peterchen's</em></strong> points</p> <blockquote> <p>Please clarify:</p> <p><strong>* do you only append blocks, or do you also need to remove / update them?</strong></p> </blockquote> <p>I only need to append to the end of the file, but will need to search through it and retrieve from any point in it</p> <blockquote> <pre><code>*** are all blocks of the same size?** </code></pre> </blockquote> <p>No, the data will vary in size - some will be free text comments (like a post here) others will be specific object-like data (sets of parameters)</p> <blockquote> <pre><code>*** is it necessary to be a single file?** </code></pre> </blockquote> <p>No, but desirable</p> <blockquote> <pre><code>*** by which criteria do you need to locate blocks?** </code></pre> </blockquote> <p>By data type and by timestamp. For example, if I periodically write out a specific set of parameters, in amognst other data, like free text, I want to find the value of those parameters at a cerain date/time - so I'll need to search for the time I wrote out those parameters nearest that date and read them back in.</p> <blockquote> <pre><code>*** must the data be readable for other applications?** </code></pre> </blockquote> <p>No.</p> <blockquote> <pre><code>*** do you need concurrent access?** </code></pre> </blockquote> <p>Yes, I may be continuing to write as I read. but should only ever do one write at a time.</p> <blockquote> <pre><code>*** Amount of data (per block / total) - kilo, mega, giga, tera?** </code></pre> </blockquote> <p>Amount of data will be low per write... from a number of bytes to a coupe hundred bytes - total should see no more than few hundred kilobytes possible a fwe megabytes. (still unsure as yet)</p> <p>**> If you need all of this, rolling your own will be a challenge, I would definitely </p> <blockquote> <p>recommend to use a database. If you need less than that, please specify so we can recommend.**</p> </blockquote> <p>A database would over complicate the system so that is not an option unfortunately.</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload