Databases
StackOverflow2013
Postsdbo
POWin32: Write to file without buffering?
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWin32: Write to file without buffering?
    text
    copied!<p>I need to create a new file handle so that any write operations to that handle get written to disk immediately. </p> <p>Extra info: The handle will be the inherited STDOUT of a child process, so I need any output from that process to immediately be written to disk.</p> <p>Studying the <code>CreateFile</code> documentation, the <code>FILE_FLAG_WRITE_THROUGH</code> flag looked like exactly what I need:</p> <blockquote> <p>Write operations will not go through any intermediate cache, they will go directly to disk.</p> </blockquote> <p>I wrote a very basic test program and, well, it's not working. I used the flag on CreateFile then used <code>WriteFile(myHandle,...)</code> in a long loop, writing about 100MB of data in about 15 seconds. (I added some <code>Sleep()</code>'s). </p> <p>I then set up a professional monitoring environment consisting of continuously hitting 'F5' in explorer. The results: the file stays at 0kB then jumps to 100MB about the time the test program ends.</p> <p>Next thing I tried was to manually flush the file after each write, with <code>FlushFileBuffers(myHandle)</code>. This makes the observed file size grow nice and steady, as expected.</p> <p>My question is, then, shouldn't the <code>FILE_FLAG_WRITE_THROUGH</code> have done this <strong>without</strong> manually flushing the file? Am I missing something? In the 'real world' program, I can't flush the file, 'cause I don't have any control over the child process that's using it.</p> <p>There's also the <code>FILE_FLAG_NO_BUFFERING</code> flag, that I can't be used for the same reason - no control over the process that's using the handle, so I can't manually align the writes as required by this flag.</p> <p>EDIT: I have made a separate project specifically for watching how the size of the file changes. It uses the .NET <a href="http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx" rel="nofollow noreferrer"><code>FileSystemWatcher</code></a> class. I also write less data - around 100kB in total.</p> <p>Here's the output. Check out the seconds in the timestamps.</p> <p>The 'builtin no-buffers' version: </p> <pre><code>25.11.2008 7:03:22 PM: 10230 bytes added. 25.11.2008 7:03:31 PM: 10240 bytes added. 25.11.2008 7:03:31 PM: 10240 bytes added. 25.11.2008 7:03:31 PM: 10240 bytes added. 25.11.2008 7:03:31 PM: 10200 bytes added. 25.11.2008 7:03:42 PM: 10240 bytes added. 25.11.2008 7:03:42 PM: 10240 bytes added. 25.11.2008 7:03:42 PM: 10240 bytes added. 25.11.2008 7:03:42 PM: 10240 bytes added. 25.11.2008 7:03:42 PM: 10190 bytes added. </code></pre> <p>... and the 'forced (manual) flush' version (<code>FlushFileBuffers()</code> is called every ~2.5 seconds):</p> <pre><code>25.11.2008 7:06:10 PM: 10230 bytes added. 25.11.2008 7:06:12 PM: 10230 bytes added. 25.11.2008 7:06:15 PM: 10230 bytes added. 25.11.2008 7:06:17 PM: 10230 bytes added. 25.11.2008 7:06:19 PM: 10230 bytes added. 25.11.2008 7:06:21 PM: 10230 bytes added. 25.11.2008 7:06:23 PM: 10230 bytes added. 25.11.2008 7:06:25 PM: 10230 bytes added. 25.11.2008 7:06:27 PM: 10230 bytes added. 25.11.2008 7:06:29 PM: 10230 bytes added. </code></pre>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload