Databases
StackOverflow2013
Postsdbo
POSecure and efficient way to modify multiple files on POSIX systems?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POSecure and efficient way to modify multiple files on POSIX systems?
    primarykey
    data
    text
    <p>I have been following the discussion on the "bug" on EXT4 that causes files to be zeroed in crash if one uses the "create temp file, write temp file, rename temp to target file" process. POSIX says that unless fsync() is called, you cannot be sure the data has been flushed to harddisk.</p> <p>Obviously doing:</p> <pre><code>0) get the file contents (read it or make it somehow) 1) open original file and truncate it 2) write new contents 3) close file </code></pre> <p>is not good even with fsync() as the computer can crash during 2) or fsync() and you end up with partially written file.</p> <p>Usually it has been thought that this is pretty safe:</p> <pre><code>0) get the file contents (read it or make it somehow) 1) open temp file 2) write contents to temp file 3) close temp file 4) rename temp file to original file </code></pre> <p>Unfortunately it isn't. To make it safe on EXT4 you would need to do:</p> <pre><code>0) get the file contents (read it or make it somehow) 1) open temp file 2) write contents to temp file 3) fsync() 4) close temp file 5) rename temp file to original file </code></pre> <p>This would be safe and on crash you should either have the new file contents or old, never zeroed contents or partial contents. But if the application uses lots of files, fsync() after every write would be slow.</p> <p>So my question is, how to modify multiple files efficiently on a system where fsync() is required to be sure that changes have been saved to disk? And I really mean modifying many files, as in thousands of files. Modifying two files and doing fsync() after each wouldn't be too bad, but fsync() does slow things down when modifying multiple files.</p> <p>EDIT: changed the fsync() close temp file to corrent order, added emphasis on writing many many many files.</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USRaynet
    1. USRaynet
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POSecure and efficient way to modify multiple files on POSIX systems?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POSecure and efficient way to modify multiple files on POSIX systems?
      1. USEugene Morozov
      1. VTFavorite
    3. VO
      singulars
      1. POSecure and efficient way to modify multiple files on POSIX systems?
      1. This table or related slice is empty.
      1. VTUpMod
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload