StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POHow to provide extend-on-write functionality for memory mapped files in Linux?
text
Body
copied!<p>I'm working on porting some code from AIX to Linux. Parts of the code use the <a href="http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=/com.ibm.aix.basetechref/doc/basetrf2/shmat.htm" rel="nofollow"><code>shmat()</code> system call</a> to create new files. When used with <code>SHM_MAP</code> in a writable mode, one can extend the file beyond its original length (of zero, in my case):</p> <blockquote> <p>When a file is mapped onto a segment, the file is referenced by accessing the segment. The memory paging system automatically takes care of the physical I/O. References beyond the end of the file cause the file to be extended in page-sized increments. The file cannot be extended beyond the next segment boundary.</p> </blockquote> <p>(A "segment" in AIX is a 256 MB chunk of address space, and a "page" is usually 4 KB.)</p> <p>What I would <em>like</em> to do on Linux is the following:</p> <ul> <li>Reserve a large-ish chunk of address space (it doesn't have to be as big as 256 MB, these aren't such large files)</li> <li>Set up the page protection bits so that a segfault is generated on the first access to a page that hasn't been touched before</li> <li>On a page fault, clear the "cause a page fault" bit and allocate committed memory for the page, allowing the write (or read) that caused the page fault to proceed</li> <li>Upon closing the shared memory area, write the modified pages to a file</li> </ul> <p>I know I can do this on Windows with the <a href="http://msdn.microsoft.com/en-us/library/aa366898%28v=vs.85%29.aspx" rel="nofollow">VirtualProtect</a> function, the <code>PAGE_GUARD</code> memory protection bit, and a <a href="http://msdn.microsoft.com/en-us/library/ms680657%28v=vs.85%29.aspx" rel="nofollow">structured exception handler</a>. What is the corresponding method on Linux to do the same? Is there perhaps a better way to implement this extend-on-write functionality on Linux?</p> <p>I've already considered:</p> <ul> <li>using <code>mmap()</code> with some fixed large-ish size, but I can't tell how much of the file was written to by the application code</li> <li>allocating an anonymous shared memory area of large-ish size, but again I can't tell how much of the area has been written</li> <li><code>mmap()</code> by itself does not seem to provide any facility to extend the length of the backing file</li> </ul> <p>Naturally I would like to do this with only minimal changes to the application code.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload