Databases
StackOverflow2013
Postsdbo
PODesign: Large archive file editor, file mapping
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PODesign: Large archive file editor, file mapping
    primarykey
    data
    text
    <p>I'm writing an editor for large <em>archive files</em> (see below) of 4GB+, in native&amp;managed C++.</p> <p>For accessing the files, I'm using <em>file mapping</em> (see below) like any sane person. This is absolutely great for reading data, but a problem arises in actually editing the archive. File mapping does not allow resizing a file while it's being accessed, so I don't know how I should proceed when the user wants to insert new data in the file (which would exceed the file's original size, when it was mapped.)</p> <p>Should I remap the whole thing every time? That's bound to be slow. However, I'd want to keep the editor real-time with exclusive file access, since that simplifies the programming a lot, and won't let the file get screwed by other applications while being modified. I wouldn't want to spend an eternity working on the editor; It's just a simple dev-tool for the actual project I'm working on.</p> <p>So I'd like to hear how you've handled similar cases, and what other archiving software and especially other games do to solve this?</p> <p>To clarify:</p> <ul> <li><p>This is not a text file, I'm writing a specific binary <em>archive file format</em>. By which I mean a big file that contains many others, in directories. Custom archive files are very common in game usage for a number of reasons. With my format, I'm aiming to a similar (but somewhat simpler) structure as with <a href="http://www.wunderboy.org/docs/gcfformat.php" rel="nofollow noreferrer">Valve Software's GCF format</a> - I would have used the GCF format as it is, but unfortunately no editor exists for the format, although there are many great implementations for reading them, like <a href="http://nemesis.thewavelength.net/index.php?p=35" rel="nofollow noreferrer">HLLib</a>.</p></li> <li><p>Accessing the file must be fast, as it is intended for storing game resources. So it's not a database. Database files would be contained inside it, along with GFX, SFX etc. files.</p></li> <li><p>"File mapping" as talked here is a specific technique on the Windows platform, which allows direct access to a large file through creating "views" to parts of it, see here: <a href="http://msdn.microsoft.com/en-us/library/aa366556(VS.85).aspx" rel="nofollow noreferrer">http://msdn.microsoft.com/en-us/library/aa366556(VS.85).aspx</a> - This technique allows minimal latency and memory usage and is a no-brainer for accessing any large files. So this does <em>not</em> mean reading the whole 4GB file into memory, it's exactly the contrary.</p></li> </ul>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USanon6439
    1. USanon6439
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. PODesign: Large archive file editor, file mapping
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PODesign: Large archive file editor, file mapping
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PODesign: Large archive file editor, file mapping
      1. USKlaim
      1. VTFavorite
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload