StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POOptimal datafile format loading on a game console
text
Body
copied!<p>I need to load large models and other structured binary data on an older CD-based game console as efficiently as possible. What's the best way to do it? The data will be exported from a Python application. This is a pretty elaborate hobby project.</p> <p>Requierements:</p> <ul> <li>no reliance on fully standard compliant STL - i might use uSTL though.</li> <li>as little overhead as possible. Aim for a solution so good. that it could be used on the original Playstation, and yet as modern and elegant as possible.</li> <li>no backward/forward compatibility necessary.</li> <li>no copying of large chunks around - preferably files get loaded into RAM in background, and all large chunks accessed directly from there later.</li> <li>should not rely on the target having the same endianness and alignment, i.e. a C plugin in Python which dumps its structs to disc would not be a very good idea.</li> <li>should allow to move the loaded data around, as with individual files 1/3 the RAM size, fragmentation might be an issue. No MMU to abuse.</li> <li>robustness is a great bonus, as my attention span is very short, i.e. i'd change saving part of the code and forget the loading one or vice versa, so at least a dumb safeguard would be nice.</li> <li>exchangeability between loaded data and runtime-generated data without runtime overhead and without severe memory management issues would be a nice bonus.</li> </ul> <p>I kind of have a semi-plan of parsing in Python trivial, limited-syntax C headers which would use structs with offsets instead of pointers, and convenience wrapper structs/classes in the main app with getters which would convert offsets to properly typed pointers/references, but i'd like to hear your suggestions.</p> <p>Clarification: the request is primarily about data loading framework and memory management issues.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload