Databases
StackOverflow2013
Postsdbo
POPreferred (or recommended) way to store large amounts of simulation configurations, runs values and final results
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POPreferred (or recommended) way to store large amounts of simulation configurations, runs values and final results
    primarykey
    data
    text
    <p>I am working with some network simulator. After making some extensions to it, I need to make a lot of different simulations and tests. I need to record:</p> <ul> <li>simulation scenario configurations</li> <li>values of some parameters (e.g. buffer sizes, signal qualities, position) per devices per time unit t</li> <li>final results computed from those recorded values</li> </ul> <p>Second data is needed to perform some visualization after simulation was performed (simple animation, showing some statistics over time).</p> <p>I am using Python with matplotlib etc. for post-processing the data and for writing a proper app (now considering pyQt or Django, but this is not the topic of the question). Now I am wondering what would be the best way to store this data?</p> <p>My first guess was to use XML files, but it can be too much overhead from the XML syntax (I mean, files can grow up to very big sizes, especially for the second part of the data type). So I tried to design a database... But this also seems to me to be not the proper way... Maybe a mix of both?</p> <p>I have tried to find some clues in Google, but found nothing special. Have you ever had a need for storing such data? How have you done that? Is there any "design pattern" for that?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USKokos
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POPreferred (or recommended) way to store large amounts of simulation configurations, runs values and final results
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POPreferred (or recommended) way to store large amounts of simulation configurations, runs values and final results
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POPreferred (or recommended) way to store large amounts of simulation configurations, runs values and final results
      1. This table or related slice is empty.
      1. VTUpMod
    1. CO"database" can have quite a few different meanings... I think the questions here are: 1/ what your data looks like (structures, types etc) 2/ how do you plan on using your data (queries, concurrent access, etc) 3/ are your data critical ? (what happens if they are lost, corrupt, whatever ?) 4/ and of course how much data you will have...
      singulars
      1. POPreferred (or recommended) way to store large amounts of simulation configurations, runs values and final results
      1. USbruno desthuilliers
    2. CO1. Currently I have simple simulation txt log files, which then I parse and create a simple collections e.g. mobile terminal position at time x is the array (Terminals x TimeSlots) of tuples (x,y). Then usually I save those parsed data to multiple csv files. But in designed app I am considering to pack this data in some classes, which then can simplify the development of visualization and gui part.
      singulars
      1. POPreferred (or recommended) way to store large amounts of simulation configurations, runs values and final results
      1. USKokos
    3. CO2. There will not be any concurrent access, I suppose that only one person will be using the same data in the same time. My first, draft vision is simple select simulation, show result, play mobility viz etc. but I want to make it flexible for further extensions. 3. I don't think so 4. from tens of MB to some (I hope no more than 3 ;)) GB per simulation run.
      singulars
      1. POPreferred (or recommended) way to store large amounts of simulation configurations, runs values and final results
      1. USKokos
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload