Databases
StackOverflow2013
Postsdbo
POFile Loaded Into Memory Bigger than on Disk
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POFile Loaded Into Memory Bigger than on Disk
    primarykey
    data
    text
    <p>I am loading a text file in java into memory. The size of the text file is 100 MB and contains thousands of lines of either String, Integer, or Double type.</p> <p>Once its loaded into memory though it is taking up an entire 1 GB or more if I compile using 64 bit JDK.</p> <p>This is a problem because I need to load even bigger files.</p> <p>They are stored as their exact data type in an arraylist. I looked at MemoryMappedFiles although I am not sure they would provide the functionality I require, the text files are tab deliminated with 40 columns that I need to display in a JTable, again I don't think I could use a memory mapped file if I need to display into a JTable but maybe I am wrong.</p> <p>Any suggestions would be appreciated.</p> <p>The Data is stored as follows, the reason I need all of it to be in memory is because at any point for purpose of calculations I need access to any of the rows in the file.</p> <p>this is in my table model for the JTable, note the rowdata passed to the tablemodel is nulled out after to make sure there isnt a duplicate copy in memory </p> <pre><code>private Object[][] data; public TableModel(ArrayList&lt;String&gt; headers, RowData[] importedData) { columnNames = new String[headers.size()]; data = new Object[importedData.length][headers.size()]; for (int i = 0; i &lt; headers.size(); i++) // extract the column names { // for the table this.columnNames[i] = headers.get(i); } for (int i = 0; i &lt; importedData.length; i++) // extracting the data { // for the table for (int j = 0; j &lt; headers.size(); j++) { this.data[i][j] = importedData[i].myList.get(j); } } } </code></pre>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USuser2020457
    1. USuser2020457
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POFile Loaded Into Memory Bigger than on Disk
      1. This table or related slice is empty.
      1. VTUpMod
    1. COPlease upload the code for this
      singulars
      1. POFile Loaded Into Memory Bigger than on Disk
      1. USNoam
    2. COWaht is the memory structure use are using?
      singulars
      1. POFile Loaded Into Memory Bigger than on Disk
      1. USalexey28
    3. CODo you need to load the entire file into memory? Displaying 100MB of data in a single JTable sounds like a bad idea.
      singulars
      1. POFile Loaded Into Memory Bigger than on Disk
      1. USDawood ibn Kareem
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload