Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Modifying your code to use an IntBuffer and LongBuffer</p> <pre><code>class LongIntParallelHashMultimap { private static final int NULL = 0; private final FileChannel channel1, channel2; private final LongBuffer keys; private final IntBuffer values; private final int capacity; private int size; public LongIntParallelHashMultimap(int capacity, String basename) throws IOException { assert (capacity &amp; (capacity - 1)) == 0 : "Capacity " + capacity + " must be a power of 2"; this.capacity = capacity; channel1 = new RandomAccessFile(basename + ".keys", "rw").getChannel(); keys = channel1.map(FileChannel.MapMode.READ_WRITE, 0, capacity * 8).order(ByteOrder.nativeOrder()).asLongBuffer(); // load keys into memory for(int i=0;i&lt;capacity;i+=512) keys.get(i); channel2 = new RandomAccessFile(basename + ".values", "rw").getChannel(); values = channel2.map(FileChannel.MapMode.READ_WRITE, 0, capacity * 4).order(ByteOrder.nativeOrder()).asIntBuffer(); for(int i=0;i&lt;capacity;i+=1024) values.get(i); } public void put(long key, int value) { long key1 = key + 1; int index = indexFor(key1); while (keys.get(index) != NULL) { index = successor(index); } values.put(index, value); keys.put(index, key1); ++size; } /** * Uses a pre-allocated array and return the count of matches. */ public int get(long key, int[] hits) { long key1 = key + 1; int index = indexFor(key1); int hitIndex = 0; while (keys.get(index) != NULL) { if (keys.get(index) == key1) { hits[hitIndex] = values.get(index); ++hitIndex; } index = successor(index); } return hitIndex; } private int indexFor(long key) { return Math.abs((int) ((key * 5700357409661598721L) &amp; (capacity - 1))); } private int successor(int index) { return (index + 1) &amp; (capacity - 1); } public int size() { return size; } public void close() { try { channel1.close(); channel2.close(); } catch (IOException ignored) { } try { ((DirectBuffer) keys).cleaner().clean(); ((DirectBuffer) values).cleaner().clean(); } catch (Throwable notSupportedOnThisPlatform) { } } } </code></pre> <hr> <p>To allow the OS to swap the pages of your data structure you need to use off heap memory or memory mapped files. The memory mapped files are easier to manage and can be up to the size of your hard drive in size.</p> <pre><code>long heap = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory(); RandomAccessFile raf = new RandomAccessFile("array.dat", "rw"); IntBuffer map = raf.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, 1 &lt;&lt; 30).order(ByteOrder.nativeOrder()).asIntBuffer(); for (int i = 0; i &lt; map.capacity(); i++) map.put(i, i); long heap2 = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory(); System.out.printf("Wrote %,d int values, heap used %,d bytes approx%n", map.capacity(), heap2 - heap); </code></pre> <p>prints</p> <pre><code>Wrote 268,435,456 int values, heap used 0 approx </code></pre> <p>BTW: You need a 64-bit JVM to map this data all at once. If you have a 32-bit JVM you will need to move the mapping around (which is a pain so use a 64-bit JVM if you can)</p> <p>If you have a 200 MB array, this should fit into a 32-bit JVM.</p>
    singulars
    1. This table or related slice is empty.
    1. POJava - Swapping Page
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USPeter Lawrey
    1. USPeter Lawrey
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POJava - Swapping Page
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTModeratorReview
    1. COThanks. But how to use memory mapped files for swapping arrays ?
      singulars
      1. PO
      1. USArpssss
    2. COYou store the arrays data in the mapped ByteBuffer(s) and the OS will swap the pages of that data as it needs to.
      singulars
      1. PO
      1. USPeter Lawrey
    3. COOk. Is this can perform fast (theoretically) ? Another thing, previously I think, swapping is done by JVM.
      singulars
      1. PO
      1. USArpssss
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload