Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>The question is (now) about storing lots of data, which can be represented using primitive types like <code>int</code>, in a Map. Some of the answers here are very misleading in my opinion. Let's see why.</p> <p>I modified the benchmark from <a href="http://trove4j.sourceforge.net/index.html" rel="noreferrer">trove</a> to measure both runtime and memory consumption. I also added <a href="http://pcj.sourceforge.net/" rel="noreferrer">PCJ</a> to this benchmark, which is another collections library for primitive types (I use that one extensively). The 'official' trove benchmark does not compare IntIntMaps to Java Collection's <code>Map&lt;Integer, Integer&gt;</code>, probably storing <code>Integers</code> and storing <code>ints</code> is not the same from a technical point of view. But a user might not care about this technical detail, he wants to store data representable with <code>ints</code> efficiently.</p> <p>First the relevant part of the code:</p> <pre><code>new Operation() { private long usedMem() { System.gc(); return Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory(); } // trove public void ours() { long mem = usedMem(); TIntIntHashMap ours = new TIntIntHashMap(SET_SIZE); for ( int i = dataset.size(); i-- &gt; 0; ) { ours.put(i, i); } mem = usedMem() - mem; System.err.println("trove " + mem + " bytes"); ours.clear(); } public void pcj() { long mem = usedMem(); IntKeyIntMap map = new IntKeyIntOpenHashMap(SET_SIZE); for ( int i = dataset.size(); i-- &gt; 0; ) { map.put(i, i); } mem = usedMem() - mem; System.err.println("pcj " + mem + " bytes"); map.clear(); } // java collections public void theirs() { long mem = usedMem(); Map&lt;Integer, Integer&gt; map = new HashMap&lt;Integer, Integer&gt;(SET_SIZE); for ( int i = dataset.size(); i-- &gt; 0; ) { map.put(i, i); } mem = usedMem() - mem; System.err.println("java " + mem + " bytes"); map.clear(); } </code></pre> <p>I assume the data comes as primitive <code>ints</code>, which seems sane. But this implies a runtime penalty for java util, because of the auto-boxing, which is not neccessary for the primitive collections frameworks. </p> <p>The runtime results (without <code>gc()</code> calls, of course) on WinXP, jdk1.6.0_10:</p> <pre> 100000 put operations 100000 contains operations java collections 1938 ms 203 ms trove 234 ms 125 ms pcj 516 ms 94 ms </pre> <p>While this might already seem drastic, this is not the reason to use such a framework. </p> <p>The reason is memory performance. The results for a Map containing 100000 <code>int</code> entries:</p> <pre> java collections oscillates between 6644536 and 7168840 bytes trove 1853296 bytes pcj 1866112 bytes </pre> <p>Java Collections needs <strong>more than three times</strong> the memory compared to the primitive collection frameworks. I.e. you can keep three times as much data in memory, without resorting to disk IO which lowers runtime performance by magnitudes. And this matters. Read <a href="http://highscalability.com/" rel="noreferrer">highscalability</a> to find out why.</p> <p>In my experience high memory consumption is the biggest performance issue with Java, which of course results in worse runtime performance as well. Primitive collection frameworks can really help here.</p> <p>So: No, java.util is not the answer. And "adding functionality" to Java collections is not the point when asking about efficiency. Also the modern JDK collections do <strong>not</strong> "out-perform even the specialized Trove collections".</p> <p>Disclaimer: The benchmark here is far from complete, nor is it perfect. It is meant to drive home the point, which I have experienced in many projects. Primitive collections are useful enough to tolerate fishy API - <strong>if</strong> you work with lots of data.</p>
    singulars
    1. This table or related slice is empty.
    1. POWhat is the most efficient Java Collections library?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. USthe.duckman
    1. USthe.duckman
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTDownMod
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COActually, I think your answer is misleading. Storing ints vs Integers is very different, and most likely the main reason for the increased memory usage. I agree a raw type collection framework could be useful, but it doesn't make trove or pcj "better" than java.util.
      singulars
      1. PO
      1. USJorn
    2. COThe question is about storing int data efficiently. Not about storing Integers. For this task trove/pcj are more efficient, as I tried to show. Using Integers imposes runtime and memory inefficiencies. Since java.util doesn't allow usage of primitives, it is not the best choice for this task.
      singulars
      1. PO
      1. USthe.duckman
    3. CO(for Russian community) here goes another benchmark: http://total-holywar.blogspot.com/2011/07/java-collections-framework.html
      singulars
      1. PO
      1. USdma_k
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload