Databases
StackOverflow2013
Postsdbo
POPython memory consumption: dict VS list of tuples
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POPython memory consumption: dict VS list of tuples
    primarykey
    data
    text
    <p>There are plenty of questions and discussion about memory consumption of different python data types. Yet few of them (if any) come to a very specific scenario. When you want to store LOTS of key-value data in memory, which data structure is more memory-efficient, a dict or a list of tuples?</p> <p>At beginning I thought dict is more powerful than list of tuples and that power must come with some price, and actually an empty dict DOES occupy more memory than an empty list or tuple (see <a href="https://stackoverflow.com/questions/1331471/in-memory-size-of-python-stucture">In-memory size of a Python structure</a>), so I thought using <code>[(key1, value1), (key2, value2), ...]</code> would be more memory-efficient than <code>{key1: value1, key2: value2, ...}</code>.</p> <p>Looks like I was wrong. Just fire up the following code snippet, and see the mem consumption reported by your OS. I am using Windows XP so that task manager tells me, a large dict eats up "only" 40MB Ram and 40MB VIRTURAL Ram, but a list of tuples eats up 60MB Ram and 60MB Virtual ram.</p> <p>How could that be?</p> <pre><code>from sys import getsizeof as g raw_input('ready, press ENTER') i = 1000000 #p = [(x, x) for x in xrange(i)] # Will print 4,348,736 40,348,736 p = dict((x, x) for x in xrange(i)) # Will print 25,165,964 37,165,964 print g(p), g(p) + sum(g(x) for x in p) raw_input("Check your process's memory consumption now, press ENTER to exit") </code></pre> <p><strong>Update:</strong></p> <p>Thanks for some of the comments below. I wanna clarify: I'm talking about memory-efficiency. And no, in this case no need to worry about key-value lookup efficiency, let's just assume my algorithm will consume them one by one via iterator.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USCommunity
    1. USRayLuo
    plurals
    1. PL
      singulars
      1. LTLinked
    2. PL
      singulars
      1. LTLinked
    3. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POPython memory consumption: dict VS list of tuples
      1. USRayLuo
      1. VTFavorite
    2. VO
      singulars
      1. POPython memory consumption: dict VS list of tuples
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POPython memory consumption: dict VS list of tuples
      1. This table or related slice is empty.
      1. VTUpMod
    1. COYou are asking the wrong question. If you need key-value look up, then go with dict. If you need an array then use list or tuple.
      singulars
      1. POPython memory consumption: dict VS list of tuples
      1. USHai Vu
    2. COPython keeps a hash table for dictionaries. [This link](http://mail.python.org/pipermail/python-list/2000-March/048085.html) was from [another answer](http://stackoverflow.com/questions/114830/is-a-python-dictionary-an-example-of-a-hash-table) I think, then, that dictionaries are faster for lookups, and tuples use less memory.
      singulars
      1. POPython memory consumption: dict VS list of tuples
      1. USmbowden
    3. COFor some kinds of data you can use something even more optimal than both your options, like a trie.
      singulars
      1. POPython memory consumption: dict VS list of tuples
      1. USwRAR
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload