Databases
StackOverflow2013
Postsdbo
POWhy is PostgreSQL eating up all my precious HD space?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhy is PostgreSQL eating up all my precious HD space?
    primarykey
    data
    text
    <p>I just finished transferring as much link-structure data concerning wikipedia (English) as I could. Basically, I downloaded a bunch of SQL dumps from wikipedia's <a href="http://download.wikimedia.org/enwiki/latest/" rel="nofollow noreferrer">latest dump repository</a>. Since I am using PostgreSQL instead of MySQL, I decided to load all these dumps into my db using <a href="http://jhacks.anzix.net/space/kocka/my2pg" rel="nofollow noreferrer">pipeline shell commands</a>.</p> <p>Anyway, one of these tables has 295 million rows: the <em>pagelinks</em> table; it contains all intra-wiki hyperlinks. From my laptop, using pgAdmin III, I sent the following command to my database server (another computer):</p> <pre><code>SELECT pl_namespace, COUNT(*) FROM pagelinks GROUP BY (pl_namespace); </code></pre> <p>Its been at it for an hour or so now. The thing is that the postmaster seems to be eating up more and more of my very limited HD space. I think it ate about 20 GB as of now. I had previously played around with the postgresql.conf file in order to give it more performance flexibility (i.e. let it use more resources) for it is running with 12 GB of RAM. I think I basically quadrupled most bytes and such related variables of this file thinking it would use more RAM to do its thing. </p> <p>However, the db does not seem to use much RAM. Using the Linux system monitor, I am able to see that the postmaster is using 1.6 GB of shared memory (RAM). Anyway, I was wondering if you guys could help me better understand what it is doing for it seems that I really do not understand <em>how PostgreSQL uses HD resources</em>.</p> <p>Concerning the metastructure of wikipedia databases, they provide a good <a href="http://upload.wikimedia.org/wikipedia/commons/4/41/Mediawiki-database-schema.png" rel="nofollow noreferrer">schema</a> that may be of use or even but of interest to you.</p> <p>Feel free to ask me for more details, thx.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USNicholas Leonard
    1. USNicholas Leonard
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload