Databases
StackOverflow2013
Postsdbo
POWhy is MongoDB storage constantly increasing?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POWhy is MongoDB storage constantly increasing?
    primarykey
    data
    text
    <p>I have a single-host database which grew up to 95% of disk space while I was not watching. To remedy the situation I created a process that automatically removes the old records from the biggest collection, so the data usage fell to about 40% of disk space. I figured I was safe as long as the data size doesn't grow near the size of preallocated files, but after a week I was proven wrong:</p> <pre><code>Wed Jan 23 18:19:22 [FileAllocator] allocating new datafile /var/lib/mongodb/xxx.101, filling with zeroes... Wed Jan 23 18:25:11 [FileAllocator] done allocating datafile /var/lib/mongodb/xxx.101, size: 2047MB, took 347.8 secs Wed Jan 23 18:25:14 [conn4243] serverStatus was very slow: { after basic: 0, middle of mem: 590, after mem: 590, after connections: 590, after extra info: 970, after counters: 970, after repl: 970, after asserts: 970, after dur: 1800, at end: 1800 } </code></pre> <p>This is the output of <code>db.stats()</code>: (note that the numbers are in <strong>MB</strong> because of scale)</p> <pre><code>&gt; db.stats(1024*1024) { "db" : "xxx", "collections" : 47, "objects" : 189307130, "avgObjSize" : 509.94713418348266, "dataSize" : 92064, "storageSize" : 131763, "numExtents" : 257, "indexes" : 78, "indexSize" : 29078, "fileSize" : 200543, "nsSizeMB" : 16, "ok" : 1 } </code></pre> <p><strong>Question:</strong> What can I do to stop MongoDB from allocating new datafiles? </p> <p>Running <code>repair</code> is difficult because I would have to install new disk. Would running <code>compact</code> help? If yes, should I be running it regularly and how can I tell when I should run it?</p> <p><strong>UPDATE:</strong> I guess I am missing something fundamental here... Could someone please elaborate on connection between data files, extents, collections and database, and how space is allocated when needed?</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USjohndodo
    1. USjohndodo
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POWhy is MongoDB storage constantly increasing?
      1. USRich
      1. VTFavorite
    2. VO
      singulars
      1. POWhy is MongoDB storage constantly increasing?
      1. USjohndodo
      1. VTFavorite
    1. COYou may want to add some detail about what your app is doing. Mongo doesn't automatically free storage space when it is no-longer used. It will replace records in place in certain situations, but otherwise it will just consume more disk.
      singulars
      1. POWhy is MongoDB storage constantly increasing?
      1. USRich
    2. COIt is basically just writing log and chart data and removing it once it is too old. However, I can't use capped collections because I need to have control over how much data I purge and which data I want to remove.
      singulars
      1. POWhy is MongoDB storage constantly increasing?
      1. USjohndodo
    3. COwhich version of mongodb are you using?
      singulars
      1. POWhy is MongoDB storage constantly increasing?
      1. USGregor
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload