Databases
StackOverflow2013
Postsdbo
POHow to improve performance of update() and save() in MongoDB?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POHow to improve performance of update() and save() in MongoDB?
    primarykey
    data
    text
    <p>I'm looking for tips on how to improve the database performance in the following situation.</p> <p>As a sample application, I wrote a fairly simple app today that uses the Twitter streaming API to search for certain keywords, then I am storing the results in MongoDB. The app is written with Node.js.</p> <p>I'm storing 2 collections. One stores the keyword and an array of tweet id's that reference each tweet found mentioning that keyword. These are being added to the database using .update() with {upsert:true} so that new id's are appended to the 'ids' array.</p> <p>A sample document from this collection looks like this:</p> <p>{ "_id": ObjectId("4e00645ef58a7ad3fc9fd9f9"), "ids": ["id1","id2","id3"], "keyword": "#chocolate" }</p> <p>Update code:</p> <pre><code> keywords.update({keyword: key_word},{$push:{ids: id}},{upsert:true}, function(err){}) </code></pre> <p>The 2nd collection looks like this and are added simply by using .save()</p> <pre><code> { "twt_id": "id1", "tweet": { //big chunk of json that doesn't need to be shown } } </code></pre> <p>I've got this running on my Macbook right now and its been going for about 2 hours. I'm storing a <em>lot</em> of data, probably several hundred documents per minute. Right now the number of objects in Mongodb is 120k+.</p> <p>What I'm noticing is that the cpu usage for the database process is hitting as high as 84% and has been constantly going up gradually since I started the latest test run.</p> <p>I was reading up on setting indexes, but since I'm adding documents and not running queries against them, I'm not sure if indexes will help. A side thought that occurred to me is that update() might be doing a lookup since I'm using $push and that an index might help with that.</p> <p>What should I be looking at to keep MongoDB from eating up ever increasing amounts of CPU?</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USGeuis
    plurals
    1. This table or related slice is empty.
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POHow to improve performance of update() and save() in MongoDB?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POHow to improve performance of update() and save() in MongoDB?
      1. USAlfred
      1. VTFavorite
    3. VO
      singulars
      1. POHow to improve performance of update() and save() in MongoDB?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COA MacBook with a slow disk is unlikely the right choice for performing benchmarks and speaking of performance problems..even with the newest MacBook hardware
      singulars
      1. POHow to improve performance of update() and save() in MongoDB?
      1. USAndreas Jung
    2. CO@Blackmoon The accepted answer suggests otherwise.
      singulars
      1. POHow to improve performance of update() and save() in MongoDB?
      1. UShplbsh
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload