Databases
StackOverflow2013
Postsdbo
POMongoDB performs bad on 600k objects, alternative DB? optimizations?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POMongoDB performs bad on 600k objects, alternative DB? optimizations?
    primarykey
    data
    text
    <p>I've started a new project using node.js and mongodb and after almost 2 days I gathered about 600k objects in MongoDB. I'm already noticing a huge (negative) impact on the performance and I'm starting to worry if I should move to another DB as long as I can, or if I should stick with Mongo and do some (more) optimizations.</p> <p>Basically I'm storing coordinates like this:</p> <pre><code>[x1] =&gt; 687 [y1] =&gt; 167 [x2] =&gt; 686 [y2] =&gt; 167 [c] =&gt; 0 [s] =&gt; 0 [m] =&gt; 1299430700312 [_id] =&gt; MongoId Object ( [$id] =&gt; 4d73bd2c82bb5926780001ec ) </code></pre> <p>Not more... and my queries look like this: </p> <pre><code>{'$or': [ { x1: {'$gte' : 0, '$lt' : 1000 }, y1: {'$gte' : 0, '$lt' : 1000 } , { x2: {'$gte' : 0, '$lt' : 1000 }, y2: {'$gte' : 0, '$lt' : 1000 } } ] } </code></pre> <p>I've tried setting the index for each of the fields: x1,y1,y1,y1 as well as for: <code>{x1:1,y1:1},{x2:1,y2:1}</code>. Furthermore I've also only fetched the required fields I need... but still, performing a query with a resultset of ~40k rows ends up in a runtime of 2-8secs. Btw: performing the same query in PHP died with with a Out-of-Memory message (256MB RAM). </p> <p>The machine is a Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz with 8GB of RAM, it's not the most dusty one in the rack ;)</p> <p>I'm really running out of ideas and I see millions and millions of rows coming for the next weeks. As you probably noticed the rows are relatively small. Would MySQL with partitioning perform better? Any other NoSQL DB? </p> <p>And please to trolling about "2-8secs isn't slow" - it's becoming a problem already. When a couple of uncached requests hit the machine at the same time, the load raises up to 4 and less than 10 users accessing it.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. USTim Cooper
    1. USSteffen
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POMongoDB performs bad on 600k objects, alternative DB? optimizations?
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POMongoDB performs bad on 600k objects, alternative DB? optimizations?
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POMongoDB performs bad on 600k objects, alternative DB? optimizations?
      1. This table or related slice is empty.
      1. VTUpMod
    1. COWhat indexes do you have setup? Try running your query in the shell and attach .explain() at the end, that should show you the query plan and what indexes are being hit.
      singulars
      1. POMongoDB performs bad on 600k objects, alternative DB? optimizations?
      1. USBrendan W. McAdams
    2. COI'm not entirely sure what you are trying to accomplish with this query but you might have better luck using geospatial indexes. http://www.mongodb.org/display/DOCS/Geospatial+Indexing
      singulars
      1. POMongoDB performs bad on 600k objects, alternative DB? optimizations?
      1. USBernie Hackett
    3. COAssuming those coordinates are, as it looks, the corners of rectangles, you should be using MongoDB's geospatial indexes: http://www.mongodb.org/display/DOCS/Geospatial+Indexing
      singulars
      1. POMongoDB performs bad on 600k objects, alternative DB? optimizations?
      1. USBrendan W. McAdams
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload