Databases
StackOverflow2013
Postsdbo
POEC2 - MongoDB - PyMongo - Debugging Query Latency
Body
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POEC2 - MongoDB - PyMongo - Debugging Query Latency
    text
    copied!<p>I have 2 EC2 instances, one as a mongodb server and the other is a python web app (same availability zone). The python sever connects to mongo server using PyMongo and everything works fine.</p> <p>The problem is, when I profile execution-time in python, in some calls (less than 5%) it takes even up to couple of second to return. I was able to narrow down the problem and the time delay was actually on the db calls to mongo server.</p> <p>The two causes that I thought were, 1. The Mongo server is slow/over-loaded 2. Network latency</p> <p>So, I tried upgrading the mongo servers to 4X faster instance but the issue still happens (Some calls takes even 3 secs to return) I assumed since both the servers are on EC2, network latency should not be a problem... but may be I was wrong.</p> <p>How can I confirm if the issue is actually the network itself? If so, what the best way to solve it? Is there any other possible cause?</p> <p>Any help is appreciated...</p> <p>Thanks,</p> <p>UPATE: The entities that I am fetching are very small (indexed) and usually the calls takes only 0.01-0.02 secs to finish.</p> <p>UPDATE:</p> <p>As suggested by "James Wahlin", I enabled profiling on my mongo server and got some interesting logs,</p> <blockquote> <p>Fri Mar 15 18:05:22 [conn88635] query db.UserInfoShared query: { $or: [ { _locked: { $exists: false } }, { _locked: { $lte: 1363370603.297361 } } ], _id: "750837091142" } nto return:1 nscanned:1 nreturned:1 reslen:47 2614ms</p> <p>Fri Mar 15 18:05:22 [conn88635] command db.$cmd command: { findAndModify: "UserInfoShared", fields: { _id: 1 }, upsert: true, query: { $or: [ { _locked: { $exists: false } }, { _locked: { $lte: 1363370603.297361 } } ], _id: "750837091142" }, update: { $set: { _locked: 1363370623.297361 } }, new: true } ntoreturn:1 reslen:153 2614ms</p> </blockquote> <p>You can see these two calls took more than 2 secs to finish. The field <code>_id</code> is unique indexed and finding it should not have taken this much time. May be I have to post a new question for it, but can the mongodb GLOBAL LOCK be the cause ?</p>
 

Querying!

 
Guidance
An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload