Databases
StackOverflow2013
Postsdbo
POMongoDB very slow deletes
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POMongoDB very slow deletes
    primarykey
    data
    text
    <p>I've got a small replica set of three mongod servers (16GB RAM each, at least 4 CPU cores and real HDDs) and one dedicated arbiter. The replicated data has about 100,000,000 records currently. Nearly all of this data is in one collection with an index on <code>_id</code> (the auto-generated Mongo ID) and <code>date</code>, which is a native Mongo date field. Periodically I delete old records from this collection using the date index, something like this (from the mongo shell):</p> <pre><code>db.repo.remove({"date" : {"$lt" : new Date(1362096000000)}}) </code></pre> <p>This does work, but it runs very, very slowly. One of my nodes has slower I/O than the other two, having just a single SATA drive. When this node is primary, the deletes run at about 5-10 documents/sec. By using rs.stepDown() I have demoted this slower primary and forced an election to get a primary with better I/O. On that server, I am getting about 100 docs/sec.</p> <p>My main question is, should I be concerned? I don't have the numbers from before I introduced replication, but I know the delete was much faster. I'm wondering if the replica set sync is causing I/O wait, or if there is some other cause. I would be totally happy with temporarily disabling sync and index updates until the delete statement finishes, but I don't know of any way to do that currently. For some reason, when I disable two of the three nodes, leaving just one node and the arbiter, the remaining node is demoted and writes are impossible (isn't the arbiter supposed to solve that?).</p> <p>To give you some indication of the general performance, if I drop and recreate the date index, it takes about 15 minutes to scan all 100M docs.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USSteveK
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. COthe reason you can't disable two of your four nodes is that there cannot be a primary without majority of the set available. Why do you have four members, by the way? You don't need an arbiter with three nodes in a replica set.
      singulars
      1. POMongoDB very slow deletes
      1. USAsya Kamsky
    2. COGotcha - I only have four nodes at the moment because the 5th node is missing a hard drive and I removed it from the cluster :) Ironically, I brought up an arbiter to help guarantee there would always be a winner in a master election. Anyway, the arbiter is a little VM that I also use for other low overhead stuff like config servers in other sharding clusters.
      singulars
      1. POMongoDB very slow deletes
      1. USSteveK
    3. COyou needed an arbiter when you had four nodes (to have five votes) but when you remove the fifth node from the replica set you should remove the arbiter as well, so that you will have three members left.
      singulars
      1. POMongoDB very slow deletes
      1. USAsya Kamsky
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload