Databases
StackOverflow2013
Postsdbo
POVery slow inserts with CouchDB?
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POVery slow inserts with CouchDB?
    primarykey
    data
    text
    <p>I wanted to check how many more inserts CouchDB can handle when compared to MySQL. My test was simple: for 10 seconds keep inserting <code>{firstName: "Testing 001", lastName: "Testing 002"}</code> and compare the number of documents/rows. The results I got were far from my expectations: </p> <ul> <li>MySQL MyIsam: 110,000 rows</li> <li>MySQL InnoDB: 52,000 rows</li> <li>CouchDB: <strong>3,300</strong> documents!</li> </ul> <p>Please correct me if I'm wrong but should a NoSQL always outperform a relational database in simple operations? I would not expect such a dramatic difference. Perhaps my test was naive and I shouldn't be comparing those databases in such a way? I know that MySQL driver has an access to the connection pool and doesn't have to re-create a TCP connection on every request but does it render such a big difference?</p> <p>Should CouchDB insert be so slow and if not how to do it right? </p> <p>I run my tests on a clean CouchDB database (without any design documents) / Macbook Pro 2.6Ghz i7, 16GB RAM, SSD / CouchDB 1.4.0</p> <p>Test script:</p> <pre><code>var nano = require('nano')('http://localhost:5984'); var async = require('async'); var db = nano.db.use('test'); var mysql = require('mysql'); var connection = mysql.createConnection({ host : 'localhost', user : 'root', password : '', database: 'test' }); /* connection.connect(function(err){ var t = new Date().getTime() + 10000; var i = 0; var page = 2, lastPage = 100; async.whilst(function () { return new Date().getTime() &lt; t; }, function (next) { connection.query('INSERT INTO test (firstName, lastName) VALUES ("Testing 001","Testing 002")', function(err, rows, fields) { i += 1; next(); }); }, function (err) { console.log( i ); connection.end(); }); }); */ var t = new Date().getTime() + 10000; var i = 0; var page = 2, lastPage = 100; async.whilst(function () { return new Date().getTime() &lt; t; }, function (next) { db.insert({firstName: "Testing 001", lastName: "Testing 002"}, 'id-' + i, function(){ i += 1; next(); }); }, function (err) { console.log( i ); connection.end(); }); </code></pre> <p><strong>// EDIT:</strong></p> <p>As it turned out the problem is not on the CouchDB side per say. There is something about client's library/drivers which make them terrible slow. A simple POST test with apache benchmark displays a very good results on the CouchDB side:</p> <pre><code>$ ab -n 10000 -c 100 -p post-data -T "application/json" "http://192.168.50.102:5984/test/" This is ApacheBench, Version 2.3 &lt;$Revision: 655654 $&gt; Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 192.168.50.102 (be patient) Completed 1000 requests Completed 2000 requests Completed 3000 requests Completed 4000 requests Completed 5000 requests Completed 6000 requests Completed 7000 requests Completed 8000 requests Completed 9000 requests Completed 10000 requests Finished 10000 requests Server Software: CouchDB/1.5.0 Server Hostname: 192.168.50.102 Server Port: 5984 Document Path: /test/ Document Length: 95 bytes Concurrency Level: 100 Time taken for tests: 1.149 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 4120412 bytes Total POSTed: 1920960 HTML transferred: 950095 bytes Requests per second: 8704.85 [#/sec] (mean) Time per request: 11.488 [ms] (mean) Time per request: 0.115 [ms] (mean, across all concurrent requests) Transfer rate: 3502.69 [Kbytes/sec] received 1632.98 kb/s sent 5135.67 kb/s total Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.1 0 2 Processing: 6 11 2.6 11 23 Waiting: 6 11 2.6 11 22 Total: 6 11 2.6 11 25 </code></pre>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USLukasz Kujawa
    1. USLukasz Kujawa
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. CO"NoSql is faster than RDBMS" is not universally true. DBs have strengths and weaknesses, and this may be one of these cases. The issue with this type of test is that it may highlight the weakness, but does nothing to show where CouchDb might excel for example. If you try to force or coerce a NoSql DB to behave like a RDBMS, the results are often mixed to poor.
      singulars
      1. POVery slow inserts with CouchDB?
      1. USWiredPrairie
    2. COI agree although I like to understand strengths and weaknesses. I assumed that a simple INSERT/SELECT shouldn't be a big challenge for any system. This is why I'm so puzzled about the results.
      singulars
      1. POVery slow inserts with CouchDB?
      1. USLukasz Kujawa
    3. COWrites are commonly the most expensive operation in a DB system. I think I'd switch the tests to actually be async to allow more than one connection/insert at a time.
      singulars
      1. POVery slow inserts with CouchDB?
      1. USWiredPrairie
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload