Databases
StackOverflow2013
Postsdbo
PO
 

Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>Actually, your workload is not really I/O bound: it is CPU bound due to the cost of jade-based dynamic page generation. I cannot guess the complexity of your jade template, but even with simple templates, generating HTML pages is expensive.</p> <p>For my tests I used this template:</p> <pre><code>html(lang="en") head title Example body h1 Jade - node template engine #container ul#users each user in items li User:#{user} </code></pre> <p>I added 100 dummy strings to the items key in Redis.</p> <p>On my box, I get 475 req/s with node.js CPU at 100% (which means 50% CPU consumption on this dual core box). Let's replace:</p> <pre><code>res.render( 'index', { items: items } ); </code></pre> <p>by:</p> <pre><code>res.send( '&lt;html lang="en"&gt;&lt;head&gt;&lt;title&gt;Example&lt;/title&gt;&lt;/head&gt;&lt;body&gt;&lt;h1&gt;Jade - node template engine&lt;/h1&gt;&lt;div id="container"&gt;&lt;ul id="users"&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;li&gt;User:NOTHING&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;&lt;/body&gt;&lt;/html&gt;' ); </code></pre> <p>Now, the result of the benchmark is close to 2700 req/s. So the bottleneck is clearly due to the formatting of the HTML page.</p> <p>Using the cluster package in this situation is a good idea, and it is straightforward. The code can be modified as follows:</p> <pre><code>var cluster = require('cluster') if ( cluster.isMaster ) { for ( var i=0; i&lt;2; ++i ) cluster.fork(); } else { var express = require( 'express' ), app = express.createServer(), redis = require( 'redis' ).createClient(); app.configure( function() { app.set( 'view options', { layout: false } ); app.set( 'view engine', 'jade' ); app.set( 'views', __dirname + '/views' ); app.use( express.bodyParser() ); }); function log( what ) { console.log( what ); } app.get( '/', function( req, res ) { redis.lrange( 'items', 0, 50, function( err, items ) { if( err ) { log( err ); } else { res.render( 'index', { items: items } ); } }); }); app.listen( 8080 ); } </code></pre> <p>Now the result of the benchmark is close to 750 req/s with 100 % CPU consumption (to be compared with the initial 475 req/s).</p>
    singulars
    1. This table or related slice is empty.
    1. POHow do I use Node.js clusters with my simple Express app?
      singulars
      1. PTQuestion
    1. PTAnswer
    1. This table or related slice is empty.
    1. USDidier Spezia
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. POHow do I use Node.js clusters with my simple Express app?
      singulars
      1. PTQuestion
    1. This table or related slice is empty.
    1. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTAcceptedByOriginator
    2. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. PO
      1. This table or related slice is empty.
      1. VTUpMod
    1. COwow, thanks!, that worked really well — imgur.com/Cpqy6 — I spent most of yesterday pushing it from 80 to 150 and now a simple if block is making all that work look like peanuts. The note about templates was very educational; I knew there would be some cost but didn't think it'd be "475 to 2700" big. Can I optimize Express? Btw, I also wanted to share the CPU stats — imgur.com/n7WPv — I went as high as 15 workers but both cores never went to 100%. I haven't enabled Redis persistence so it shouldn't touch the disk, but around the 30K-request mark my disk spins crazy. Am I overlooking something?
      singulars
      1. PO
      1. USvjk2005
    2. COWith a dual core box, the optimal number of worker processes is 2. I don't know how much memory you have, but if you start too many workers, you may trigger swapping. On my box, one node process takes around 50 MB in memory.
      singulars
      1. PO
      1. USDidier Spezia
    3. COThe performance bottleneck is not express but jade. You could try another templating engine (still working with express). Here is a list: https://github.com/joyent/node/wiki/modules#wiki-templating
      singulars
      1. PO
      1. USDidier Spezia
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload