Databases
StackOverflow2013
Postsdbo
POLong connections with Node.js, how to reduce memory usage and prevent memory leak? Also related with V8 and webkit-devtools
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POLong connections with Node.js, how to reduce memory usage and prevent memory leak? Also related with V8 and webkit-devtools
    primarykey
    data
    text
    <p>Here is what I'm trying to do: I'm developing a Node.js http server, which will hold long connections for pushing purpose(collaborate with redis) from tens of thousands of mobile clients in a single machine.</p> <p>Test environment： </p> <pre><code>1.80GHz*2 CPU/2GB RAM/Unbuntu12.04/Node.js 0.8.16 </code></pre> <p>At the first time, I used "express" module, with which I could reach about 120k concurrent connections before swap being used which means the RAM is not enough. Then, I switched to native "http" module, I got the concurrency up to about 160k. But I realized that there are still too many functionality I don't need in native http module, so I switched it to native "net" module(this means I need to handle http protocol by myself, but that's ok). now, I can reach about 250k concurrent connections per single machine.</p> <p>Here is the main structure of my codes:</p> <pre><code>var net = require('net'); var redis = require('redis'); var pendingClients = {}; var redisClient = redis.createClient(26379, 'localhost'); redisClient.on('message', function (channel, message) { var client = pendingClients[channel]; if (client) { client.res.write(message); } }); var server = net.createServer(function (socket) { var buffer = ''; socket.setEncoding('utf-8'); socket.on('data', onData); function onData(chunk) { buffer += chunk; // Parse request data. // ... if ('I have got all I need') { socket.removeListener('data', onData); var req = { clientId: 'whatever' }; var res = new ServerResponse(socket); server.emit('request', req, res); } } }); server.on('request', function (req, res) { if (res.socket.destroyed) { return; } pendingClinets[req.clientId] = { res: res }; redisClient.subscribe(req.clientId); res.socket.on('error', function (err) { console.log(err); }); res.socket.on('close', function () { delete pendingClients[req.clientId]; redisClient.unsubscribe(req.clientId); }); }); server.listen(3000); function ServerResponse(socket) { this.socket = socket; } ServerResponse.prototype.write = function(data) { this.socket.write(data); } </code></pre> <p>Finally, here are my questions:</p> <ol> <li><p>How can I reduce the memory usage so that increase the concurrency farther? </p></li> <li><p>I'm really confused about how to calculate the memory usage of Node.js process. I know Node.js powered by Chrome V8, there is <strong>process.memoryUsage()</strong> api and it return three values: rss/heapTotal/heapUsed, what's the difference between them, which part should I concern more, and what's the exactly composition of the memory used by the Node.js process?</p></li> <li><p>I worried about memory leak even though I have done some tests and there don't seem to be a problem. Are there any points I should concern or any advises?</p></li> <li><p>I found a doc about <a href="https://developers.google.com/v8/design?hl=en#prop_access" rel="noreferrer">V8 <strong>hidden class</strong></a>, as it described, does that mean whenever I add a property named by <strong>clientId</strong> to my global object <strong>pendingClients</strong> just like my codes above, there will be a new hidden class be generated? Dose it will cause memory leak?</p></li> <li><p>I used <a href="https://github.com/c4milo/node-webkit-agent" rel="noreferrer">webkit-devtools-agent</a> to analyze heap map of the Node.js process. I started the process and took a heap snapshot, then I sent 10k requests to it and disconnected them later, after that I took a heap snapshot again. I used the <strong>comparison</strong> perspective to see the difference between these two snapshots. Here is what I got: <img src="https://i.stack.imgur.com/8ect0.png" alt="enter image description here"> Could anyone explain this? The number and size of (array)/(compiled code)/(string)/Command/Array increased a lot, what does this mean?</p></li> </ol> <p><strong>EDIT</strong>: How did I run the loading test?<br> 1. Firstly, I modified some parameters both on server machine and client machines（to achieve more than 60k concurrency need more than one client machine, because one machine only have 60k+ ports(represented by 16 bit) at most）<br> 1.1. Both one the server and the client machines, I modified the file descriptor use these commands in the shell where the test program will be run in:</p> <pre><code>ulimit -Hn 999999 ulimit -Sn 999999 </code></pre> <p>1.2. On the server machine, I also modified some net/tcp related kernel parameters, the most important ones are:</p> <pre><code>net.ipv4.tcp_mem = 786432 1048576 26777216 net.ipv4.tcp_rmem = 4096 16384 33554432 net.ipv4.tcp_wmem = 4096 16384 33554432 </code></pre> <p>1.3. As to the client machines:</p> <pre><code>net.ipv4.ip_local_port_range = 1024 65535 </code></pre> <p>2. Secondly, I wrote a custom simulate client program using Node.js, since most load test tools, ab, siege, etc, are for short connections, but I'm using long connections and have some special requirements.<br> 3. Then I started the server program on a single machine, and three client program on the other three separated machines.</p> <p><strong>EDIT</strong>: I did reach 250k concurrent connections on a single machine(2GB RAM), but turned out, it's not very meaningful and practical. Because when a connection connected, I just let the connection pending, nothing else. When I tried to sent response to them, the concurrency number dropped down to 150k around. As I calculated, there is about 4KB more memory usage per connection, I guess it's related to <strong>net.ipv4.tcp_wmem</strong> which I set to <strong>4096 16384 33554432</strong>, but even I modified it to smaller, nothing changed. I can't figure out why.</p> <p><strong>EDIT</strong>: Actually, now I'm more interested in how much memory per tcp connection uses and what's the exactly composition of the memory used by a single connection? According to my test data:</p> <blockquote> <p>150k concurrency consumed about 1800M RAM(from <strong>free -m</strong> output), and the Node.js process had about 600M RSS</p> </blockquote> <p>Then, I assumed this: </p> <blockquote> <ul> <li><p>(1800M - 600M) / 150k = 8k, this is the kernel TCP stack memory usage of a single connection, it consists of two parts: read buffer(4KB) + write buffer(4KB)(Actually, this doesn't match my setting of <strong>net.ipv4.tcp_rmem</strong> and <strong>net.ipv4.tcp_wmem</strong> above, how does the system determine how much memory to use for these buffers?)</p></li> <li><p>600M / 150k = 4k, this is the Node.js memory usage of a single connection </p></li> </ul> </blockquote> <p>Am I right? How can I reduce the memory usage in both aspects?</p> <p>If there are anywhere I didn't describe well, let me know, I'll refine it! Any explanations or advises will be appreciated, thanks!</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USAaron Wang
    1. USAaron Wang
    plurals
    1. This table or related slice is empty.
    1. PL
      singulars
      1. LTLinked
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POLong connections with Node.js, how to reduce memory usage and prevent memory leak? Also related with V8 and webkit-devtools
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POLong connections with Node.js, how to reduce memory usage and prevent memory leak? Also related with V8 and webkit-devtools
      1. USjfriend00
      1. VTFavorite
    3. VO
      singulars
      1. POLong connections with Node.js, how to reduce memory usage and prevent memory leak? Also related with V8 and webkit-devtools
      1. This table or related slice is empty.
      1. VTUpMod
    1. COFirst impression is that 250k on a machine with those specs is amazing. Might be time to focus on getting all those users you're worried about now. =P
      singulars
      1. POLong connections with Node.js, how to reduce memory usage and prevent memory leak? Also related with V8 and webkit-devtools
      1. UStehgeekmeister
    2. COHow are you measuring the number of concurrent connections?
      singulars
      1. POLong connections with Node.js, how to reduce memory usage and prevent memory leak? Also related with V8 and webkit-devtools
      1. UStehgeekmeister
    3. COSide note: it is best to stick to a smaller number of specific questions when asking on a stackexchange site. You'll get more answers that way.
      singulars
      1. POLong connections with Node.js, how to reduce memory usage and prevent memory leak? Also related with V8 and webkit-devtools
      1. UStehgeekmeister
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload