StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

POHow to most efficently handle large numbers of file descriptors?
text
Body
copied!<p>There appear to be several options available to programs that handle large numbers of socket connections (such as web services, p2p systems, etc).</p> <ol> <li>Spawn a separate thread to handle I/O for each socket.</li> <li>Use the <a href="http://linux.die.net/man/2/select" rel="nofollow noreferrer">select</a> system call to multiplex the I/O into a single thread.</li> <li>Use the <a href="http://linux.die.net/man/2/poll" rel="nofollow noreferrer">poll</a> system call to multiplex the I/O (replacing the select).</li> <li>Use the <a href="http://linux.die.net/man/4/epoll" rel="nofollow noreferrer">epoll</a> system calls to avoid having to repeatedly send sockets fd's through the user/system boundaries.</li> <li>Spawn a number of I/O threads that each multiplex a relatively small set of the total number of connections using the poll API.</li> <li>As per #5 except using the epoll API to create a separate epoll object for each independent I/O thread.</li> </ol> <p>On a multicore CPU I would expect that #5 or #6 would have the best performance, but I don't have any hard data backing this up. Searching the web turned up <a href="http://lse.sourceforge.net/epoll/index.html" rel="nofollow noreferrer">this</a> page describing the experiences of the author testing approaches #2, #3 and #4 above. Unfortunately this web page appears to be around 7 years old with no obvious recent updates to be found.</p> <p>So my question is which of these approaches have people found to be most efficient and/or is there another approach that works better than any of those listed above? References to real life graphs, whitepapers and/or web available writeups will be appreciated.</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload