Databases
StackOverflow2013
Postsdbo
POIntermittently no data delivered through boost::asio / io completion port
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POIntermittently no data delivered through boost::asio / io completion port
    primarykey
    data
    text
    <h3>Problem</h3> <p>I am using <a href="http://www.boost.org/doc/libs/1_45_0/doc/html/boost_asio.html" rel="nofollow noreferrer">boost::asio</a> for a project where two processes on the same machine communicate using TCP/IP. One generates data to be read by the other, but I am encountering a problem where intermittently no data is being sent through the connection. I've boiled this down to a very simple example below, based on the <a href="http://www.boost.org/doc/libs/1_45_0/doc/html/boost_asio/example/echo/async_tcp_echo_server.cpp" rel="nofollow noreferrer">async tcp echo server example</a>.</p> <p>The processes (source code below) start out fine, delivering data at a fast rate from the sender to the receiver. Then all of a sudden, no data at all is delivered for about five seconds. Then data is delivered again until the next inexplicable pause. During these five seconds, the processes eat 0% CPU and no other processes seem to do anything in particular. The pause is always the same length - five seconds.</p> <p>I am trying to figure out how to get rid of these stalls and what causes them.</p> <h3>CPU usage during an entire run:</h3> <p><img src="https://i.stack.imgur.com/6gvjH.png" alt="CPU usage during a single run" /></p> <p>Notice how there are three dips of CPU usage in the middle of the run - a "run" is a single invocation of the server process and the client process. During these dips, no data was delivered. The number of dips and their timing differs between runs - some times no dips at all, some times many.</p> <p>I am able to affect the "probability" of these stalls by changing the size of the read buffer - for instance if I make the read buffer a multiple of the send chunk size it appears that this problem <em>almost</em> goes away, but not entirely.</p> <h3>Source and test description</h3> <p>I've compiled the below code with Visual Studio 2005, using Boost 1.43 and Boost 1.45. I have tested on Windows Vista 64 bit (on a quad-core) and Windows 7 64 bit (on both a quad-core and a dual-core).</p> <p>The server accepts a connection and then simply reads and discards data. Whenever a read is performed a new read is issued.</p> <p>The client connects to the server, then puts a bunch of packets into a send queue. After this it writes the packets one at the time. Whenever a write has completed, the next packet in the queue is written. A separate thread monitors the queue size and prints this to stdout every second. During the io stalls, the queue size remains exactly the same.</p> <p>I have tried to used scatter io (writing multiple packets in one system call), but the result is the same. If I disable IO completion ports in Boost using <code>BOOST_ASIO_DISABLE_IOCP</code>, the problem appears to go away but at the price of significantly lower throughput.</p> <pre><code>// Example is adapted from async_tcp_echo_server.cpp which is // Copyright (c) 2003-2010 Christopher M. Kohlhoff (chris at kohlhoff dot com) // // Start program with -s to start as the server #ifndef _WIN32_WINNT #define _WIN32_WINNT 0x0501 #endif #include &lt;iostream&gt; #include &lt;tchar.h&gt; #include &lt;boost/asio.hpp&gt; #include &lt;boost/bind.hpp&gt; #include &lt;boost/thread.hpp&gt; #define PORT "1234" using namespace boost::asio::ip; using namespace boost::system; class session { public: session(boost::asio::io_service&amp; io_service) : socket_(io_service) {} void do_read() { socket_.async_read_some(boost::asio::buffer(data_, max_length), boost::bind(&amp;session::handle_read, this, _1, _2)); } boost::asio::ip::tcp::socket&amp; socket() { return socket_; } protected: void handle_read(const error_code&amp; ec, size_t bytes_transferred) { if (!ec) { do_read(); } else { delete this; } } private: tcp::socket socket_; enum { max_length = 1024 }; char data_[max_length]; }; class server { public: explicit server(boost::asio::io_service&amp; io_service) : io_service_(io_service) , acceptor_(io_service, tcp::endpoint(tcp::v4(), atoi(PORT))) { session* new_session = new session(io_service_); acceptor_.async_accept(new_session-&gt;socket(), boost::bind(&amp;server::handle_accept, this, new_session, _1)); } void handle_accept(session* new_session, const error_code&amp; ec) { if (!ec) { new_session-&gt;do_read(); new_session = new session(io_service_); acceptor_.async_accept(new_session-&gt;socket(), boost::bind(&amp;server::handle_accept, this, new_session, _1)); } else { delete new_session; } } private: boost::asio::io_service&amp; io_service_; boost::asio::ip::tcp::acceptor acceptor_; }; class client { public: explicit client(boost::asio::io_service &amp;io_service) : io_service_(io_service) , socket_(io_service) , work_(new boost::asio::io_service::work(io_service)) { io_service_.post(boost::bind(&amp;client::do_init, this)); } ~client() { packet_thread_.join(); } protected: void do_init() { // Connect to the server tcp::resolver resolver(io_service_); tcp::resolver::query query(tcp::v4(), "localhost", PORT); tcp::resolver::iterator iterator = resolver.resolve(query); socket_.connect(*iterator); // Start packet generation thread packet_thread_.swap(boost::thread( boost::bind(&amp;client::generate_packets, this, 8000, 5000000))); } typedef std::vector&lt;unsigned char&gt; packet_type; typedef boost::shared_ptr&lt;packet_type&gt; packet_ptr; void generate_packets(long packet_size, long num_packets) { // Add a single dummy packet multiple times, then start writing packet_ptr buf(new packet_type(packet_size, 0)); write_queue_.insert(write_queue_.end(), num_packets, buf); queue_size = num_packets; do_write_nolock(); // Wait until all packets are sent. while (long queued = InterlockedExchangeAdd(&amp;queue_size, 0)) { std::cout &lt;&lt; "Queue size: " &lt;&lt; queued &lt;&lt; std::endl; Sleep(1000); } // Exit from run(), ignoring socket shutdown work_.reset(); } void do_write_nolock() { const packet_ptr &amp;p = write_queue_.front(); async_write(socket_, boost::asio::buffer(&amp;(*p)[0], p-&gt;size()), boost::bind(&amp;client::on_write, this, _1)); } void on_write(const error_code &amp;ec) { if (ec) { throw system_error(ec); } write_queue_.pop_front(); if (InterlockedDecrement(&amp;queue_size)) { do_write_nolock(); } } private: boost::asio::io_service &amp;io_service_; tcp::socket socket_; boost::shared_ptr&lt;boost::asio::io_service::work&gt; work_; long queue_size; std::list&lt;packet_ptr&gt; write_queue_; boost::thread packet_thread_; }; int _tmain(int argc, _TCHAR* argv[]) { try { boost::asio::io_service io_svc; bool is_server = argc &gt; 1 &amp;&amp; 0 == _tcsicmp(argv[1], _T("-s")); std::auto_ptr&lt;server&gt; s(is_server ? new server(io_svc) : 0); std::auto_ptr&lt;client&gt; c(is_server ? 0 : new client(io_svc)); io_svc.run(); } catch (std::exception&amp; e) { std::cerr &lt;&lt; "Exception: " &lt;&lt; e.what() &lt;&lt; "\n"; } return 0; } </code></pre> <p>So my question is basically:</p> <p><strong>How do I get rid of these stalls?</strong></p> <p><strong>What causes this to happen?</strong></p> <p><strong>Update:</strong> There appears to be some correlation with disk activity contrary to what I stated above, so it appears that if I start a large directory copy on the disk while the test is running this might increase the frequency of the io stalls. This could indicate that this is the <a href="http://www.microsoft.com/whdc/driver/priorityio.mspx" rel="nofollow noreferrer">Windows IO Prioritization</a> that kicks in? Since the pauses are always the same length, that does sound somewhat like a timeout somewhere in the OS io code...</p>
    singulars
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PTQuestion
    1. USvillintehaspam
    1. USvillintehaspam
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POIntermittently no data delivered through boost::asio / io completion port
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POIntermittently no data delivered through boost::asio / io completion port
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POIntermittently no data delivered through boost::asio / io completion port
      1. USMoo-Juice
      1. VTFavorite
    1. COI can't find it now, but when I was looking through the documentation before on ASIO there was some type of warning about using IOCP with ASIO.
      singulars
      1. POIntermittently no data delivered through boost::asio / io completion port
      1. USdiverscuba23
    2. CO@diverscuba23: If there were any problems with using IOCP with asio, it probably would not be the default method to use. There is a comment in the source indicating that there has previously been a problem with GetQueuedCompletionStatus not getting any completion packets if no timeout is used (on some versions of Windows), maybe this is what you mean?
      singulars
      1. POIntermittently no data delivered through boost::asio / io completion port
      1. USvillintehaspam
    3. COthat may be. As I said I could not find it when I went back to look for it, as it was just a link at the bottom of the page. I never bothered to follow the link, as it was for windows and I do my development for *NIX systems.
      singulars
      1. POIntermittently no data delivered through boost::asio / io completion port
      1. USdiverscuba23
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload