Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>This is what the Linux man page says about <code>write</code> and <code>EPIPE</code>:</p> <pre><code> EPIPE fd is connected to a pipe or socket whose reading end is closed. When this happens the writing process will also receive a SIG- PIPE signal. (Thus, the write return value is seen only if the program catches, blocks or ignores this signal.) </code></pre> <p>When Linux is using a <code>pipe</code> or a <code>socketpair</code>, it can and will check the <em>reading end</em> of the pair, as these two programs would demonstrate:</p> <pre><code>void test_socketpair () { int pair[2]; socketpair(PF_LOCAL, SOCK_STREAM, 0, pair); close(pair[0]); if (send(pair[1], "a", 1, MSG_NOSIGNAL) &lt; 0) perror("send"); } void test_pipe () { int pair[2]; pipe(pair); close(pair[0]); signal(SIGPIPE, SIG_IGN); if (write(pair[1], "a", 1) &lt; 0) perror("send"); signal(SIGPIPE, SIG_DFL); } </code></pre> <p>Linux is able to do so, because the kernel has innate knowledge about the other end of the pipe or connected pair. However, when using <code>connect</code>, the state about the socket is maintained by the protocol stack. Your test demonstrates this behavior, but below is a program that does it all in a single thread, similar to the two tests above:</p> <pre><code>int a_sock = socket(PF_INET, SOCK_STREAM, 0); const int one = 1; setsockopt(a_sock, SOL_SOCKET, SO_REUSEADDR, &amp;one, sizeof(one)); struct sockaddr_in a_sin = {0}; a_sin.sin_port = htons(4321); a_sin.sin_family = AF_INET; a_sin.sin_addr.s_addr = htonl(INADDR_LOOPBACK); bind(a_sock, (struct sockaddr *)&amp;a_sin, sizeof(a_sin)); listen(a_sock, 1); int c_sock = socket(PF_INET, SOCK_STREAM, 0); fcntl(c_sock, F_SETFL, fcntl(c_sock, F_GETFL, 0)|O_NONBLOCK); connect(c_sock, (struct sockaddr *)&amp;a_sin, sizeof(a_sin)); fcntl(c_sock, F_SETFL, fcntl(c_sock, F_GETFL, 0)&amp;~O_NONBLOCK); struct sockaddr_in s_sin = {0}; socklen_t s_sinlen = sizeof(s_sin); int s_sock = accept(a_sock, (struct sockaddr *)&amp;s_sin, &amp;s_sinlen); struct pollfd c_pfd = { c_sock, POLLOUT, 0 }; if (poll(&amp;c_pfd, 1, -1) != 1) perror("poll"); int erropt = -1; socklen_t errlen = sizeof(erropt); getsockopt(c_sock, SOL_SOCKET, SO_ERROR, &amp;erropt, &amp;errlen); if (erropt != 0) { errno = erropt; perror("connect"); } puts("P|Recv-Q|Send-Q|Local Address|Foreign Address|State|"); char cmd[256]; snprintf(cmd, sizeof(cmd), "netstat -tn | grep ':%hu ' | sed 's/ */|/g'", ntohs(s_sin.sin_port)); puts("before close on client"); system(cmd); close(c_sock); puts("after close on client"); system(cmd); if (send(s_sock, "a", 1, MSG_NOSIGNAL) &lt; 0) perror("send"); puts("after send on server"); system(cmd); puts("end of test"); sleep(5); </code></pre> <p>If you run the above program, you will get output similar to this:</p> <pre><code>P|Recv-Q|Send-Q|Local Address|Foreign Address|State| before close on client tcp|0|0|127.0.0.1:35790|127.0.0.1:4321|ESTABLISHED| tcp|0|0|127.0.0.1:4321|127.0.0.1:35790|ESTABLISHED| after close on client tcp|0|0|127.0.0.1:35790|127.0.0.1:4321|FIN_WAIT2| tcp|1|0|127.0.0.1:4321|127.0.0.1:35790|CLOSE_WAIT| after send on server end of test </code></pre> <p>This shows it took one <code>write</code> for the sockets to transition to the <code>CLOSED</code> states. To find out why this occurred, a TCP dump of the transaction can be useful:</p> <pre><code>16:45:28 127.0.0.1 &gt; 127.0.0.1 .809578 IP .35790 &gt; .4321: S 1062313174:1062313174(0) win 32792 &lt;mss 16396,sackOK,timestamp 3915671437 0,nop,wscale 7&gt; .809715 IP .4321 &gt; .35790: S 1068622806:1068622806(0) ack 1062313175 win 32768 &lt;mss 16396,sackOK,timestamp 3915671437 3915671437,nop,wscale 7&gt; .809583 IP .35790 &gt; .4321: . ack 1 win 257 &lt;nop,nop,timestamp 3915671437 3915671437&gt; .840364 IP .35790 &gt; .4321: F 1:1(0) ack 1 win 257 &lt;nop,nop,timestamp 3915671468 3915671437&gt; .841170 IP .4321 &gt; .35790: . ack 2 win 256 &lt;nop,nop,timestamp 3915671469 3915671468&gt; .865792 IP .4321 &gt; .35790: P 1:2(1) ack 2 win 256 &lt;nop,nop,timestamp 3915671493 3915671468&gt; .865809 IP .35790 &gt; .4321: R 1062313176:1062313176(0) win 0 </code></pre> <p>The first three lines represent the 3-way handshake. The fourth line is the <code>FIN</code> packet the client sends to the server, and the fifth line is the <code>ACK</code> from the server, acknowledging receipt. The sixth line is the server trying to send 1 byte of data to the client with the <code>PUSH</code> flag set. The final line is the client <code>RESET</code> packet, which causes the TCP state for the connection to be freed, and is why the third <code>netstat</code> command did not result in any output in the test above.</p> <p>So, the server doesn't know the client will reset the connection until after it tries to send some data to it. The reason for the reset is because the client called <code>close</code>, instead of something else.</p> <p>The server cannot know for certain what system call the client has actually issued, it can only follow the TCP state. For example, we could replace the <code>close</code> call with a call to <code>shutdown</code> instead.</p> <pre><code>//close(c_sock); shutdown(c_sock, SHUT_WR); </code></pre> <p>The difference between <code>shutdown</code> and <code>close</code> is that <code>shutdown</code> only governs the state of the connection, while <code>close</code> also governs the state of the <em>file descriptor</em> that represents the socket. A <code>shutdown</code> will not <code>close</code> a socket.</p> <p>The output will be different with the <code>shutdown</code> change:</p> <pre><code>P|Recv-Q|Send-Q|Local Address|Foreign Address|State| before close on client tcp|0|0|127.0.0.1:4321|127.0.0.1:56355|ESTABLISHED| tcp|0|0|127.0.0.1:56355|127.0.0.1:4321|ESTABLISHED| after close on client tcp|1|0|127.0.0.1:4321|127.0.0.1:56355|CLOSE_WAIT| tcp|0|0|127.0.0.1:56355|127.0.0.1:4321|FIN_WAIT2| after send on server tcp|1|0|127.0.0.1:4321|127.0.0.1:56355|CLOSE_WAIT| tcp|1|0|127.0.0.1:56355|127.0.0.1:4321|FIN_WAIT2| end of test </code></pre> <p>The TCP dump will show also show something different:</p> <pre><code>17:09:18 127.0.0.1 &gt; 127.0.0.1 .722520 IP .56355 &gt; .4321: S 2558095134:2558095134(0) win 32792 &lt;mss 16396,sackOK,timestamp 3917101399 0,nop,wscale 7&gt; .722594 IP .4321 &gt; .56355: S 2563862019:2563862019(0) ack 2558095135 win 32768 &lt;mss 16396,sackOK,timestamp 3917101399 3917101399,nop,wscale 7&gt; .722615 IP .56355 &gt; .4321: . ack 1 win 257 &lt;nop,nop,timestamp 3917101399 3917101399&gt; .748838 IP .56355 &gt; .4321: F 1:1(0) ack 1 win 257 &lt;nop,nop,timestamp 3917101425 3917101399&gt; .748956 IP .4321 &gt; .56355: . ack 2 win 256 &lt;nop,nop,timestamp 3917101426 3917101425&gt; .764894 IP .4321 &gt; .56355: P 1:2(1) ack 2 win 256 &lt;nop,nop,timestamp 3917101442 3917101425&gt; .764903 IP .56355 &gt; .4321: . ack 2 win 257 &lt;nop,nop,timestamp 3917101442 3917101442&gt; 17:09:23 .786921 IP .56355 &gt; .4321: R 2:2(0) ack 2 win 257 &lt;nop,nop,timestamp 3917106464 3917101442&gt; </code></pre> <p>Notice the reset at the end comes 5 seconds after the last <code>ACK</code> packet. This reset is due to the program shutting down without properly closing the sockets. It is the <code>ACK</code> packet from the client to the server before the reset that is different than before. This is the indication that the client did not use <code>close</code>. In TCP, the <code>FIN</code> indication is really an indication that there is no more data to be sent. But since a TCP connection is bi-directional, the server that receives the <code>FIN</code> assumes the client can still receive data. In the case above, the client in fact does accept the data.</p> <p>Whether the client uses <code>close</code> or <code>SHUT_WR</code> to issue a <code>FIN</code>, in either case you can detect the arrival of the <code>FIN</code> by polling on the server socket for a readable event. If after calling <code>read</code> the result is <code>0</code>, then you know the <code>FIN</code> has arrived, and you can do what you wish with that information.</p> <pre><code>struct pollfd s_pfd = { s_sock, POLLIN|POLLOUT, 0 }; if (poll(&amp;s_pfd, 1, -1) != 1) perror("poll"); if (s_pfd.revents|POLLIN) { char c; int r; while ((r = recv(s_sock, &amp;c, 1, MSG_DONTWAIT)) == 1) {} if (r == 0) { /*...FIN received...*/ } else if (errno == EAGAIN) { /*...no more data to read for now...*/ } else { /*...some other error...*/ perror("recv"); } } </code></pre> <p>Now, it is trivially true that if the server issues <code>SHUT_WR</code> with <code>shutdown</code> before it tries to do a write, it will in fact get the <code>EPIPE</code> error.</p> <pre><code>shutdown(s_sock, SHUT_WR); if (send(s_sock, "a", 1, MSG_NOSIGNAL) &lt; 0) perror("send"); </code></pre> <p>If, instead, you want the client to indicate an immediate reset to the server, you can force that to happen on most TCP stacks by enabling the linger option, with a linger timeout of <code>0</code> prior to calling <code>close</code>.</p> <pre><code>struct linger lo = { 1, 0 }; setsockopt(c_sock, SOL_SOCKET, SO_LINGER, &amp;lo, sizeof(lo)); close(c_sock); </code></pre> <p>With the above change, the output of the program becomes:</p> <pre><code>P|Recv-Q|Send-Q|Local Address|Foreign Address|State| before close on client tcp|0|0|127.0.0.1:35043|127.0.0.1:4321|ESTABLISHED| tcp|0|0|127.0.0.1:4321|127.0.0.1:35043|ESTABLISHED| after close on client send: Connection reset by peer after send on server end of test </code></pre> <p>The <code>send</code> gets an immediate error in this case, but it is not <code>EPIPE</code>, it is <code>ECONNRESET</code>. The TCP dump reflects this as well:</p> <pre><code>17:44:21 127.0.0.1 &gt; 127.0.0.1 .662163 IP .35043 &gt; .4321: S 498617888:498617888(0) win 32792 &lt;mss 16396,sackOK,timestamp 3919204411 0,nop,wscale 7&gt; .662176 IP .4321 &gt; .35043: S 497680435:497680435(0) ack 498617889 win 32768 &lt;mss 16396,sackOK,timestamp 3919204411 3919204411,nop,wscale 7&gt; .662184 IP .35043 &gt; .4321: . ack 1 win 257 &lt;nop,nop,timestamp 3919204411 3919204411&gt; .691207 IP .35043 &gt; .4321: R 1:1(0) ack 1 win 257 &lt;nop,nop,timestamp 3919204440 3919204411&gt; </code></pre> <p>The <code>RESET</code> packet comes right after the 3-way handshake completes. However, using this option has its dangers. If the other end has unread data in the socket buffer when the <code>RESET</code> arrives, that data will be purged, causing the data to be lost. Forcing a <code>RESET</code> to be sent is usually used in request/response style protocols. The sender of the request can know there can be no data lost when it receives the entire response to its request. Then, it is safe for the request sender to force a <code>RESET</code> to be sent on the connection.</p>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    3. VO
      singulars
      1. This table or related slice is empty.
    1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload