Databases
StackOverflow2013
Postsdbo
POCauses of Linux UDP packet drops
 

Note that there are some explanatory texts on larger screens.

plurals
  1. POCauses of Linux UDP packet drops
    primarykey
    data
    text
    <p>I have a Linux C++ application which receives sequenced UDP packets. Because of the sequencing, I can easily determine when a packet is lost or re-ordered, i.e. when a "gap" is encountered. The system has a recovery mechanism to handle gaps, however, it is best to avoid gaps in the first place. Using a simple libpcap-based packet sniffer, I have determined that there are no gaps in the data at the hardware level. However, I am seeing a lot of gaps in my application. This suggests the kernel is dropping packets; it is confirmed by looking at the <strong>/proc/net/snmp</strong> file. When my application encounters a gap, the <strong>Udp InErrors</strong> counter increases.</p> <p>At the system level, we have increased the max receive buffer:</p> <pre><code># sysctl net.core.rmem_max net.core.rmem_max = 33554432 </code></pre> <p>At the application level, we have increased the receive buffer size:</p> <pre><code>int sockbufsize = 33554432 int ret = setsockopt(my_socket_fd, SOL_SOCKET, SO_RCVBUF, (char *)&amp;sockbufsize, (int)sizeof(sockbufsize)); // check return code sockbufsize = 0; ret = getsockopt(my_socket_fd, SOL_SOCKET, SO_RCVBUF, (char*)&amp;sockbufsize, &amp;size); // print sockbufsize </code></pre> <p>After the call to getsockopt(), the printed value is always 2x what it is set to (67108864 in the example above), but I believe that is to be expected.</p> <p>I know that failure to consume data quickly enough can result in packet loss. However, all this application does is check the sequencing, then push the data into a queue; the actual processing is done in another thread. Furthermore, the machine is modern (dual Xeon X5560, 8 GB RAM) and very lightly loaded. We have literally dozens of identical applications receiving data at a <em>much higher rate</em> that do not experience this problem.</p> <p>Besides a too-slow consuming application, are there other reasons why the Linux kernel might drop UDP packets?</p> <p>FWIW, this is on CentOS 4, with kernel 2.6.9-89.0.25.ELlargesmp.</p>
    singulars
    1. PO
      singulars
      1. PTAnswer
    1. This table or related slice is empty.
    1. PTQuestion
    1. This table or related slice is empty.
    1. USMatt
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. PO
      singulars
      1. PTAnswer
    2. PO
      singulars
      1. PTAnswer
    3. PO
      singulars
      1. PTAnswer
    1. VO
      singulars
      1. POCauses of Linux UDP packet drops
      1. This table or related slice is empty.
      1. VTUpMod
    2. VO
      singulars
      1. POCauses of Linux UDP packet drops
      1. This table or related slice is empty.
      1. VTUpMod
    3. VO
      singulars
      1. POCauses of Linux UDP packet drops
      1. This table or related slice is empty.
      1. VTUpMod
    1. COCheck your cabling, bit errors happens, especially if you wrap your cable around an eletrical noise source. Could be a buggy driver/nic, see if you can turn off/on checksum offloading. Keep in mind that network elements such as switches can drop packets. Monitor your traffic with wireshark and look for suspicious things. You might need to isntrument your app to verify you're not having excessive delays anywhere (e.g. waiting on a mutex for too long), so you're sure you're reading fast enough.
      singulars
      1. POCauses of Linux UDP packet drops
      1. USnos
    2. COThe pcap application confirmed that the NIC is receiving all packets, so it's not a wiring or switch issue. As for the program itself, I'm using the exact same code shared by dozens of other applications. This connection is the only one with the problem.
      singulars
      1. POCauses of Linux UDP packet drops
      1. USMatt
    3. COcould the inbound checksums be incorrect?
      singulars
      1. POCauses of Linux UDP packet drops
      1. USAlnitak
 

Querying!

 
Guidance
A row detail

Detail views are divided into sections. All the information in the data section comes from columns in the selected row. The other sections display data from other, related rows.

Related data can be related in a to-one or a to-many fashion. Captions of data related in a to-many fashion link to a list view showing a filtered view of the table.

Try moving around until you find a non-empty to-many entry and click on the label to get to one. You can move back to the root by clicking on the database name in the header.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload