Better visibility into Linux packet-dropping choices

Better visibility into Linux packet-dropping choices

[LWN subscriber-only content]

Welcome to

The following subscription-fully relate has been made on hand to you
by an LWN subscriber. Thousands of subscribers count on LWN for the
easiest recordsdata from the Linux and free utility communities. Whenever you happen to enjoy this
article, please relish in thoughts subscribing to LWN. Thank you
for visiting!

By Jonathan Corbet
February 25, 2022

Dropped packets are a fact of life in networking; there would maybe well also furthermore be any number
of the reason why a packet would maybe well also not survive the scramble to its destination.
Indeed, there are so many programs that a packet can meet its demise that it
would maybe well also furthermore be laborious for an administrator to repeat why packets are being dropped.
That, in flip, can form life complicated in times when users are complaining
about excessive packet-loss charges. Starting with 5.17, the kernel is getting
some improved instrumentation that ought to peaceable shed some mild on why the kernel
decides to route packets into the bit bucket.

This challenge is just not unique, and neither are attempts to address it. The
kernel for the time being incorporates a “drop_monitor” functionality that used to be
equipped within the two.6.30 kernel assist in 2009. Through the years, it has
won some functionality nonetheless has managed to remain totally and
diligently undocumented. This characteristic appears to be like to enhance a netlink API
that would maybe well ship notifications when packets are dropped. Those
notifications consist of an address within the kernel showing the set apart the
dedication to fall the packet used to be made, and would maybe maybe optionally consist of the
dropped packets themselves. User-dwelling code can flip the addresses
into characteristic names; determined administrators can then dig thru the
kernel source to are attempting to determine what’s occurring.

It appears to be like love there deserve to be a more in-depth formula. Because it happens, the starting up set apart
of the infrastructure to offer that better formula used to be contributed to 5.17 by
Menglong Dong. The inner kernel characteristic that frees the memory preserving
a packet is kfree_skb(); in 5.17, that characteristic has turn out to be:

    void kfree_skb_reason(struct sk_buff *skb, enum skb_drop_reason cause);

The cause argument is exclusive; it is some distance meant to negate why the
packet handed as skb has reached the tip of the road. This
knowledge is just not in fact functional to the kernel, nonetheless it has been
added to the reward kfree_skb tracepoint, making it on hand to
any program that connects to that tracepoint. Evaluation scripts can quick
print out why packets are being dropped; administrators would maybe well also furthermore set BPF
programs to, to illustrate, construct a histogram of causes for dropped

A unique model of kfree_skb() has also been added; it simply calls
kfree_skb_reason() with “unspecified” as the cause.

In 5.17, the utilize of this infrastructure is comparatively diminutive. There are about a
TCP-level fall locations which relish been instrumented with the unique call, in conjunction with
code that drops packets for being smaller than the TCP header dimension, not
being connected to an reward TCP socket, exhibiting checksum failures,
or having been explicitly dropped by an add-on socket filter program. The
UDP subsystem has also been enhanced to demonstrate these self same causes for dropped

The be troubled is situation to reinforce considerably in 5.18; patches already in
linux-subsequent add a series of unique causes. These doc packets dropped by the
netfilter subsystem, that dangle IP-header errors, or relish been identified as a
spoofed packet by the reverse-course filter (rp_filter) mechanism.
Administrators will be ready to behold IP packets which relish been dropped attributable to
an unsupported higher-level protocol. Reasons relish also been added for
UDP packets dropped by the IPSec XFRM
coverage or a lack of memory within the kernel.

There is yet
one other situation
of cause annotations that has been accredited, nonetheless which has
not yet stumbled on its formula into linux-subsequent; probabilities are that these will demonstrate up
in 5.18 as successfully. They extend the XFRM-coverage annotation to TCP, demonstrate
packets dropped attributable to missing or wrong MD5 hashes (which
are evidently peaceable a part in 2020), as successfully as these containing invalid
TCP flags or sequence
numbers delivery air of the contemporary TCP window. These patches also add unique
cases of the opposite causes considerable above; some eventualities would maybe well also furthermore be detected
in a variety of locations.

While the above situation of causes would maybe well also appear long, this work will be considered as
having accurate begun. In contemporary linux-subsequent, there are over 2,700 calls to
kfree_skb(), when in contrast with 18 to kfree_skb_reason().
That means that a variety of packets will peaceable be dropped for unspecified
causes. Nonetheless, this work represents a functional step forward, person that
ought to peaceable form many of the explanations for packet loss extra readily on hand to
scheme administrators.

The section that continues to be missing, pointless to order, is the patron-dwelling side. The
contemporary cause codes are all outlined
, which is just not section of the externally
on hand kernel API. Engaging them to a separate file under the
uapi directory would form them extra accessible to developers.
Additionally precious, pointless to order, would be to relish some documentation for this
mechanism and solutions to make utilize of it (and account for the outcomes), nonetheless even your
editor, typically cited for naive optimism, would maybe well also not be preserving his breath for
that to demonstrate up.

Meanwhile, though, a extraordinarily critical allotment of the kernel’s network functionality
is turning accurate into a diminutive extra transparent to users. That ought to form life
more straightforward for scheme administrators who will be ready to employ less time making an strive
to determine why packets don’t seem like making it thru the scheme.
Unfortunately, though, this work offers no assist for users who’re wondering
why their packets are disappearing someplace within the some distance reaches of the

(Log in to submit comments)

Read More



β€œSimplicity, patience, compassion.
These three are your greatest treasures.
Simple in actions and thoughts, you return to the source of being.
Patient with both friends and enemies,
you accord with the way things are.
Compassionate toward yourself,
you reconcile all beings in the world.”
― Lao Tzu, Tao Te Ching

you're currently offline