diff options
| author | Alexander Lobakin <alobakin@pm.me> | 2021-03-14 11:11:50 +0000 |
|---|---|---|
| committer | David S. Miller <davem@davemloft.net> | 2021-03-14 14:48:02 -0700 |
| commit | d206121faf8bb2239cd970af0bd32f5203780427 (patch) | |
| tree | c6bf285f9bd54f1a0b57d604226f34895f4034f2 | |
| parent | 59753ce8b196de60211a989c75ece8aeb0d9d57c (diff) | |
skbuff: micro-optimize {,__}skb_header_pointer()
{,__}skb_header_pointer() helpers exist mainly for preventing
accesses-beyond-end of the linear data.
In the vast majorify of cases, they bail out on the first condition.
All code going after is mostly a fallback.
Mark the most common branch as 'likely' one to move it in-line.
Also, skb_copy_bits() can return negative values only when the input
arguments are invalid, e.g. offset is greater than skb->len. It can
be safely marked as 'unlikely' branch, assuming that hotpath code
provides sane input to not fail here.
These two bump the throughput with a single Flow Dissector pass on
every packet (e.g. with RPS or driver that uses eth_get_headlen())
on 20 Mbps per flow/core.
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Signed-off-by: David S. Miller <davem@davemloft.net>
| -rw-r--r-- | include/linux/skbuff.h | 5 |
1 files changed, 2 insertions, 3 deletions
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h index 46c61e127e9f..ecc029674ae4 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -3680,11 +3680,10 @@ static inline void * __must_check __skb_header_pointer(const struct sk_buff *skb, int offset, int len, const void *data, int hlen, void *buffer) { - if (hlen - offset >= len) + if (likely(hlen - offset >= len)) return (void *)data + offset; - if (!skb || - skb_copy_bits(skb, offset, buffer, len) < 0) + if (!skb || unlikely(skb_copy_bits(skb, offset, buffer, len) < 0)) return NULL; return buffer; |