diff options
| author | Paolo Abeni <pabeni@redhat.com> | 2017-06-12 11:23:42 +0200 |
|---|---|---|
| committer | David S. Miller <davem@davemloft.net> | 2017-06-12 10:01:29 -0400 |
| commit | 0a463c78d25b9464b77311d9dda297550a2d6aa5 (patch) | |
| tree | 87152851619730150299fe4acf3a7a0eff8a6f4b /net/core | |
| parent | 3889a803e1da9bd7cd10d6504bf281ee7e55dfd6 (diff) | |
udp: avoid a cache miss on dequeue
Since UDP no more uses sk->destructor, we can clear completely
the skb head state before enqueuing. Amend and use
skb_release_head_state() for that.
All head states share a single cacheline, which is not
normally used/accesses on dequeue. We can avoid entirely accessing
such cacheline implementing and using in the UDP code a specialized
skb free helper which ignores the skb head state.
This saves a cacheline miss at skb deallocation time.
v1 -> v2:
replaced secpath_reset() with skb_release_head_state()
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Acked-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Diffstat (limited to 'net/core')
| -rw-r--r-- | net/core/skbuff.c | 24 |
1 files changed, 20 insertions, 4 deletions
diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 747263c49838..304602784c3b 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -643,12 +643,10 @@ fastpath: kmem_cache_free(skbuff_fclone_cache, fclones); } -static void skb_release_head_state(struct sk_buff *skb) +void skb_release_head_state(struct sk_buff *skb) { skb_dst_drop(skb); -#ifdef CONFIG_XFRM - secpath_put(skb->sp); -#endif + secpath_reset(skb); if (skb->destructor) { WARN_ON(in_irq()); skb->destructor(skb); @@ -751,6 +749,24 @@ void consume_skb(struct sk_buff *skb) } EXPORT_SYMBOL(consume_skb); +/** + * consume_stateless_skb - free an skbuff, assuming it is stateless + * @skb: buffer to free + * + * Works like consume_skb(), but this variant assumes that all the head + * states have been already dropped. + */ +void consume_stateless_skb(struct sk_buff *skb) +{ + if (!skb_unref(skb)) + return; + + trace_consume_skb(skb); + if (likely(skb->head)) + skb_release_data(skb); + kfree_skbmem(skb); +} + void __kfree_skb_flush(void) { struct napi_alloc_cache *nc = this_cpu_ptr(&napi_alloc_cache); |