summaryrefslogtreecommitdiff
path: root/drivers/virtio/virtio_ring.c
AgeCommit message (Collapse)Author
2015-02-11virtio: Avoid possible kernel panic if DEBUG is enabled.Tetsuo Handa
The virtqueue_add() calls START_USE() upon entry. The virtqueue_kick() is called if vq->num_added == (1 << 16) - 1 before calling END_USE(). The virtqueue_kick_prepare() called via virtqueue_kick() calls START_USE() upon entry, and will call panic() if DEBUG is enabled. Move this virtqueue_kick() call to after END_USE() call. Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2015-01-21virtio_ring: coding style fixMichael S. Tsirkin
Most of our code has struct foo { } Fix one instances where ring is inconsistent. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2014-12-09virtio: make VIRTIO_F_VERSION_1 a transport bitMichael S. Tsirkin
Activate VIRTIO_F_VERSION_1 automatically unless legacy_only is set. Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2014-12-09virtio: allow transports to get avail/used addressesCornelia Huck
For virtio-1, we can theoretically have a more complex virtqueue layout with avail and used buffers not on a contiguous memory area with the descriptor table. For now, it's fine for a transport driver to stay with the old layout: It needs, however, a way to access the locations of the avail/used rings so it can register them with the host. Reviewed-by: David Hildenbrand <dahi@linux.vnet.ibm.com> Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2014-12-09virtio_ring: switch to new memory access APIsMichael S. Tsirkin
Use virtioXX_to_cpu and friends for access to all multibyte structures in memory. Note: this is intentionally mechanical. A follow-up patch will split long lines etc. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
2014-12-09virtio: use u32, not bitmap for featuresMichael S. Tsirkin
It seemed like a good idea to use bitmap for features in struct virtio_device, but it's actually a pain, and seems to become even more painful when we get more than 32 feature bits. Just change it to a u32 for now. Based on patch by Rusty. Suggested-by: David Hildenbrand <dahi@linux.vnet.ibm.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Cornelia Huck <cornelia.huck@de.ibm.com>
2014-09-13virtio_ring: unify direct/indirect code paths.Rusty Russell
virtqueue_add() populates the virtqueue descriptor table from the sgs given. If it uses an indirect descriptor table, then it puts a single descriptor in the descriptor table pointing to the kmalloc'ed indirect table where the sg is populated. Previously vring_add_indirect() did the allocation and the simple linear layout. We replace that with alloc_indirect() which allocates the indirect table then chains it like the normal descriptor table so we can reuse the core logic. This slows down pktgen by less than 1/2 a percent (which uses direct descriptors), as well as vring_bench, but it's far neater. vring_bench before: 1061485790-1104800648(1.08254e+09+/-6.6e+06)ns vring_bench after: 1125610268-1183528965(1.14172e+09+/-8e+06)ns pktgen before: 787781-796334(793165+/-2.4e+03)pps 365-369(367.5+/-1.2)Mb/sec (365530384-369498976(3.68028e+08+/-1.1e+06)bps) errors: 0 pktgen after: 779988-790404(786391+/-2.5e+03)pps 361-366(364.35+/-1.3)Mb/sec (361914432-366747456(3.64885e+08+/-1.2e+06)bps) errors: 0 Now, if we make force indirect descriptors by turning off any_header_sg in virtio_net.c: pktgen before: 713773-721062(718374+/-2.1e+03)pps 331-334(332.95+/-0.92)Mb/sec (331190672-334572768(3.33325e+08+/-9.6e+05)bps) errors: 0 pktgen after: 710542-719195(714898+/-2.4e+03)pps 329-333(331.15+/-1.1)Mb/sec (329691488-333706480(3.31713e+08+/-1.1e+06)bps) errors: 0 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Signed-off-by: David S. Miller <davem@davemloft.net>
2014-09-13virtio_ring: assume sgs are always well-formed.Rusty Russell
We used to have several callers which just used arrays. They're gone, so we can use sg_next() everywhere, simplifying the code. On my laptop, this slowed down vring_bench by 15%: vring_bench before: 936153354-967745359(9.44739e+08+/-6.1e+06)ns vring_bench after: 1061485790-1104800648(1.08254e+09+/-6.6e+06)ns However, a more realistic test using pktgen on a AMD FX(tm)-8320 saw a few percent improvement: pktgen before: 767390-792966(785159+/-6.5e+03)pps 356-367(363.75+/-2.9)Mb/sec (356068960-367936224(3.64314e+08+/-3e+06)bps) errors: 0 pktgen after: 787781-796334(793165+/-2.4e+03)pps 365-369(367.5+/-1.2)Mb/sec (365530384-369498976(3.68028e+08+/-1.1e+06)bps) errors: 0 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Signed-off-by: David S. Miller <davem@davemloft.net>
2014-04-28virtio: virtio_break_device() to mark all virtqueues broken.Rusty Russell
Good for post-apocalyptic scenarios, like S/390 hotplug. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2014-03-13virtio: fail adding buffer on broken queues.Rusty Russell
Heinz points out that adding buffers to a broken virtqueue (which should "never happen") still works. Failing allows drivers to detect and complain about broken devices. Now drivers are robust, we can add this extra check. Reported-by: Heinz Graalfs <graalfs@linux.vnet.ibm.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2014-03-13tools/virtio: fix missing kmemleak_ignore symbolJoel Stanley
In commit bb478d8b167 virtio_ring: plug kmemleak false positive, kmemleak_ignore was introduced. This broke compilation of virtio_test: cc -g -O2 -Wall -I. -I ../../usr/include/ -Wno-pointer-sign -fno-strict-overflow -fno-strict-aliasing -fno-common -MMD -U_FORTIFY_SOURCE -c -o virtio_ring.o ../../drivers/virtio/virtio_ring.c ../../drivers/virtio/virtio_ring.c: In function ‘vring_add_indirect’: ../../drivers/virtio/virtio_ring.c:177:2: warning: implicit declaration of function ‘kmemleak_ignore’ [-Wimplicit-function-declaration] kmemleak_ignore(desc); ^ cc virtio_test.o virtio_ring.o -o virtio_test virtio_ring.o: In function `vring_add_indirect': tools/virtio/../../drivers/virtio/virtio_ring.c:177: undefined reference to `kmemleak_ignore' Add a dummy header for tools/virtio, and add #incldue <linux/kmemleak.h> to drivers/virtio/virtio_ring.c so it is picked up by the userspace tools. Signed-off-by: Joel Stanley <joel@jms.id.au> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2013-11-05virtio_ring: adapt to notify() returning boolHeinz Graalfs
Correct if statement to check for bool returned by notify() (introduced in 5b1bf7cb673a). Signed-off-by: Heinz Graalfs <graalfs@linux.vnet.ibm.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2013-10-29virtio_ring: add new function virtqueue_is_broken()Heinz Graalfs
Add new function virtqueue_is_broken(). Callers of virtqueue_get_buf() should check for a broken queue. Signed-off-by: Heinz Graalfs <graalfs@linux.vnet.ibm.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2013-10-29virtio_ring: let virtqueue_{kick()/notify()} return a boolHeinz Graalfs
virtqueue_{kick()/notify()} should exploit the new host notification API. If the notify call returned with a negative value the host kick failed (e.g. a kick triggered after a device was hot-unplugged). In this case the virtqueue is set to 'broken' and false is returned, otherwise true. Signed-off-by: Heinz Graalfs <graalfs@linux.vnet.ibm.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2013-10-29virtio_ring: change host notification APIHeinz Graalfs
Currently a host kick error is silently ignored and not reflected in the virtqueue of a particular virtio device. Changing the notify API for guest->host notification seems to be one prerequisite in order to be able to handle such errors in the context where the kick is triggered. This patch changes the notify API. The notify function must return a bool return value. It returns false if the host notification failed. Signed-off-by: Heinz Graalfs <graalfs@linux.vnet.ibm.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2013-10-17virtio_ring: plug kmemleak false positive.Rusty Russell
unreferenced object 0xffff88003d467e20 (size 32): comm "softirq", pid 0, jiffies 4295197765 (age 6.364s) hex dump (first 32 bytes): 28 19 bf 3d 00 00 00 00 0c 00 00 00 01 00 01 00 (..=............ 02 dc 51 3c 00 00 00 00 56 00 00 00 00 00 00 00 ..Q<....V....... backtrace: [<ffffffff8152db19>] kmemleak_alloc+0x59/0xc0 [<ffffffff81102e93>] __kmalloc+0xf3/0x180 [<ffffffff812db5d6>] vring_add_indirect+0x36/0x280 [<ffffffff812dc59f>] virtqueue_add_outbuf+0xbf/0x4e0 [<ffffffff813a8b30>] start_xmit+0x1a0/0x3b0 [<ffffffff81445861>] dev_hard_start_xmit+0x2d1/0x4d0 [<ffffffff81460052>] sch_direct_xmit+0xf2/0x1c0 [<ffffffff81445c28>] dev_queue_xmit+0x1c8/0x460 [<ffffffff814e3187>] ip6_finish_output2+0x1d7/0x470 [<ffffffff814e34b0>] ip6_finish_output+0x90/0xb0 [<ffffffff814e3507>] ip6_output+0x37/0xb0 [<ffffffff815021eb>] igmp6_send+0x2db/0x470 [<ffffffff81502645>] igmp6_timer_handler+0x95/0xa0 [<ffffffff8104b57c>] call_timer_fn+0x2c/0x90 [<ffffffff8104b7ba>] run_timer_softirq+0x1da/0x1f0 [<ffffffff81045721>] __do_softirq+0xd1/0x1b0 Address gets embedded in a descriptor via virt_to_phys(). See detach_buf, which frees it: if (vq->vring.desc[i].flags & VRING_DESC_F_INDIRECT) kfree(phys_to_virt(vq->vring.desc[i].addr)); Reported-by: Christoph Paasch <christoph.paasch@uclouvain.be> Fix-suggested-by: Christoph Paasch <christoph.paasch@uclouvain.be> Typing-done-by: Rusty Russell <rusty@rustcorp.com.au> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2013-07-09Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-nextLinus Torvalds
Pull networking updates from David Miller: "This is a re-do of the net-next pull request for the current merge window. The only difference from the one I made the other day is that this has Eliezer's interface renames and the timeout handling changes made based upon your feedback, as well as a few bug fixes that have trickeled in. Highlights: 1) Low latency device polling, eliminating the cost of interrupt handling and context switches. Allows direct polling of a network device from socket operations, such as recvmsg() and poll(). Currently ixgbe, mlx4, and bnx2x support this feature. Full high level description, performance numbers, and design in commit 0a4db187a999 ("Merge branch 'll_poll'") From Eliezer Tamir. 2) With the routing cache removed, ip_check_mc_rcu() gets exercised more than ever before in the case where we have lots of multicast addresses. Use a hash table instead of a simple linked list, from Eric Dumazet. 3) Add driver for Atheros CQA98xx 802.11ac wireless devices, from Bartosz Markowski, Janusz Dziedzic, Kalle Valo, Marek Kwaczynski, Marek Puzyniak, Michal Kazior, and Sujith Manoharan. 4) Support reporting the TUN device persist flag to userspace, from Pavel Emelyanov. 5) Allow controlling network device VF link state using netlink, from Rony Efraim. 6) Support GRE tunneling in openvswitch, from Pravin B Shelar. 7) Adjust SOCK_MIN_RCVBUF and SOCK_MIN_SNDBUF for modern times, from Daniel Borkmann and Eric Dumazet. 8) Allow controlling of TCP quickack behavior on a per-route basis, from Cong Wang. 9) Several bug fixes and improvements to vxlan from Stephen Hemminger, Pravin B Shelar, and Mike Rapoport. In particular, support receiving on multiple UDP ports. 10) Major cleanups, particular in the area of debugging and cookie lifetime handline, to the SCTP protocol code. From Daniel Borkmann. 11) Allow packets to cross network namespaces when traversing tunnel devices. From Nicolas Dichtel. 12) Allow monitoring netlink traffic via AF_PACKET sockets, in a manner akin to how we monitor real network traffic via ptype_all. From Daniel Borkmann. 13) Several bug fixes and improvements for the new alx device driver, from Johannes Berg. 14) Fix scalability issues in the netem packet scheduler's time queue, by using an rbtree. From Eric Dumazet. 15) Several bug fixes in TCP loss recovery handling, from Yuchung Cheng. 16) Add support for GSO segmentation of MPLS packets, from Simon Horman. 17) Make network notifiers have a real data type for the opaque pointer that's passed into them. Use this to properly handle network device flag changes in arp_netdev_event(). From Jiri Pirko and Timo Teräs. 18) Convert several drivers over to module_pci_driver(), from Peter Huewe. 19) tcp_fixup_rcvbuf() can loop 500 times over loopback, just use a O(1) calculation instead. From Eric Dumazet. 20) Support setting of explicit tunnel peer addresses in ipv6, just like ipv4. From Nicolas Dichtel. 21) Protect x86 BPF JIT against spraying attacks, from Eric Dumazet. 22) Prevent a single high rate flow from overruning an individual cpu during RX packet processing via selective flow shedding. From Willem de Bruijn. 23) Don't use spinlocks in TCP md5 signing fast paths, from Eric Dumazet. 24) Don't just drop GSO packets which are above the TBF scheduler's burst limit, chop them up so they are in-bounds instead. Also from Eric Dumazet. 25) VLAN offloads are missed when configured on top of a bridge, fix from Vlad Yasevich. 26) Support IPV6 in ping sockets. From Lorenzo Colitti. 27) Receive flow steering targets should be updated at poll() time too, from David Majnemer. 28) Fix several corner case regressions in PMTU/redirect handling due to the routing cache removal, from Timo Teräs. 29) We have to be mindful of ipv4 mapped ipv6 sockets in upd_v6_push_pending_frames(). From Hannes Frederic Sowa. 30) Fix L2TP sequence number handling bugs, from James Chapman." * git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next: (1214 commits) drivers/net: caif: fix wrong rtnl_is_locked() usage drivers/net: enic: release rtnl_lock on error-path vhost-net: fix use-after-free in vhost_net_flush net: mv643xx_eth: do not use port number as platform device id net: sctp: confirm route during forward progress virtio_net: fix race in RX VQ processing virtio: support unlocked queue poll net/cadence/macb: fix bug/typo in extracting gem_irq_read_clear bit Documentation: Fix references to defunct linux-net@vger.kernel.org net/fs: change busy poll time accounting net: rename low latency sockets functions to busy poll bridge: fix some kernel warning in multicast timer sfc: Fix memory leak when discarding scattered packets sit: fix tunnel update via netlink dt:net:stmmac: Add dt specific phy reset callback support. dt:net:stmmac: Add support to dwmac version 3.610 and 3.710 dt:net:stmmac: Allocate platform data only if its NULL. net:stmmac: fix memleak in the open method ipv6: rt6_check_neigh should successfully verify neigh if no NUD information are available net: ipv6: fix wrong ping_v6_sendmsg return value ...
2013-07-09virtio: support unlocked queue pollMichael S. Tsirkin
This adds a way to check ring empty state after enable_cb outside any locks. Will be used by virtio_net. Note: there's room for more optimization: caller is likely to have a memory barrier already, which means we might be able to get rid of a barrier here. Deferring this optimization until we do some benchmarking. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2013-05-20virtio: remove virtqueue_add_buf().Rusty Russell
All users changed to virtqueue_add_sg() or virtqueue_add_outbuf/inbuf. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2013-03-20virtio_ring: virtqueue_add_outbuf / virtqueue_add_inbuf.Rusty Russell
These are specialized versions of virtqueue_add_buf(), which cover over 80% of cases and are far clearer. In particular, the scatterlists passed to these functions don't have to be clean (ie. we ignore end markers). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2013-03-20virtio_ring: virtqueue_add_sgs, to add multiple sgs.Rusty Russell
virtio_scsi can really use this, to avoid the current hack of copying the whole sg array. Some other things get slightly neater, too. This causes a slowdown in virtqueue_add_buf(), which is implemented as a wrapper. This is addressed in the next patches. for i in `seq 50`; do /usr/bin/time -f 'Wall time:%e' ./vringh_test --indirect --eventidx --parallel --fast-vringh; done 2>&1 | stats --trim-outliers: Before: Using CPUS 0 and 3 Guest: notified 0, pinged 39009-39063(39062) Host: notified 39009-39063(39062), pinged 0 Wall time:1.700000-1.950000(1.723542) After: Using CPUS 0 and 3 Guest: notified 0, pinged 39062-39063(39063) Host: notified 39062-39063(39063), pinged 0 Wall time:1.760000-2.220000(1.789167) Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Reviewed-by: Wanlong Gao <gaowanlong@cn.fujitsu.com> Reviewed-by: Asias He <asias@redhat.com>
2013-03-20virtio_ring: expose virtio barriers for use in vringh.Rusty Russell
The host side of ring needs this logic too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-12-18virtio: make virtqueue_add_buf() returning 0 on success, not capacity.Rusty Russell
Now noone relies on this behavior, we simplify virtqueue_add_buf() so it return 0 or -errno. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Acked-by: Michael S. Tsirkin <mst@redhat.com>
2012-12-18virtio: move queue_index and num_free fields into core struct virtqueue.Rusty Russell
They're generic concepts, so hoist them. This also avoids accessor functions (though kept around for merge with DaveM's net tree). This goes even further than Jason Wang's 17bb6d4088 patch ("virtio-ring: move queue_index to vring_virtqueue") which moved the queue_index from the specific transport. Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-10-22virtio: force vring descriptors to be allocated from lowmemWill Deacon
Virtio devices may attempt to add descriptors to a virtqueue from atomic context using GFP_ATOMIC allocation. This is problematic because such allocations can fall outside of the lowmem mapping, causing virt_to_phys to report bogus physical addresses which are subsequently passed to userspace via the buffers for the virtual device. This patch masks out __GFP_HIGH and __GFP_HIGHMEM from the requested flags when allocating descriptors for a virtqueue. If an atomic allocation is requested and later fails, we will return -ENOSPC which will be handled by the driver. Cc: stable@kernel.org Cc: Sasha Levin <levinsasha928@gmail.com> Signed-off-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-09-28virtio-ring: move queue_index to vring_virtqueueJason Wang
Instead of storing the queue index in transport-specific virtio structs, this patch moves them to vring_virtqueue and introduces an helper to get the value. This lets drivers simplify their management and tracing of virtqueues. Signed-off-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-01-28virtio: correct the memory barrier in virtqueue_kick_prepare()Jason Wang
Use virtio_mb() to make sure the available index to be exposed before checking the the avail event. Otherwise we may get stale value of avail event in guest and never kick the host after. Note: this fixes a bug introduced by ee7cd8981e15bcb365fc762afe3fc47b8242f630. Signed-off-by: Jason Wang <jasowang@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Cc: stable@kernel.org
2012-01-28virtio: fix typos of memory barriersJason Wang
Note: this fixes a bug introduced recently in 7b21e34fd1c272e3a8c3846168f2f6287a4cd72b. Signed-off-by: Jason Wang <jasowang@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-01-12virtio: add debugging if driver doesn't kick.Rusty Russell
Under the existing #ifdef DEBUG, check that they don't have more than 1/10 of a second between an add_buf() and a virtqueue_notify()/virtqueue_kick_prepare() call. We could get false positives on a really busy system, but good for development. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-01-12virtio: expose added descriptors immediately.Rusty Russell
A virtio driver does virtqueue_add_buf() multiple times before finally calling virtqueue_kick(); previously we only exposed the added buffers in the virtqueue_kick() call. This means we don't need a memory barrier in virtqueue_add_buf(), but it reduces concurrency as the device (ie. host) can't see the buffers until the kick. In the unusual (but now possible) case where a driver does add_buf() and get_buf() without doing a kick, we do need to insert one before our counter wraps. Otherwise we could wrap num_added, and later on not realize that we have passed the marker where we should have kicked. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-01-12virtio: avoid modulus operation.Rusty Russell
Since we know vq->vring.num is a power of 2, modulus is lazy (it's asserted in vring_new_virtqueue()). Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-01-12virtio: support unlocked queue kickRusty Russell
Based on patch by Christoph for virtio_blk speedup: Split virtqueue_kick to be able to do the actual notification outside the lock protecting the virtqueue. This patch was originally done by Stefan Hajnoczi, but I can't find the original one anymore and had to recreated it from memory. Pointers to the original or corrections for the commit message are welcome. Stefan's patch was here: https://github.com/stefanha/linux/commit/a6d06644e3a58e57a774e77d7dc34c4a5a2e7496 http://www.spinics.net/lists/linux-virtualization/msg14616.html Third time's the charm! Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2012-01-12virtio: rename virtqueue_add_buf_gfp to virtqueue_add_bufRusty Russell
Remove wrapper functions. This makes the allocation type explicit in all callers; I used GPF_KERNEL where it seemed obvious, left it at GFP_ATOMIC otherwise. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Reviewed-by: Christoph Hellwig <hch@lst.de>
2012-01-12virtio: document functions better.Rusty Russell
The old documentation is left over from when we used a structure with strategy pointers. And move the documentation to the C file as per kernel practice. Though I disagree... Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Reviewed-by: Christoph Hellwig <hch@lst.de>
2012-01-12virtio: harsher barriers for rpmsg.Rusty Russell
We were cheating with our barriers; using the smp ones rather than the real device ones. That was fine, until rpmsg came along, which is used to talk to a real device (a non-SMP CPU). Unfortunately, just putting back the real barriers (reverting d57ed95d) causes a performance regression on virtio-pci. In particular, Amos reports netbench's TCP_RR over virtio_net CPU utilization increased up to 35% while throughput went down by up to 14%. By comparison, this branch is in the noise. Reference: https://lkml.org/lkml/2011/12/11/22 Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2011-10-31virtio: Add module.h to drivers/virtio users.Paul Gortmaker
Up to now, the module.h header was as hard to keep out as sunlight. But we are cleaning that up. Fix the virtio users who simply expect module.h to be there in every C file. Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
2011-10-24Add ethtool -g support to virtio_netRick Jones
Add support for reporting ring sizes via ethtool -g to the virtio_net driver. Signed-off-by: Rick Jones <rick.jones2@hp.com> Acked-by: Rusty Russell <rusty@rustcorp.com.au> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2011-05-30virtio: add api for delayed callbacksMichael S. Tsirkin
Add an API that tells the other side that callbacks should be delayed until a lot of work has been done. Implement using the new event_idx feature. Note: it might seem advantageous to let the drivers ask for a callback after a specific capacity has been reached. However, as a single head can free many entries in the descriptor table, we don't really have a clue about capacity until get_buf is called. The API is the simplest to implement at the moment, we'll see what kind of hints drivers can pass when there's more than one user of the feature. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2011-05-30virtio_ring: support event idx featureMichael S. Tsirkin
Support for the new event idx feature: 1. When enabling interrupts, publish the current avail index value to the host to get interrupts on the next update. 2. Use the new avail_event feature to reduce the number of exits from the guest. Simple test with the simulator: [virtio]# time ./virtio_test spurious wakeus: 0x7 real 0m0.169s user 0m0.140s sys 0m0.019s [virtio]# time ./virtio_test --no-event-idx spurious wakeus: 0x11 real 0m0.649s user 0m0.295s sys 0m0.335s Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2011-04-21virtio: Decrement avail idx on buffer detachAmit Shah
When detaching a buffer from a vq, the avail.idx value should be decremented as well. This was noticed by hot-unplugging a virtio console port and then plugging in a new one on the same number (re-using the vqs which were just 'disowned'). qemu reported 'Guest moved used index from 0 to 256' when any IO was attempted on the new port. CC: stable@kernel.org Reported-by: juzhang <juzhang@redhat.com> Signed-off-by: Amit Shah <amit.shah@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-11-24virtio: return correct capacity to usersMichael S. Tsirkin
We can't rely on indirect buffers for capacity calculations because they need a memory allocation which might fail. In particular, virtio_net can get into this situation under stress, and it drops packets and performs badly. So return the number of buffers we can guarantee users. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Reported-By: Krishna Kumar2 <krkumar2@in.ibm.com>
2010-07-26virtio: fix oops on OOMMichael S. Tsirkin
virtio ring was changed to return an error code on OOM, but one caller was missed and still checks for vq->vring.num. The fix is just to check for <0 error code. Long term it might make sense to change goto add_head to just return an error on oom instead, but let's apply a minimal fix for 2.6.35. Reported-by: Chris Mason <chris.mason@oracle.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Tested-by: Chris Mason <chris.mason@oracle.com> Cc: stable@kernel.org # .34.x Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2010-06-23virtio: return ENOMEM on out of memoryMichael S. Tsirkin
add_buf returns ring size on out of memory, this is not what devices expect. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Acked-by: Amit Shah <amit.shah@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Cc: stable@kernel.org # .34.x
2010-05-19virtio: add_buf_gfpMichael S. Tsirkin
Add an add_buf variant that gets gfp parameter. Use that to allocate indirect buffers. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-05-19virtio_ring: remove a level of indirectionMichael S. Tsirkin
We have a single virtqueue_ops implementation, and it seems unlikely we'll get another one at this point. So let's remove an unnecessary level of indirection: it would be very easy to re-add it if another implementation surfaces. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-03-30include cleanup: Update gfp.h and slab.h includes to prepare for breaking ↵Tejun Heo
implicit slab.h inclusion from percpu.h percpu.h is included by sched.h and module.h and thus ends up being included when building most .c files. percpu.h includes slab.h which in turn includes gfp.h making everything defined by the two files universally available and complicating inclusion dependencies. percpu.h -> slab.h dependency is about to be removed. Prepare for this change by updating users of gfp and slab facilities include those headers directly instead of assuming availability. As this conversion needs to touch large number of source files, the following script is used as the basis of conversion. http://userweb.kernel.org/~tj/misc/slabh-sweep.py The script does the followings. * Scan files for gfp and slab usages and update includes such that only the necessary includes are there. ie. if only gfp is used, gfp.h, if slab is used, slab.h. * When the script inserts a new include, it looks at the include blocks and try to put the new include such that its order conforms to its surrounding. It's put in the include block which contains core kernel includes, in the same order that the rest are ordered - alphabetical, Christmas tree, rev-Xmas-tree or at the end if there doesn't seem to be any matching order. * If the script can't find a place to put a new include (mostly because the file doesn't have fitting include block), it prints out an error message indicating which .h file needs to be added to the file. The conversion was done in the following steps. 1. The initial automatic conversion of all .c files updated slightly over 4000 files, deleting around 700 includes and adding ~480 gfp.h and ~3000 slab.h inclusions. The script emitted errors for ~400 files. 2. Each error was manually checked. Some didn't need the inclusion, some needed manual addition while adding it to implementation .h or embedding .c file was more appropriate for others. This step added inclusions to around 150 files. 3. The script was run again and the output was compared to the edits from #2 to make sure no file was left behind. 4. Several build tests were done and a couple of problems were fixed. e.g. lib/decompress_*.c used malloc/free() wrappers around slab APIs requiring slab.h to be added manually. 5. The script was run on all .h files but without automatically editing them as sprinkling gfp.h and slab.h inclusions around .h files could easily lead to inclusion dependency hell. Most gfp.h inclusion directives were ignored as stuff from gfp.h was usually wildly available and often used in preprocessor macros. Each slab.h inclusion directive was examined and added manually as necessary. 6. percpu.h was updated not to include slab.h. 7. Build test were done on the following configurations and failures were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my distributed build env didn't work with gcov compiles) and a few more options had to be turned off depending on archs to make things build (like ipr on powerpc/64 which failed due to missing writeq). * x86 and x86_64 UP and SMP allmodconfig and a custom test config. * powerpc and powerpc64 SMP allmodconfig * sparc and sparc64 SMP allmodconfig * ia64 SMP allmodconfig * s390 SMP allmodconfig * alpha SMP allmodconfig * um on x86_64 SMP allmodconfig 8. percpu.h modifications were reverted so that it could be applied as a separate patch and serve as bisection point. Given the fact that I had only a couple of failures from tests on step 6, I'm fairly confident about the coverage of this conversion patch. If there is a breakage, it's likely to be something in one of the arch headers which should be easily discoverable easily on most builds of the specific arch. Signed-off-by: Tejun Heo <tj@kernel.org> Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-02-24virtio: Initialize vq->data entries to NULLAmit Shah
vq operations depend on vq->data[i] being NULL to figure out if the vq entry is in use (since the previous patch). We have to initialize them to NULL to ensure we don't work with junk data and trigger false BUG_ONs. Signed-off-by: Amit Shah <amit.shah@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Cc: Shirley Ma <xma@us.ibm.com>
2010-02-24virtio: Add ability to detach unused buffers from vringsShirley Ma
There's currently no way for a virtio driver to ask for unused buffers, so it has to keep a list itself to reclaim them at shutdown. This is redundant, since virtio_ring stores that information. So add a new hook to do this. Signed-off-by: Shirley Ma <xma@us.ibm.com> Signed-off-by: Amit Shah <amit.shah@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-24virtio: use smp_XX barriers on SMPMichael S. Tsirkin
virtio is communicating with a virtual "device" that actually runs on another host processor. Thus SMP barriers can be used to control memory access ordering. Where possible, we should use SMP barriers which are more lightweight than mandatory barriers, because mandatory barriers also control MMIO effects on accesses through relaxed memory I/O windows (which virtio does not use) (compare specifically smp_rmb and rmb on x86_64). We can't just use smp_mb and friends though, because we must force memory ordering even if guest is UP since host could be running on another CPU, but SMP barriers are defined to barrier() in that configuration. So, for UP fall back to mandatory barriers instead. Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2010-02-24virtio: remove bogus barriers from DEBUG version of virtio_ring.cRusty Russell
With DEBUG defined, we add an ->in_use flag to detect if the caller invokes two virtio methods in parallel. The barriers attempt to ensure timely update of the ->in_use flag. But they're voodoo: if we need these barriers it implies that the calling code doesn't have sufficient synchronization to ensure the code paths aren't invoked at the same time anyway, and we want to detect it. Also, adding barriers changes timing, so turning on debug has more chance of hiding real problems. Thanks to MST for drawing my attention to this code... CC: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
generated by cgit v1.2.3 (git 2.25.1) at 2024-12-01 11:37:20 +0000