summaryrefslogtreecommitdiff
path: root/drivers/pci/switch
AgeCommit message (Collapse)Author
2019-01-08cross-tree: phase out dma_zalloc_coherent()Luis Chamberlain
We already need to zero out memory for dma_alloc_coherent(), as such using dma_zalloc_coherent() is superflous. Phase it out. This change was generated with the following Coccinelle SmPL patch: @ replace_dma_zalloc_coherent @ expression dev, size, data, handle, flags; @@ -dma_zalloc_coherent(dev, size, handle, flags) +dma_alloc_coherent(dev, size, handle, flags) Suggested-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> [hch: re-ran the script on the latest tree] Signed-off-by: Christoph Hellwig <hch@lst.de>
2018-12-13switchtec: Add MRPC DMA mode supportWesley Sheng
MRPC normal mode requires the host to read the MRPC command status and output data from BAR. This results in high latency responses from the Memory Read TLP and potential Completion Timeout (CTO). Add support for MRPC DMA mode, including related macro definitions and data structures and code to: * Retrieve MRPC DMA mode version from adapter firmware * Allocate DMA buffer, register ISR, and enable DMA during init * Check MRPC execution status and get execution results from DMA buffer * Release DMA buffer and disable DMA function when unloading module MRPC DMA mode is a new feature of firmware, and the driver will fall back to MRPC normal mode if there is no support in the legacy firmware. Add a module parameter, "use_dma_mrpc", to select between MRPC DMA mode and MRPC normal mode. Since the driver automatically detects DMA support in the firmware, this parameter is just for debugging and testing. Include <linux/io-64-nonatomic-lo-hi.h> so that readq/writeq is replaced by two readl/writel on systems that do not support it. Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com> [bhelgaas: changelog, simplify dma_ver check] Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
2018-12-13switchtec: Improve MRPC efficiency by enabling write combiningKelvin Cao
The MRPC Input buffer is mostly memory without any side effects, so we can improve the access time by enabling write combining on this region of the BAR. In a few places, we still need to flush the WC buffer. To do this, we simply read from the Outbound Doorbell register because reads to this register are processed by low latency hardware. Signed-off-by: Kelvin Cao <kelvin.cao@microchip.com> Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
2018-12-12switchtec: Fix SWITCHTEC_IOCTL_EVENT_IDX_ALL flags overwriteJoey Zhang
In the ioctl_event_ctl() SWITCHTEC_IOCTL_EVENT_IDX_ALL case, we call event_ctl() several times with the same "ctl" struct. Each call clobbers ctl.flags, which leads to the problem that we may not actually enable or disable all events as the user requested. Preserve the event flag value with a temporary variable. Fixes: 52eabba5bcdb ("switchtec: Add IOCTLs to the Switchtec driver") Signed-off-by: Joey Zhang <joey.zhang@microchip.com> Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com> [bhelgaas: changelog] Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
2018-12-12switchtec: Set DMA coherent maskBoris Glimcher
Switchtec hardware supports 64-bit DMA, so set the correct DMA mask. This allows the CMA to allocate larger buffers for memory windows. Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com> Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
2018-12-12switchtec: Remove immediate status check after submitting MRPC commandKelvin Cao
After submitting a Firmware Download MRPC command, Switchtec firmware will delay Management EP BAR MemRd TLP responses by more than 10ms. This is a firmware limitation. Delayed MemRd completions are a problem for systems with a low Completion Timeout (CTO). The current driver checks the MRPC status immediately after submitting an MRPC command, which results in a delayed MemRd completion that may cause a Completion Timeout. Remove the immediate status check and rely on the check after receiving an interrupt or timing out. This is only a software workaround to the READ issue and a proper fix of this should be done in firmware. Fixes: 080b47def5e5 ("MicroSemi Switchtec management interface driver") Signed-off-by: Kelvin Cao <kelvin.cao@microchip.com> Signed-off-by: Wesley Sheng <wesley.sheng@microchip.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
2018-09-11switchtec: Fix Spectre v1 vulnerabilityGustavo A. R. Silva
p.port can is indirectly controlled by user-space, hence leading to a potential exploitation of the Spectre variant 1 vulnerability. This issue was detected with the help of Smatch: drivers/pci/switch/switchtec.c:912 ioctl_port_to_pff() warn: potential spectre issue 'pcfg->dsp_pff_inst_id' [r] Fix this by sanitizing p.port before using it to index pcfg->dsp_pff_inst_id Notice that given that speculation windows are large, the policy is to kill the speculation on the first load and not worry if it can be completed with a dependent load/store [1]. [1] https://marc.info/?l=linux-kernel&m=152449131114778&w=2 Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Logan Gunthorpe <logang@deltatee.com> Cc: stable@vger.kernel.org
2018-06-29switchtec: Use generic PCI Vendor ID and Class CodeDoug Meyer
Move the Microsemi Switchtec PCI Vendor ID (same as PCI_VENDOR_ID_PMC_Sierra) to pci_ids.h. Also, replace Microsemi class constants with the standard PCI definitions. Signed-off-by: Doug Meyer <dmeyer@gigaio.com> [bhelgaas: restore SPDX (I assume it was removed by mistake), remove device ID definitions] Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
2018-02-11vfs: do bulk POLL* -> EPOLL* replacementLinus Torvalds
This is the mindless scripted replacement of kernel use of POLL* variables as described by Al, done by this script: for V in IN OUT PRI ERR RDNORM RDBAND WRNORM WRBAND HUP RDHUP NVAL MSG; do L=`git grep -l -w POLL$V | grep -v '^t' | grep -v /um/ | grep -v '^sa' | grep -v '/poll.h$'|grep -v '^D'` for f in $L; do sed -i "-es/^\([^\"]*\)\(\<POLL$V\>\)/\\1E\\2/" $f; done done with de-mangling cleanups yet to come. NOTE! On almost all architectures, the EPOLL* constants have the same values as the POLL* constants do. But they keyword here is "almost". For various bad reasons they aren't the same, and epoll() doesn't actually work quite correctly in some cases due to this on Sparc et al. The next patch from Al will sort out the final differences, and we should be all done. Scripted-by: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-02-06Merge tag 'pci-v4.16-changes' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci Pull PCI updates from Bjorn Helgaas: - skip AER driver error recovery callbacks for correctable errors reported via ACPI APEI, as we already do for errors reported via the native path (Tyler Baicar) - fix DPC shared interrupt handling (Alex Williamson) - print full DPC interrupt number (Keith Busch) - enable DPC only if AER is available (Keith Busch) - simplify DPC code (Bjorn Helgaas) - calculate ASPM L1 substate parameter instead of hardcoding it (Bjorn Helgaas) - enable Latency Tolerance Reporting for ASPM L1 substates (Bjorn Helgaas) - move ASPM internal interfaces out of public header (Bjorn Helgaas) - allow hot-removal of VGA devices (Mika Westerberg) - speed up unplug and shutdown by assuming Thunderbolt controllers don't support Command Completed events (Lukas Wunner) - add AtomicOps support for GPU and Infiniband drivers (Felix Kuehling, Jay Cornwall) - expose "ari_enabled" in sysfs to help NIC naming (Stuart Hayes) - clean up PCI DMA interface usage (Christoph Hellwig) - remove PCI pool API (replaced with DMA pool) (Romain Perier) - deprecate pci_get_bus_and_slot(), which assumed PCI domain 0 (Sinan Kaya) - move DT PCI code from drivers/of/ to drivers/pci/ (Rob Herring) - add PCI-specific wrappers for dev_info(), etc (Frederick Lawler) - remove warnings on sysfs mmap failure (Bjorn Helgaas) - quiet ROM validation messages (Alex Deucher) - remove redundant memory alloc failure messages (Markus Elfring) - fill in types for compile-time VGA and other I/O port resources (Bjorn Helgaas) - make "pci=pcie_scan_all" work for Root Ports as well as Downstream Ports to help AmigaOne X1000 (Bjorn Helgaas) - add SPDX tags to all PCI files (Bjorn Helgaas) - quirk Marvell 9128 DMA aliases (Alex Williamson) - quirk broken INTx disable on Ceton InfiniTV4 (Bjorn Helgaas) - fix CONFIG_PCI=n build by adding dummy pci_irqd_intx_xlate() (Niklas Cassel) - use DMA API to get MSI address for DesignWare IP (Niklas Cassel) - fix endpoint-mode DMA mask configuration (Kishon Vijay Abraham I) - fix ARTPEC-6 incorrect IS_ERR() usage (Wei Yongjun) - add support for ARTPEC-7 SoC (Niklas Cassel) - add endpoint-mode support for ARTPEC (Niklas Cassel) - add Cadence PCIe host and endpoint controller driver (Cyrille Pitchen) - handle multiple INTx status bits being set in dra7xx (Vignesh R) - translate dra7xx hwirq range to fix INTD handling (Vignesh R) - remove deprecated Exynos PHY initialization code (Jaehoon Chung) - fix MSI erratum workaround for HiSilicon Hip06/Hip07 (Dongdong Liu) - fix NULL pointer dereference in iProc BCMA driver (Ray Jui) - fix Keystone interrupt-controller-node lookup (Johan Hovold) - constify qcom driver structures (Julia Lawall) - rework Tegra config space mapping to increase space available for endpoints (Vidya Sagar) - simplify Tegra driver by using bus->sysdata (Manikanta Maddireddy) - remove PCI_REASSIGN_ALL_BUS usage on Tegra (Manikanta Maddireddy) - add support for Global Fabric Manager Server (GFMS) event to Microsemi Switchtec switch driver (Logan Gunthorpe) - add IDs for Switchtec PSX 24xG3 and PSX 48xG3 (Kelvin Cao) * tag 'pci-v4.16-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (140 commits) PCI: cadence: Add EndPoint Controller driver for Cadence PCIe controller dt-bindings: PCI: cadence: Add DT bindings for Cadence PCIe endpoint controller PCI: endpoint: Fix EPF device name to support multi-function devices PCI: endpoint: Add the function number as argument to EPC ops PCI: cadence: Add host driver for Cadence PCIe controller dt-bindings: PCI: cadence: Add DT bindings for Cadence PCIe host controller PCI: Add vendor ID for Cadence PCI: Add generic function to probe PCI host controllers PCI: generic: fix missing call of pci_free_resource_list() PCI: OF: Add generic function to parse and allocate PCI resources PCI: Regroup all PCI related entries into drivers/pci/Makefile PCI/DPC: Reformat DPC register definitions PCI/DPC: Add and use DPC Status register field definitions PCI/DPC: Squash dpc_rp_pio_get_info() into dpc_process_rp_pio_error() PCI/DPC: Remove unnecessary RP PIO register structs PCI/DPC: Push dpc->rp_pio_status assignment into dpc_rp_pio_get_info() PCI/DPC: Squash dpc_rp_pio_print_error() into dpc_rp_pio_get_info() PCI/DPC: Make RP PIO log size check more generic PCI/DPC: Rename local "status" to "dpc_status" PCI/DPC: Squash dpc_rp_pio_print_tlp_header() into dpc_rp_pio_print_error() ...
2018-02-01Merge branch 'pci/spdx' into nextBjorn Helgaas
* pci/spdx: PCI: Add SPDX GPL-2.0+ to replace implicit GPL v2 or later statement PCI: Add SPDX GPL-2.0+ to replace GPL v2 or later boilerplate PCI: Add SPDX GPL-2.0 to replace COPYING boilerplate PCI: Add SPDX GPL-2.0 to replace GPL v2 boilerplate PCI: Add SPDX GPL-2.0 when no license was specified
2018-01-28PCI: Add SPDX GPL-2.0 to replace GPL v2 boilerplateBjorn Helgaas
Add SPDX GPL-2.0 to all PCI files that specified the GPL version 2 license. Remove the boilerplate GPL version 2 language, relying on the assertion in b24413180f56 ("License cleanup: add SPDX GPL-2.0 license identifier to files with no license") that the SPDX identifier may be used instead of the full boilerplate text. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-01-26PCI: Add SPDX GPL-2.0 when no license was specifiedBjorn Helgaas
b24413180f56 ("License cleanup: add SPDX GPL-2.0 license identifier to files with no license") added SPDX GPL-2.0 to several PCI files that previously contained no license information. Add SPDX GPL-2.0 to all other PCI files that did not contain any license information and hence were under the default GPL version 2 license of the kernel. Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-12-18switchtec: Add device IDs for PSX 24xG3 and PSX 48xG3Kelvin Cao
Add device IDs for PSX 24xG3 and PSX 48xG3. These are valid devices that were missing from the existing device ID table for the Switchtec driver. Signed-off-by: Kelvin Cao <kelvin.cao@microsemi.com> Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-12-18switchtec: Add Global Fabric Manager Server (GFMS) eventLogan Gunthorpe
Add a new event type that is newly exposed by recent firmware. The event will never occur if the firmware is too old. If user space tries to use this event in an older kernel, it will just get an EINVAL which is perfectly acceptable in the existing user space code. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-11-28the rest of drivers/*: annotate ->poll() instancesAl Viro
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2017-11-19Merge tag 'ntb-4.15' of git://github.com/jonmason/ntbLinus Torvalds
Pull ntb updates from Jon Mason: "Support for the switchtec ntb and related changes. Also, a couple of bug fixes" [ The timing isn't great. I had asked people to send me pull requests before my family vacation, and this code has not even been in linux-next as far as I can tell. But Logan Gunthorpe pleaded for its inclusion because the Switchtec driver has apparently been around for a while, just never in linux-next - Linus ] * tag 'ntb-4.15' of git://github.com/jonmason/ntb: ntb: intel: remove b2b memory window workaround for Skylake NTB NTB: make idt_89hpes_cfg const NTB: switchtec_ntb: Update switchtec documentation with notes for NTB NTB: switchtec_ntb: Add memory window support NTB: switchtec_ntb: Implement scratchpad registers NTB: switchtec_ntb: Implement doorbell registers NTB: switchtec_ntb: Add link management NTB: switchtec_ntb: Add skeleton NTB driver NTB: switchtec_ntb: Initialize hardware for doorbells and messages NTB: switchtec_ntb: Initialize hardware for memory windows NTB: switchtec_ntb: Introduce initial NTB driver NTB: Add check and comment for link up to mw_count() and mw_get_align() NTB: Ensure ntb_mw_get_align() is only called when the link is up NTB: switchtec: Add link event notifier callback NTB: switchtec: Add NTB hardware register definitions NTB: switchtec: Export class symbol for use in upper layer driver NTB: switchtec: Move structure definitions into a common header ntb: update maintainer list for Intel NTB driver
2017-11-18NTB: switchtec_ntb: Introduce initial NTB driverLogan Gunthorpe
Seeing the Switchtec NTB hardware shares the same endpoint as the management endpoint we utilize the class_interface API to register an NTB driver for every Switchtec device in the system that has the NTB class code. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Stephen Bates <sbates@raithlin.com> Reviewed-by: Kurt Schwemmer <kurt.schwemmer@microsemi.com> Acked-by: Allen Hubbe <Allen.Hubbe@dell.com> Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-11-18NTB: switchtec: Add link event notifier callbackLogan Gunthorpe
In order for the Switchtec NTB code to handle link change events we create a notifier callback in the switchtec code which gets called whenever an appropriate event interrupt occurs. In order to preserve userspace's ability to follow these events, we compare the event count with a stored copy from last time we checked. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Stephen Bates <sbates@raithlin.com> Reviewed-by: Kurt Schwemmer <kurt.schwemmer@microsemi.com> Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-11-18NTB: switchtec: Export class symbol for use in upper layer driverLogan Gunthorpe
We export the class pointer symbol and add an extern define in the Switchtec header file. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Stephen Bates <sbates@raithlin.com> Reviewed-by: Kurt Schwemmer <kurt.schwemmer@microsemi.com> Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-11-18NTB: switchtec: Move structure definitions into a common headerLogan Gunthorpe
Create the switchtec.h header in include/linux with hardware defines and the switchtec_dev structure. Both moved directly from switchtec.c. This is a prep patch for creating an NTB driver for Switchtec. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Reviewed-by: Stephen Bates <sbates@raithlin.com> Reviewed-by: Kurt Schwemmer <kurt.schwemmer@microsemi.com> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Jon Mason <jdmason@kudzu.us>
2017-10-05switchtec: Make struct event_regs staticColin Ian King
The structure event_regs is local to the source and does not need to be in global scope, so make it static. Cleans up sparse warning: symbol 'event_regs' was not declared. Should it be static Signed-off-by: Colin Ian King <colin.king@canonical.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Logan Gunthorpe <logang@deltatee.com>
2017-07-02Merge branch 'pci/switchtec' into nextBjorn Helgaas
* pci/switchtec: switchtec: Add device IDs for additional Switchtec products switchtec: Add "running" status flag to fw partition info ioctl
2017-06-27switchtec: Add device IDs for additional Switchtec productsLogan Gunthorpe
The switchtec driver also supports the PAX, PFXL and PFXI products which have the same management interface. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Kurt Schwemmer <kurt.schwemmer@microsemi.com>
2017-06-27switchtec: Add "running" status flag to fw partition info ioctlLogan Gunthorpe
This flag lets userspace know which firmware partitions are currently in use as opposed to just active. "Active" means they will be in use for the next reboot, whereas "running" means they are currently in use. If an old kernel is in use, or the firmware doesn't support these fields, the new flag will not be set in the output. Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Kurt Schwemmer <kurt.schwemmer@microsemi.com>
2017-05-22switchtec: Fix minor bug with partition ID registerLogan Gunthorpe
When a switch endpoint is configured without NTB, the mmio_ntb registers will read all zeros. However, in corner case configurations where the partition ID is not zero and NTB is not enabled, the code will have the wrong partition ID and this causes the driver to use the wrong set of drivers. To fix this we simply take the partition ID from the system info region. Reported-by: Dingbao Chen <dingbao.chen@microsemi.com> Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-05-22switchtec: Use new cdev_device_add() helper functionLogan Gunthorpe
Convert from "cdev_add() + device_add()" to cdev_device_add(), and from "device_del() + cdev_del()" to cdev_device_del(). [bhelgaas: changelog] Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-04-12switchtec: Add IOCTLs to the Switchtec driverLogan Gunthorpe
Add a couple of special IOCTLs to: * Inform userspace of firmware partition locations * Pass event counts and allow userspace to wait on events * Translate PFF numbers used by the switch to port numbers [Dan Carpenter <dan.carpenter@oracle.com>: fix off-by-one in ioctl_event_ctl()] Tested-by: Krishna Dhulipala <krishnad@fb.com> Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Signed-off-by: Stephen Bates <stephen.bates@microsemi.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Wei Zhang <wzhang@fb.com> Reviewed-by: Jens Axboe <axboe@fb.com>
2017-04-12switchtec: Add sysfs attributes to the Switchtec driverLogan Gunthorpe
Add a few read-only sysfs attributes which provide some device information that is exposed from the devices, primarily component and device names and versions. These are documented in Documentation/ABI/testing/sysfs-class-switchtec. Tested-by: Krishna Dhulipala <krishnad@fb.com> Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Signed-off-by: Stephen Bates <stephen.bates@microsemi.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Wei Zhang <wzhang@fb.com> Reviewed-by: Jens Axboe <axboe@fb.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-03-06MicroSemi Switchtec management interface driverLogan Gunthorpe
Microsemi's "Switchtec" line of PCI switch devices is already well supported by the kernel with standard PCI switch drivers. However, the Switchtec device advertises a special management endpoint with a separate PCI function address and class code. This endpoint enables some additional functionality which includes: * Packet and Byte Counters * Switch Firmware Upgrades * Event and Error logs * Querying port link status * Custom user firmware commands Add a switchtec kernel module which provides PCI driver that exposes a char device. The char device provides userspace access to this interface through read, write and (optionally) poll calls. A userspace tool and library which utilizes this interface is available at [1]. This tool takes inspiration (and borrows some code) from nvme-cli [2]. The tool is largely complete at this time but additional features may be added in the future. [1] https://github.com/sbates130272/switchtec-user [2] https://github.com/linux-nvme/nvme-cli [Dan Carpenter <dan.carpenter@oracle.com>: don't invert error codes] [Christophe JAILLET <christophe.jaillet@wanadoo.fr>: fix switchtec_dev_open() error handling] Tested-by: Krishna Dhulipala <krishnad@fb.com> Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Signed-off-by: Stephen Bates <stephen.bates@microsemi.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Wei Zhang <wzhang@fb.com> Reviewed-by: Jens Axboe <axboe@fb.com> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>