summaryrefslogtreecommitdiff
path: root/drivers/iommu/iommu.c
AgeCommit message (Collapse)Author
2016-04-07iommu: Don't overwrite domain pointer when there is no default_domainJoerg Roedel
IOMMU drivers that do not support default domains, but make use of the the group->domain pointer can get that pointer overwritten with NULL on device add/remove. Make sure this can't happen by only overwriting the domain pointer when it is NULL. Cc: stable@vger.kernel.org # v4.4+ Fixes: 1228236de5f9 ('iommu: Move default domain allocation to iommu_group_get_for_dev()') Signed-off-by: Joerg Roedel <jroedel@suse.de>
2016-02-25iommu: Fix second argument of trace_map() to report correct paddrYoshihiro Shimoda
Since iommu_map() code added pgsize value to the paddr, trace_map() used wrong paddr. So, this patch adds "orig_paddr" value in the iommu_map() to use for the trace_map(). Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-12-15Revert "scatterlist: use sg_phys()"Dan Williams
commit db0fa0cb0157 "scatterlist: use sg_phys()" did replacements of the form: phys_addr_t phys = page_to_phys(sg_page(s)); phys_addr_t phys = sg_phys(s) & PAGE_MASK; However, this breaks platforms where sizeof(phys_addr_t) > sizeof(unsigned long). Revert for 4.3 and 4.4 to make room for a combined helper in 4.5. Cc: <stable@vger.kernel.org> Cc: Jens Axboe <axboe@fb.com> Cc: Christoph Hellwig <hch@lst.de> Cc: Russell King <linux@arm.linux.org.uk> Cc: David Woodhouse <dwmw2@infradead.org> Cc: Andrew Morton <akpm@linux-foundation.org> Fixes: db0fa0cb0157 ("scatterlist: use sg_phys()") Suggested-by: Joerg Roedel <joro@8bytes.org> Reported-by: Vitaly Lavrov <vel21ripn@gmail.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2015-10-22iommu: Move default domain allocation to iommu_group_get_for_dev()Joerg Roedel
Now that the iommu core support for iommu groups is not pci-centric anymore, we can move default domain allocation to the bus independent iommu_group_get_for_dev() function. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-22iommu: Remove is_pci_dev() fall-back from iommu_group_get_for_devJoerg Roedel
All callers of iommu_group_get_for_dev() provide a device_group call-back now, so this fall-back is no longer needed. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-22iommu: Add generic_device_group() functionJoerg Roedel
This function can be used as a device_group call-back and just allocates one iommu-group per device. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-22iommu: Export and rename iommu_group_get_for_pci_dev()Joerg Roedel
Rename that function to pci_device_group() and export it, so that IOMMU drivers can use it as their device_group call-back. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-10-22iommu: Revive device_group iommu-ops call-backJoerg Roedel
That call-back is currently unused, change it into a call-back function for finding the right IOMMU group for a device. This is a first step to remove the hard-coded PCI dependency in the iommu-group code. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-08-17scatterlist: use sg_phys()Dan Williams
Coccinelle cleanup to replace open coded sg to physical address translations. This is in preparation for introducing scatterlists that reference __pfn_t. // sg_phys.cocci: convert usage page_to_phys(sg_page(sg)) to sg_phys(sg) // usage: make coccicheck COCCI=sg_phys.cocci MODE=patch virtual patch @@ struct scatterlist *sg; @@ - page_to_phys(sg_page(sg)) + sg->offset + sg_phys(sg) @@ struct scatterlist *sg; @@ - page_to_phys(sg_page(sg)) + sg_phys(sg) & PAGE_MASK Signed-off-by: Dan Williams <dan.j.williams@intel.com> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-06-29iommu: Ignore -ENODEV errors from add_device call-backJoerg Roedel
The -ENODEV error just means that the device is not translated by an IOMMU. We shouldn't bail out of iommu driver initialization when that happens, as this is a common scenario on ARM. Not returning -ENODEV in the drivers would be a bad idea, as the IOMMU core would have no indication whether a device is translated or not. This indication is not used at the moment, but will probably be in the future. Fixes: 19762d7 ("iommu: Propagate error in add_iommu_group") Tested-by: Marek Szyprowski <m.szyprowski@samsung.com> Tested-by: Eric Auger <eric.auger@linaro.org> Tested-by: Heiko Stuebner <heiko@sntech.de> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-19Merge branches 'arm/rockchip', 'arm/exynos', 'arm/smmu', 'x86/vt-d', ↵Joerg Roedel
'x86/amd', 'default-domains' and 'core' into next
2015-06-11iommu: Checking for NULL instead of IS_ERRDan Carpenter
The iommu_group_alloc() and iommu_group_get_for_dev() functions return error pointers, they never return NULL. Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-11iommu: Introduce iommu_request_dm_for_dev()Joerg Roedel
This function can be called by an IOMMU driver to request that a device's default domain is direct mapped. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-09iommu: Add function to query the default domain of a groupJoerg Roedel
This will be used to handle unity mappings in the iommu drivers. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-09iommu: Create direct mappings in default domainsJoerg Roedel
Use the information exported by the IOMMU drivers to create direct mapped regions in the default domains. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-09iommu: Introduce direct mapped region handlingJoerg Roedel
Add two new functions to the IOMMU-API to allow the IOMMU drivers to export the requirements for direct mapped regions per device. This is useful for exporting the information in Intel VT-d's RMRR entries or AMD-Vi's unity mappings. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-09iommu: Add iommu_get_domain_for_dev functionJoerg Roedel
This function can be used to request the current domain a device is attached to. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-09iommu: Make sure a device is always attached to a domainJoerg Roedel
Make use of the default domain and re-attach a device to it when it is detached from another domain. Also enforce that a device has to be in the default domain before it can be attached to a different domain. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-09iommu: Limit iommu_attach/detach_device to devices with their own groupJoerg Roedel
This patch changes the behavior of the iommu_attach_device and iommu_detach_device functions. With this change these functions only work on devices that have their own group. For all other devices the iommu_group_attach/detach functions must be used. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-09iommu: Allocate a default domain for iommu groupsJoerg Roedel
The default domain will be used (if supported by the iommu driver) when the devices in the iommu group are not attached to any other domain. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-05iommu: Call remove_device call-back after driver releaseJoerg Roedel
Do not remove the device from the IOMMU while the driver is still attached. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-05iommu: Clean up after a failed bus initializationJoerg Roedel
Make sure we call the ->remove_device call-back on all devices already initialized with ->add_device when the bus initialization fails. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-05iommu: Propagate error in add_iommu_groupJoerg Roedel
Make sure any errors reported from the IOMMU drivers get progapated back to the IOMMU core. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-05iommu: Add a few printk messages to group handling codeJoerg Roedel
Write a message to the kernel log when a device is added or removed from a group and add debug messages to group allocation and release routines. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-06-05iommu: Remove function name from pr_fmt()Joerg Roedel
Including the function name is only useful for debugging messages. They don't belong into other messages from the iommu core. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-05-29iommu: Init iommu-groups support earlier, in core_initcallMarek Szyprowski
iommu_group_alloc might be called very early in case of iommu controllers activated from of_iommu, so ensure that this part of subsystem is ready when devices are being populated from device-tree (core_initcall seems to be okay for this case). Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com> Tested-by: Javier Martinez Canillas <javier.martinez@collabora.co.uk> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-03-31iommu: Remove domain_init and domain_free iommu_opsJoerg Roedel
All drivers have been converted to the new domain_alloc and domain_free iommu-ops. So remove the old ones and get rid of iommu_domain->priv too, as this is no longer needed when the struct iommu_domain is embedded in the private structures of the iommu drivers. Tested-by: Thierry Reding <treding@nvidia.com> Tested-by: Heiko Stuebner <heiko@sntech.de> Reviewed-by: Alex Williamson <alex.williamson@redhat.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-03-31iommu: Only allow iommu_map/unmap for paging domainsJoerg Roedel
Check for the new __IOMMU_DOMAIN_PAGING flag before calling into the iommu drivers ->map and ->unmap call-backs. Tested-by: Thierry Reding <treding@nvidia.com> Tested-by: Heiko Stuebner <heiko@sntech.de> Reviewed-by: Alex Williamson <alex.williamson@redhat.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-03-31iommu: Introduce iommu domain typesJoerg Roedel
This allows to handle domains differently based on their type in the future. An IOMMU driver can implement certain optimizations for DMA-API domains for example. The domain types can be extended later and some of the existing domain attributes can be migrated to become domain flags. Tested-by: Thierry Reding <treding@nvidia.com> Tested-by: Heiko Stuebner <heiko@sntech.de> Reviewed-by: Alex Williamson <alex.williamson@redhat.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-03-31iommu: Introduce domain_alloc and domain_free iommu_opsJoerg Roedel
These new call-backs defer the allocation and destruction of 'struct iommu_domain' to the iommu driver. This allows drivers to embed this struct into their private domain structures and to get rid of the domain_init and domain_destroy call-backs when all drivers have been converted. Tested-by: Thierry Reding <treding@nvidia.com> Tested-by: Heiko Stuebner <heiko@sntech.de> Reviewed-by: Alex Williamson <alex.williamson@redhat.com> Acked-by: Will Deacon <will.deacon@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-02-04iommu: Update my email addressJoerg Roedel
The AMD address is dead for a long time already, replace it with a working one. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-01-26iommu: Fix trace_map() to report original iova and original sizeShuah Khan
iommu_map() calls trace_map() with iova and size. trace_map() should report original iova and original size as opposed to iova and size after they get changed during mapping. size is always zero at the end of mapping which is useless to report and iova as it gets incremented, it is not as useful as the original iova. Change iommu_map() to call trace_map() to report original iova and original size. Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com> Reported-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-01-19iommu: Change trace unmap api to report unmapped sizeShuah Khan
Currently map and unmap are implemented as events under a common trace class declaration. The common class forces trace_unmap() to require a bogus physical address argument that it doesn't use. Changing unmap to report unmapped size will provide useful information for debugging. Remove common map_unmap trace class and change map and unmap into separate events as opposed to events under the same class to allow for differences in the reporting information. In addition, map and unmap are changed to handle size value as size_t instead of int to match the passed size value and avoid overflow. Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com> Suggested-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2015-01-19iommu: Fix trace_unmap() to report original iovaShuah Khan
iommu_unmap() calls trace_unmap() with changed iova and original size. trace_unmap() should report original iova instead. Change iommu_unmap() to call trace_unmap() with original iova. Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com> Reported-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-12-16Merge tag 'iommu-config-for-linus' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc Pull ARM SoC/iommu configuration update from Arnd Bergmann: "The iomm-config branch contains work from Will Deacon, quoting his description: This series adds automatic IOMMU and DMA-mapping configuration for OF-based DMA masters described using the generic IOMMU devicetree bindings. Although there is plenty of future work around splitting up iommu_ops, adding default IOMMU domains and sorting out automatic IOMMU group creation for the platform_bus, this is already useful enough for people to port over their IOMMU drivers and start using the new probing infrastructure (indeed, Marek has patches queued for the Exynos IOMMU). The branch touches core ARM and IOMMU driver files, and the respective maintainers (Russell King and Joerg Roedel) agreed to have the contents merged through the arm-soc tree. The final version was ready just before the merge window, so we ended up delaying it a bit longer than the rest, but we don't expect to see regressions because this is just additional infrastructure that will get used in drivers starting in 3.20 but is unused so far" * tag 'iommu-config-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc: iommu: store DT-probed IOMMU data privately arm: dma-mapping: plumb our iommu mapping ops into arch_setup_dma_ops arm: call iommu_init before of_platform_populate dma-mapping: detect and configure IOMMU in of_dma_configure iommu: fix initialization without 'add_device' callback iommu: provide helper function to configure an IOMMU for an of master iommu: add new iommu_ops callback for adding an OF device dma-mapping: replace set_arch_dma_coherent_ops with arch_setup_dma_ops iommu: provide early initialisation hook for IOMMU drivers
2014-12-02iommu: Decouple iommu_map_sg from CPU page sizeRobin Murphy
If the IOMMU supports pages smaller than the CPU page size, segments which lie at offsets within the CPU page may be mapped based on the finer-grained IOMMU page boundaries. This minimises the amount of non-buffer memory between the CPU page boundary and the start of the segment which must be mapped and therefore exposed to the device, and brings the default iommu_map_sg implementation in line with iommu_map/unmap with respect to alignment. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-12-01iommu: fix initialization without 'add_device' callbackMarek Szyprowski
IOMMU drivers can be initialized from of_iommu helpers. Such drivers don't need to provide device_add callbacks to operate properly, so there is no need to fail initialization if the callback is missing. Acked-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com> Signed-off-by: Will Deacon <will.deacon@arm.com>
2014-11-04iommu: Improve error handling when setting bus iommuHeiko Stübner
When some part of bus_set_iommu fails it should undo any made changes and not simply leave everything as is. This includes unregistering the bus notifier in iommu_bus_init when add_iommu_group fails and also setting the bus->iommu_ops back to NULL. Signed-off-by: Heiko Stuebner <heiko@sntech.de> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-11-04iommu: Do more input validation in iommu_map_sg()Joerg Roedel
The IOMMU-API works on page boundarys, unlike the DMA-API which can work with sub-page buffers. The sg->offset field does not make sense on the IOMMU level, so force it to be 0. Do some error-path consolidation while at it. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-11-04iommu: Add iommu_map_sg() functionOlav Haugan
Mapping and unmapping are more often than not in the critical path. map_sg allows IOMMU driver implementations to optimize the process of mapping buffers into the IOMMU page tables. Instead of mapping a buffer one page at a time and requiring potentially expensive TLB operations for each page, this function allows the driver to map all pages in one go and defer TLB maintenance until after all pages have been mapped. Additionally, the mapping operation would be faster in general since clients does not have to keep calling map API over and over again for each physically contiguous chunk of memory that needs to be mapped to a virtually contiguous region. Signed-off-by: Olav Haugan <ohaugan@codeaurora.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-09-25iommu: Fix bus notifier breakageMark Salter
iommu_bus_init() registers a bus notifier on the given bus by using a statically defined notifier block: static struct notifier_block iommu_bus_nb = { .notifier_call = iommu_bus_notifier, }; This same notifier block is used for all busses. This causes a problem for notifiers registered after iommu has registered this callback on multiple busses. The problem is that a subsequent notifier being registered on a bus which has this iommu notifier will also get linked in to the notifier list of all other busses which have this iommu notifier. This patch fixes this by allocating the notifier_block at runtime. Some error checking is also added to catch any allocation failure or notifier registration error. Signed-off-by: Mark Salter <msalter@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-09-25iommu: Rework iommu_group_get_for_pci_dev()Alex Williamson
It turns out that our assumption that aliases are always to the same slot isn't true. One particular platform reports an IVRS alias of the SATA controller (00:11.0) for the legacy IDE controller (00:14.1). When we hit this, we attempt to use a single IOMMU group for everything on the same bus, which in this case is the root complex. We already have multiple groups defined for the root complex by this point, resulting in multiple WARN_ON hits. This patch makes these sorts of aliases work again with IOMMU groups by reworking how we search through the PCI address space to find existing groups. This should also now handle looped dependencies and all sorts of crazy inter-dependencies that we'll likely never see. The recursion used here should never be very deep. It's unlikely to have individual aliases and only theoretical that we'd ever see a chain where one alias causes us to search through to yet another alias. We're also only dealing with PCIe device on a single bus, which means we'll typically only see multiple slots in use on the root complex. Loops are also a theoretically possibility, which I've tested using fake DMA alias quirks and prevent from causing problems using a bitmap of the devfn space that's been visited. Signed-off-by: Alex Williamson <alex.williamson@redhat.com> Cc: stable@vger.kernel.org # 3.17 Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-09-25iommu: Remove iommu_domain_has_cap() API functionJoerg Roedel
Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-09-25iommu: Introduce iommu_capable API functionJoerg Roedel
This function will replace the current iommu_domain_has_cap function and clean up the interface while at it. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-09-25iommu: Convert iommu-caps from define to enumJoerg Roedel
Allow compile-time type-checking. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-08-26iommu/core: Make iommu_group_get_for_dev() more robustJoerg Roedel
When a non-PCI device is passed to that function it might pass group == NULL to iommu_group_add_device() which then dereferences it and cause a crash this way. Fix it by just returning an error for non-PCI devices. Fixes: 104a1c13ac66e40cf8c6ae74d76ff14ff24b9b01 Cc: Alex Williamson <alex.williamson@redhat.com> Acked-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-08-19iommu/core: Check for the right function pointer in iommu_map()Joerg Roedel
Check for the ->map and not the ->unmap pointer. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-07iommu: Constify struct iommu_opsThierry Reding
This structure is read-only data and should never be modified. Signed-off-by: Thierry Reding <treding@nvidia.com> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-04iommu/core: Create central IOMMU group lookup/creation interfaceAlex Williamson
Currently each IOMMU driver that supports IOMMU groups has its own code for discovering the base device used in grouping. This code is generally not specific to the IOMMU hardware, but to the bus of the devices managed by the IOMMU. We can therefore create a common interface for supporting devices on different buses. Signed-off-by: Alex Williamson <alex.williamson@redhat.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2013-11-01Merge branches 'iommu/fixes', 'tracing', 'core', 'arm/tegra', 'x86/vt-d', ↵Joerg Roedel
'arm/smmu' and 'arm/shmobile' into next