summaryrefslogtreecommitdiff
path: root/drivers/iommu/amd_iommu_v2.c
AgeCommit message (Collapse)Author
2014-09-24kvm: Fix page ageing bugsAndres Lagar-Cavilla
1. We were calling clear_flush_young_notify in unmap_one, but we are within an mmu notifier invalidate range scope. The spte exists no more (due to range_start) and the accessed bit info has already been propagated (due to kvm_pfn_set_accessed). Simply call clear_flush_young. 2. We clear_flush_young on a primary MMU PMD, but this may be mapped as a collection of PTEs by the secondary MMU (e.g. during log-dirty). This required expanding the interface of the clear_flush_young mmu notifier, so a lot of code has been trivially touched. 3. In the absence of shadow_accessed_mask (e.g. EPT A bit), we emulate the access bit by blowing the spte. This requires proper synchronizing with MMU notifier consumers, like every other removal of spte's does. Signed-off-by: Andres Lagar-Cavilla <andreslc@google.com> Acked-by: Rik van Riel <riel@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2014-07-30iommu/amd: Fix 2 typos in commentsJoerg Roedel
amd_iommu_pasid_bind -> amd_iommu_bind_pasid Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-30iommu/amd: Fix device_state reference countingJoerg Roedel
The references to the device state are not dropped everywhere. This might cause a dead-lock in amd_iommu_free_device(). Fix it. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-30iommu/amd: Remove change_pte mmu_notifier call-backJoerg Roedel
All calls to this call-back are wrapped with mmu_notifer_invalidate_range_start()/end(), making this notifier pretty useless, so remove it. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-30iommu/amd: Don't set pasid_state->mm to NULL in unbind_pasidJoerg Roedel
With calling te mmu_notifier_register function we hold a reference to the mm_struct that needs to be released in mmu_notifier_unregister. This is true even if the notifier was already unregistered from exit_mmap and the .release call-back has already run. So make sure we call mmu_notifier_unregister unconditionally in amd_iommu_unbind_pasid. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Oded Gabbay <oded.gabbay@amd.com>
2014-07-10iommu/amd: Don't call the inv_ctx_cb when pasid is not set upJoerg Roedel
On the error path of amd_iommu_bind_pasid() we call mmu_notifier_unregister() for cleanup. This calls mn_release() which calls the users inv_ctx_cb function if one is available. Since the pasid is not set up yet there is nothing the user can to tear down in this call-back. So don't call inv_ctx_cb on the error path of amd_iommu_unbind_pasid() and make life of the users simpler. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10iommu/amd: Don't hold a reference to task_structJoerg Roedel
Since we are only caring about the lifetime of the mm_struct and not the task we can't safely keep a reference to it. The reference is also not needed anymore, so remove that code entirely. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10iommu/amd: Don't hold a reference to mm_structJoerg Roedel
With mmu_notifiers we don't need to hold a reference to the mm_struct during the time the pasid is bound to a device. We can rely on the .mn_release call back to inform us when the mm_struct goes away. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10iommu/amd: Add pasid_state->invalid flagJoerg Roedel
This is used to signal the ppr_notifer function that no more faults should be processes on this pasid_state. This way we can keep the pasid_state safely in the state-table so that it can be freed in the amd_iommu_unbind_pasid() function. This allows us to not hold a reference to the mm_struct during the whole pasid-binding-time. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10iommu/amd: Drop pasid_state reference in ppr_notifer error pathJoerg Roedel
In case we are not able to allocate a fault structure a reference to the pasid_state will be leaked. Fix that by dropping the reference in the error path in case we hold one. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10iommu/amd: Get rid of __unbind_pasidJoerg Roedel
Unbind_pasid is only called from mn_release which already has the pasid_state. Use this to simplify the unbind_pasid path and get rid of __unbind_pasid. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10iommu/amd: Don't free pasid_state in mn_release pathJoerg Roedel
The mmu_notifier state is part of pasid_state so it can't be freed in the mn_release path. Free the pasid_state after mmu_notifer_unregister has completed. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10iommu/amd: Don't call mmu_notifer_unregister in __unbind_pasidJoerg Roedel
This function is called only in the mn_release() path, so there is no need to unregister the mmu_notifer here. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Oded Gabbay <Oded.Gabbay@amd.com>
2014-07-10iommu/amd: Fix typo in amd_iommu_v2 driverJoerg Roedel
Fix typo in a comment. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-07-09iommu/amd: Fix for pasid initializationAlexey Skidanov
The pasid wasn't properly initialized before caling to invalid PPR calback Signed-off-by: Alexey Skidanov <Alexey.Skidanov@amd.com> Signed-off-by: Oded Gabbay <oded.gabbay@amd.com> Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-06-20iommu/amd: Fix small race between invalidate_range_end/startJoerg Roedel
Commit e79df31 introduced mmu_notifer_count to protect against parallel mmu_notifier_invalidate_range_start/end calls. The patch left a small race condition when invalidate_range_end() races with a new invalidate_range_start() the empty page-table may be reverted leading to stale TLB entries in the IOMMU and the device. Use a spin_lock instead of just an atomic variable to eliminate the race. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-05-26iommu/amd: Handle parallel invalidate_range_start/end calls correctlyJoerg Roedel
Add a counter to the pasid_state so that we do not restore the original page-table before all invalidate_range_start to invalidate_range_end sections have finished. Signed-off-by: Joerg Roedel <jroedel@suse.de>
2014-05-26iommu/amd: Remove IOMMUv2 pasid_state_listJoerg Roedel
This list was only used for the task_exit notifier function. Now that it is gone we can remove it. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Jay Cornwall <Jay.Cornwall@amd.com>
2014-05-26iommu/amd: Implement mmu_notifier_release call-backJoerg Roedel
Since mmu_notifier call-backs can sleep (because they use SRCU now) we can use them to tear down PASID mappings. This allows us to finally remove the hack to use the task_exit notifier from oprofile to get notified when a process dies. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Jay Cornwall <Jay.Cornwall@amd.com>
2014-05-26iommu/amd: Convert IOMMUv2 state_table into state_listJoerg Roedel
The state_table consumes 512kb of memory and is only sparsly populated. Convert it into a list to save memory. There should be no measurable performance impact. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Jay Cornwall <Jay.Cornwall@amd.com>
2014-05-26iommu/amd: Don't access IOMMUv2 state_table directlyJoerg Roedel
This is a preparation for converting the state_table into a state_list. Signed-off-by: Joerg Roedel <jroedel@suse.de> Tested-by: Jay Cornwall <Jay.Cornwall@amd.com>
2014-05-13iommu/amd: Take mmap_sem when calling get_user_pagesJay Cornwall
get_user_pages requires caller to hold a read lock on mmap_sem. Signed-off-by: Jay Cornwall <jay.cornwall@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Joerg Roedel <joro@8bytes.org>
2012-07-24Merge tag 'iommu-updates-v3.6-rc1' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu Pull IOMMU updates from Joerg Roedel: "The most important part of these updates is the IOMMU groups code enhancement written by Alex Williamson. It abstracts the problem that a given hardware IOMMU can't isolate any given device from any other device (e.g. 32 bit PCI devices can't usually be isolated). Devices that can't be isolated are grouped together. This code is required for the upcoming VFIO framework. Another IOMMU-API change written by me is the introduction of domain attributes. This makes it easier to handle GART-like IOMMUs with the IOMMU-API because now the start-address and the size of the domain address space can be queried. Besides that there are a few cleanups and fixes for the NVidia Tegra IOMMU drivers and the reworked init-code for the AMD IOMMU. The latter is from my patch-set to support interrupt remapping. The rest of this patch-set requires x86 changes which are not mergabe yet. So full support for interrupt remapping with AMD IOMMUs will come in a future merge window." * tag 'iommu-updates-v3.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (33 commits) iommu/amd: Fix hotplug with iommu=pt iommu/amd: Add missing spin_lock initialization iommu/amd: Convert iommu initialization to state machine iommu/amd: Introduce amd_iommu_init_dma routine iommu/amd: Move unmap_flush message to amd_iommu_init_dma_ops() iommu/amd: Split enable_iommus() routine iommu/amd: Introduce early_amd_iommu_init routine iommu/amd: Move informational prinks out of iommu_enable iommu/amd: Split out PCI related parts of IOMMU initialization iommu/amd: Use acpi_get_table instead of acpi_table_parse iommu/amd: Fix sparse warnings iommu/tegra: Don't call alloc_pdir with as->lock iommu/tegra: smmu: Fix unsleepable memory allocation at alloc_pdir() iommu/tegra: smmu: Remove unnecessary sanity check at alloc_pdir() iommu/exynos: Implement DOMAIN_ATTR_GEOMETRY attribute iommu/tegra: Implement DOMAIN_ATTR_GEOMETRY attribute iommu/msm: Implement DOMAIN_ATTR_GEOMETRY attribute iommu/omap: Implement DOMAIN_ATTR_GEOMETRY attribute iommu/vt-d: Implement DOMAIN_ATTR_GEOMETRY attribute iommu/amd: Implement DOMAIN_ATTR_GEOMETRY attribute ...
2012-07-24iommu: Fix typo in iommuMasanari Iida
Correct spelling typo in debug messages and comments in drivers/iommu. Signed-off-by: Masanari Iida <standby24x7@gmail.com> Signed-off-by: Jiri Kosina <jkosina@suse.cz>
2012-07-23Merge branches 'iommu/fixes', 'x86/amd', 'groups', 'arm/tegra' and ↵Joerg Roedel
'api/domain-attr' into next Conflicts: drivers/iommu/iommu.c include/linux/iommu.h
2012-07-19iommu/amd: Add missing spin_lock initializationJoerg Roedel
Add missing spin_lock initialization in amd_iommu_bind_pasid() function and make lockdep happy again. Cc: stable@vger.kernel.org # >= v3.3 Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2012-07-17iommu/amd: Fix sparse warningsJoerg Roedel
A few sparse warnings fire in drivers/iommu/amd_iommu_init.c. Fix most of them with this patch. Also fix the sparse warnings in drivers/iommu/irq_remapping.c while at it. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2012-03-15iommu/amd: Don't initialize IOMMUv2 resources when not requiredJoerg Roedel
Add a check to the init-path of the AMD IOMMUv2 driver if the hardware is available in the system. Only allocate all the resources if it is really available. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2011-12-15iommu/amd: Add invalidate-context call-backJoerg Roedel
This call-back is invoked when the task that is bound to a pasid is about to exit. The driver can use it to shutdown all context related to that context in a safe way. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2011-12-14iommu/amd: Add invalid_ppr callbackJoerg Roedel
This callback can be used to change the PRI response code sent to a device when a PPR fault fails. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2011-12-14iommu/amd: Implement notifiers for IOMMUv2Joerg Roedel
Since pages are not pinned anymore we need notifications when the VMM changes the page-tables. Use mmu_notifiers for that. Also use the task_exit notifier from the profiling subsystem to shutdown all contexts related to this task. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2011-12-12iommu/amd: Implement IO page-fault handlerJoerg Roedel
Register the notifier for PPR faults and handle them as necessary. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2011-12-12iommu/amd: Add routines to bind/unbind a pasidJoerg Roedel
This patch adds routines to bind a specific process address-space to a given PASID. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2011-12-12iommu/amd: Implement device aquisition code for IOMMUv2Joerg Roedel
This patch adds the amd_iommu_init_device() and amd_iommu_free_device() functions which make a device and the IOMMU ready for IOMMUv2 usage. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2011-12-12iommu/amd: Add driver stub for AMD IOMMUv2 supportJoerg Roedel
Add a Kconfig option for the optional driver. Since it is optional it can be compiled as a module and will only be loaded when required by another driver. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>