Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Shuah Khan
On Fri, 2012-09-28 at 20:15 +0100, David Woodhouse wrote: > On Fri, 2012-09-28 at 12:36 -0400, Linda Knippers wrote: > > I can only speak to the HP servers. We have been shipping devices > > 'for a while' that provide sensor-type data to the platform. The > > device does DMA writes to a range of

RE: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Mingarelli, Thomas
That is correct, David. It is moved out early on when it is found to be a 32bit DMA device and its RMRR info is lost. The movement of devices between the si domain and *others* only happens in iommu=pt mode. This si domain doesn't come into play when simply booting with intel_iommu=on. Thanks,

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread David Woodhouse
On Fri, 2012-09-28 at 12:36 -0400, Linda Knippers wrote: > I can only speak to the HP servers. We have been shipping devices > 'for a while' that provide sensor-type data to the platform. The > device does DMA writes to a range of memory (the RMRR) and > iLO does DMA reads of that data. > > This

[git pull] IOMMU Fixes for v3.6-rc7

2012-09-28 Thread Joerg Roedel
Hi Linus, The following changes since commit 979570e02981d4a8fc20b3cc8fd651856c98ee9d: Linux 3.6-rc7 (2012-09-23 18:10:57 -0700) are available in the git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git tags/iommu-fixes-v3.6-rc7 for you to fetch changes up to 07

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Alex Williamson
On Fri, 2012-09-28 at 19:01 +0200, Joerg Roedel wrote: > On Fri, Sep 28, 2012 at 12:36:05PM -0400, Linda Knippers wrote: > > I can only speak to the HP servers. We have been shipping devices > > 'for a while' that provide sensor-type data to the platform. The > > device does DMA writes to a range

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Joerg Roedel
On Fri, Sep 28, 2012 at 07:01:06PM +0200, Joerg Roedel wrote: > On Fri, Sep 28, 2012 at 12:36:05PM -0400, Linda Knippers wrote: > > I can only speak to the HP servers. We have been shipping devices > > 'for a while' that provide sensor-type data to the platform. The > > device does DMA writes to

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Joerg Roedel
On Fri, Sep 28, 2012 at 12:36:05PM -0400, Linda Knippers wrote: > I can only speak to the HP servers. We have been shipping devices > 'for a while' that provide sensor-type data to the platform. The > device does DMA writes to a range of memory (the RMRR) and > iLO does DMA reads of that data. A

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Linda Knippers
David Woodhouse wrote: > > HP may have been shipping such things 'for a while' but it's never > actually worked right, yes? This thread is about the patch that's > intended to *fix* that? I can only speak to the HP servers. We have been shipping devices 'for a while' that provide sensor-type dat

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Alex Williamson
On Fri, 2012-09-28 at 16:52 +0100, David Woodhouse wrote: > HP may have been shipping such things 'for a while' but it's never > actually worked right, yes? This thread is about the patch that's > intended to *fix* that? I believe it currently works in non-passthrough mode since the RMRRs stick fo

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread David Woodhouse
HP may have been shipping such things 'for a while' but it's never actually worked right, yes? This thread is about the patch that's intended to *fix* that? If they could just manage to make their firmware-owned DMA appear to be from a different PCIe device/function from the one the OS gets to

Re: [PATCH 07/16] iommu/amd: Add IRTE invalidation routine

2012-09-28 Thread Joerg Roedel
On Fri, Sep 28, 2012 at 10:20:18AM -0400, Konrad Rzeszutek Wilk wrote: > On Fri, Sep 28, 2012 at 02:23:57PM +0200, Joerg Roedel wrote: > > +static void iommu_flush_irt_all(struct amd_iommu *iommu) > > +{ > > + u32 devid; > > + > > + for (devid = 0; devid <= 0x; ++devid) > > Should this be

Re: [PATCH 10/16] iommu/amd: Implement MSI routines for interrupt remapping

2012-09-28 Thread Joerg Roedel
On Fri, Sep 28, 2012 at 10:49:34AM -0400, Konrad Rzeszutek Wilk wrote: > On Fri, Sep 28, 2012 at 02:24:00PM +0200, Joerg Roedel wrote: > > + if (index >= MAX_IRQS_PER_TABLE) > > + return 0; > > Hmm? Not an error code? No, when this case happens it just means that the interrupt is not

Re: [PATCH 08/16] iommu/amd: Add routines to manage irq remapping tables

2012-09-28 Thread Joerg Roedel
On Fri, Sep 28, 2012 at 10:40:13AM -0400, Konrad Rzeszutek Wilk wrote: > On Fri, Sep 28, 2012 at 02:23:58PM +0200, Joerg Roedel wrote: > > + irte_info = &cfg->irq_2_iommu; > > + irte_info->sub_handle = devid; > > + irte_info->irte_in

Re: [PATCH 05/16] iommu/amd: Split device table initialization into irq and dma part

2012-09-28 Thread Joerg Roedel
On Fri, Sep 28, 2012 at 11:00:10AM -0400, Konrad Rzeszutek Wilk wrote: > On Fri, Sep 28, 2012 at 04:25:55PM +0200, Joerg Roedel wrote: > > On Fri, Sep 28, 2012 at 10:17:53AM -0400, Konrad Rzeszutek Wilk wrote: > > > On Fri, Sep 28, 2012 at 02:23:55PM +0200, Joerg Roedel wrote: > > > > When the IOMM

Re: [PATCH 05/16] iommu/amd: Split device table initialization into irq and dma part

2012-09-28 Thread Konrad Rzeszutek Wilk
On Fri, Sep 28, 2012 at 04:25:55PM +0200, Joerg Roedel wrote: > On Fri, Sep 28, 2012 at 10:17:53AM -0400, Konrad Rzeszutek Wilk wrote: > > On Fri, Sep 28, 2012 at 02:23:55PM +0200, Joerg Roedel wrote: > > > When the IOMMU is enabled very early (as with irq-remapping) > > > some devices are still in

Re: [PATCH 10/16] iommu/amd: Implement MSI routines for interrupt remapping

2012-09-28 Thread Konrad Rzeszutek Wilk
On Fri, Sep 28, 2012 at 02:24:00PM +0200, Joerg Roedel wrote: > Add routines to setup interrupt remapping for MSI > interrupts. > > Signed-off-by: Joerg Roedel > --- > drivers/iommu/amd_iommu.c | 74 > + > 1 file changed, 74 insertions(+) > > diff

Re: [PATCH 09/16] iommu/amd: Add IOAPIC remapping routines

2012-09-28 Thread Konrad Rzeszutek Wilk
On Fri, Sep 28, 2012 at 02:23:59PM +0200, Joerg Roedel wrote: > Add the routine to setup interrupt remapping for ioapic > interrupts. Also add a routine to change the affinity of an > irq and to free an irq allocation for interrupt remapping. > The last two functions will also be used for MSI inter

Re: [PATCH 08/16] iommu/amd: Add routines to manage irq remapping tables

2012-09-28 Thread Konrad Rzeszutek Wilk
On Fri, Sep 28, 2012 at 02:23:58PM +0200, Joerg Roedel wrote: > Add routines to: > > * Alloc remapping tables and single entries from these > tables > * Change entries in the tables > * Free entries in the table > > Signed-off-by: Joerg Roedel > --- > drivers/iommu/amd_iommu.c | 228 > +

Re: [PATCH 04/16] iommu/amd: Check if IOAPIC information is correct

2012-09-28 Thread Joerg Roedel
On Fri, Sep 28, 2012 at 10:16:45AM -0400, Konrad Rzeszutek Wilk wrote: > On Fri, Sep 28, 2012 at 02:23:54PM +0200, Joerg Roedel wrote: > +static bool __init check_ioapic_information(void) > > +{ > > + int idx; > > + > > + for (idx = 0; idx < nr_ioapics; ++idx) { > > That ++idx looks odd - and

Re: [PATCH 01/16] iommu/amd: Keep track of HPET and IOAPIC device ids

2012-09-28 Thread Joerg Roedel
On Fri, Sep 28, 2012 at 10:08:34AM -0400, Konrad Rzeszutek Wilk wrote: > On Fri, Sep 28, 2012 at 02:23:51PM +0200, Joerg Roedel wrote: > > + if (type != 1 && type != 2) > > + return -EINVAL; > > Should this be perhaps an enum? or #define? Yup, that'll be better. I change it. > Reviwe

Re: [PATCH 05/16] iommu/amd: Split device table initialization into irq and dma part

2012-09-28 Thread Joerg Roedel
On Fri, Sep 28, 2012 at 10:17:53AM -0400, Konrad Rzeszutek Wilk wrote: > On Fri, Sep 28, 2012 at 02:23:55PM +0200, Joerg Roedel wrote: > > When the IOMMU is enabled very early (as with irq-remapping) > > some devices are still in BIOS hand. When dma is blocked > > early this can cause lots of IO_PA

Re: [PATCH 07/16] iommu/amd: Add IRTE invalidation routine

2012-09-28 Thread Konrad Rzeszutek Wilk
On Fri, Sep 28, 2012 at 02:23:57PM +0200, Joerg Roedel wrote: > Add routine to invalidate the IOMMU cache for interupt > translations. Also include the IRTE caches when flushing all > IOMMU caches. > > Signed-off-by: Joerg Roedel > --- > drivers/iommu/amd_iommu.c | 27 +++

Re: [PATCH 05/16] iommu/amd: Split device table initialization into irq and dma part

2012-09-28 Thread Konrad Rzeszutek Wilk
On Fri, Sep 28, 2012 at 02:23:55PM +0200, Joerg Roedel wrote: > When the IOMMU is enabled very early (as with irq-remapping) > some devices are still in BIOS hand. When dma is blocked > early this can cause lots of IO_PAGE_FAULTs. So delay the > DMA initialization and do it right before the dma_ops

Re: [PATCH 04/16] iommu/amd: Check if IOAPIC information is correct

2012-09-28 Thread Konrad Rzeszutek Wilk
On Fri, Sep 28, 2012 at 02:23:54PM +0200, Joerg Roedel wrote: > When the IOAPIC information provided in the IVRS table is > not correct or not complete the system may not boot at all > when interrupt remapping is enabled. So check if this > information is correct and print out a firmware bug messag

Re: [PATCH 01/16] iommu/amd: Keep track of HPET and IOAPIC device ids

2012-09-28 Thread Konrad Rzeszutek Wilk
On Fri, Sep 28, 2012 at 02:23:51PM +0200, Joerg Roedel wrote: > The IVRS ACPI table provides information about the IOAPICs > and the HPETs available in the system and which PCI device > ID they use in transactions. Save that information for later > usage in interrupt remapping. > > Signed-off-by:

Re: 3.6-rc7 boot crash + bisection

2012-09-28 Thread Roedel, Joerg
On Wed, Sep 26, 2012 at 04:04:03PM -0600, Alex Williamson wrote: > Here's a lockdep clean version of it: > > amd_iommu: Handle aliases not backed by devices > > Aliases sometimes don't have a struct pci_dev backing them. This breaks > our attempt to figure out the topology and device quirks tha

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Alex Williamson
On Fri, 2012-09-28 at 14:52 +0200, Joerg Roedel wrote: > On Fri, Sep 28, 2012 at 06:40:08AM -0600, Alex Williamson wrote: > > On Fri, 2012-09-28 at 11:43 +0200, Joerg Roedel wrote: > > > > I don't think so. The concept of RMRR is just not defined well enough > > > (like the concept of unity mappin

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Joerg Roedel
On Fri, Sep 28, 2012 at 06:40:08AM -0600, Alex Williamson wrote: > On Fri, 2012-09-28 at 11:43 +0200, Joerg Roedel wrote: > > I don't think so. The concept of RMRR is just not defined well enough > > (like the concept of unity mappings on the AMD side which is similar to > > RMRR). The definition

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Alex Williamson
On Fri, 2012-09-28 at 11:43 +0200, Joerg Roedel wrote: > On Thu, Sep 27, 2012 at 03:34:07PM -0600, Alex Williamson wrote: > > > It really seems like RMRRs are incompatible with IOMMU API use > > though. > > I don't think so. The concept of RMRR is just not defined well enough > (like the concept

[PATCH 03/16] iommu/amd: Allocate data structures to keep track of irq remapping tables

2012-09-28 Thread Joerg Roedel
To easily map device ids to interrupt remapping table entries a new lookup table is necessary. Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu_init.c | 16 drivers/iommu/amd_iommu_types.h |9 + 2 files changed, 25 insertions(+) diff --git a/drivers/iommu/

[PATCH 02/16] iommu/amd: Add slab-cache for irq remapping tables

2012-09-28 Thread Joerg Roedel
The irq remapping tables for the AMD IOMMU need to be aligned on a 128 byte boundary. Create a seperate slab-cache to guarantee this alignment. Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu.c |2 ++ drivers/iommu/amd_iommu_init.c | 23 +++ drivers/iommu

[PATCH 12/16] iommu/amd: Add initialization routines for AMD interrupt remapping

2012-09-28 Thread Joerg Roedel
Add the six routines required to setup interrupt remapping with the AMD IOMMU. Also put it all together into the AMD specific irq_remap_ops. Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu.c | 16 +++ drivers/iommu/amd_iommu_init.c | 42

[PATCH 14/16] iommu/irq: Use amd_iommu_irq_ops if supported

2012-09-28 Thread Joerg Roedel
Finally enable interrupt remapping for AMD systems. Signed-off-by: Joerg Roedel --- drivers/iommu/irq_remapping.c |5 + 1 file changed, 5 insertions(+) diff --git a/drivers/iommu/irq_remapping.c b/drivers/iommu/irq_remapping.c index 151690d..faf85d6 100644 --- a/drivers/iommu/irq_remapp

[PATCH 05/16] iommu/amd: Split device table initialization into irq and dma part

2012-09-28 Thread Joerg Roedel
When the IOMMU is enabled very early (as with irq-remapping) some devices are still in BIOS hand. When dma is blocked early this can cause lots of IO_PAGE_FAULTs. So delay the DMA initialization and do it right before the dma_ops are initialized. To be secure, block all interrupts by default when i

[PATCH 16/16] iommu/amd: Report irq remapping through IOMMU-API

2012-09-28 Thread Joerg Roedel
Report the availability of irq remapping through the IOMMU-API to allow KVM device passthrough again without additional module parameter overrides. Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu.c |2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/iommu/amd_iommu.c b/drive

[PATCH 10/16] iommu/amd: Implement MSI routines for interrupt remapping

2012-09-28 Thread Joerg Roedel
Add routines to setup interrupt remapping for MSI interrupts. Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu.c | 74 + 1 file changed, 74 insertions(+) diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index 3df01b2..84efec

[PATCH 01/16] iommu/amd: Keep track of HPET and IOAPIC device ids

2012-09-28 Thread Joerg Roedel
The IVRS ACPI table provides information about the IOAPICs and the HPETs available in the system and which PCI device ID they use in transactions. Save that information for later usage in interrupt remapping. Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu.c |3 ++ drivers/iomm

[PATCH 13/16] iommu/amd: Make sure irq remapping still works on dma init failure

2012-09-28 Thread Joerg Roedel
Do not deinitialize the AMD IOMMU driver completly when interrupt remapping is already in use but the initialization of the DMA layer fails for some reason. Make sure the IOMMU can still be used to remap interrupts. Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu_init.c | 40 ++

[PATCH 07/16] iommu/amd: Add IRTE invalidation routine

2012-09-28 Thread Joerg Roedel
Add routine to invalidate the IOMMU cache for interupt translations. Also include the IRTE caches when flushing all IOMMU caches. Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu.c | 27 +++ drivers/iommu/amd_iommu_types.h |1 + 2 files changed, 28 inse

[PATCH 09/16] iommu/amd: Add IOAPIC remapping routines

2012-09-28 Thread Joerg Roedel
Add the routine to setup interrupt remapping for ioapic interrupts. Also add a routine to change the affinity of an irq and to free an irq allocation for interrupt remapping. The last two functions will also be used for MSI interrupts. Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu.c |

[PATCH 11/16] iommu/amd: Add call-back routine for HPET MSI

2012-09-28 Thread Joerg Roedel
Add a routine to setup a HPET MSI interrupt for remapping. Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu.c | 26 ++ 1 file changed, 26 insertions(+) diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index 84efecd..af245f4 100644 --- a/drivers

[PATCH 15/16] iommu/amd: Print message to system log when irq remapping is enabled

2012-09-28 Thread Joerg Roedel
Print an indicator to dmesg to easily find out if interrupt remapping is enabled of a given system. Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu_init.c |2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c index 0608ef

[PATCH 08/16] iommu/amd: Add routines to manage irq remapping tables

2012-09-28 Thread Joerg Roedel
Add routines to: * Alloc remapping tables and single entries from these tables * Change entries in the tables * Free entries in the table Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu.c | 228 + 1 file changed, 228 insertions(+) diff --g

[PATCH 04/16] iommu/amd: Check if IOAPIC information is correct

2012-09-28 Thread Joerg Roedel
When the IOAPIC information provided in the IVRS table is not correct or not complete the system may not boot at all when interrupt remapping is enabled. So check if this information is correct and print out a firmware bug message when it is not. Signed-off-by: Joerg Roedel --- drivers/iommu/Kco

[PATCH 06/16] iommu/amd: Make sure IOMMU is not considered to translate itself

2012-09-28 Thread Joerg Roedel
The IVRS table usually includes the IOMMU device. But the IOMMU does never translate itself, so make sure the IOMMU driver knows this. Signed-off-by: Joerg Roedel --- drivers/iommu/amd_iommu_init.c |7 +++ 1 file changed, 7 insertions(+) diff --git a/drivers/iommu/amd_iommu_init.c b/dri

[PATCH 0/16] Interrupt remapping support for AMD IOMMU

2012-09-28 Thread Joerg Roedel
Hi, here is a new version of the remaining patches to add interrupt remapping support to the AMD IOMMU driver. The patches have been rebased to v3.6-rc7 and tested again on various machines. As with the previous version no issued were found with these patches too. Regards, Joerg Diffsta

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Joerg Roedel
On Fri, Sep 28, 2012 at 11:23:23AM +0100, David Woodhouse wrote: > On Fri, 2012-09-28 at 11:46 +0200, Joerg Roedel wrote: > > Even on modern hardware with modern (IOMMU aware) kernels there is still > > this small time window when the OS has enabled the IOMMU and the USB > > driver is not initializ

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread David Woodhouse
On Fri, 2012-09-28 at 11:46 +0200, Joerg Roedel wrote: > Even on modern hardware with modern (IOMMU aware) kernels there is still > this small time window when the OS has enabled the IOMMU and the USB > driver is not initialized yet. In this time window the RMRR memory > region is still necessary,

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Joerg Roedel
On Thu, Sep 27, 2012 at 10:50:05PM +0100, David Woodhouse wrote: > That would include fairly much any USB host controller. The whole RMRR > concept is completely broken and should never have been invented. The > idea that firmware-controlled DMA should continue to happen *after* the > operating sys

Re: [PATCH v2] Intel IOMMU patch to reprocess RMRR info

2012-09-28 Thread Joerg Roedel
On Thu, Sep 27, 2012 at 03:34:07PM -0600, Alex Williamson wrote: > It really seems like RMRRs are incompatible with IOMMU API use > though. I don't think so. The concept of RMRR is just not defined well enough (like the concept of unity mappings on the AMD side which is similar to RMRR). The def