[PATCH 0/1] Mask VT-d DMAR faults for IGD devices
From: Brendan Kerrigan The Intel IOMMU for at least 8th and 9th generation Core processors has a bug where the Fault Processing Disable bit is not respected for the Intel Graphics Device (IGD). The resulting behavior was reported previously[1]. The underlying cause is described by Intel as Errata ID 049 in the Core spec update document[2], along with a suggested workaround, which is implemented in the patch. [1] https://lists.xen.org/archives/html/xen-devel/2015-08/msg01955.html [2] https://www.intel.com/content/dam/www/public/us/en/documents/specification-updates/8th-gen-core-spec-update.pdf Brendan Kerrigan (1): x86/vtd: Mask DMAR faults for IGD devices xen/drivers/passthrough/vtd/iommu.c | 9 + 1 file changed, 9 insertions(+) -- 2.17.1
[PATCH 1/1] x86/vtd: Mask DMAR faults for IGD devices
From: Brendan Kerrigan
The Intel graphics device records DMAR faults regardless
of the Fault Process Disable bit being set. When this fault
occurs, enable the Interrupt Mask (IM) bit in the Fault
Event Control (FECTL) register to prevent the continued
recording of the fault.
Signed-off-by: Brendan Kerrigan
---
xen/drivers/passthrough/vtd/iommu.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/xen/drivers/passthrough/vtd/iommu.c
b/xen/drivers/passthrough/vtd/iommu.c
index 07d40b37fe..288399d816 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -41,6 +41,8 @@
#include "vtd.h"
#include "../ats.h"
+#define IS_IGD(seg, id) (0 == seg && 0 == PCI_BUS(id) && 2 == PCI_SLOT(id) &&
0 == PCI_FUNC(id))
+
struct mapped_rmrr {
struct list_head list;
u64 base, end;
@@ -872,6 +874,13 @@ static int iommu_page_fault_do_one(struct vtd_iommu
*iommu, int type,
printk(XENLOG_G_WARNING VTDPREFIX "%s: reason %02x - %s\n",
kind, fault_reason, reason);
+if ( DMA_REMAP == fault_type && type && IS_IGD(seg, source_id) ) {
+u32 fectl = dmar_readl(iommu->reg, DMAR_FECTL_REG);
+fectl |= DMA_FECTL_IM;
+dmar_writel(iommu->reg, DMAR_FECTL_REG, fectl);
+printk(XENLOG_G_WARNING VTDPREFIX "Disabling DMAR faults for IGD\n");
+}
+
if ( iommu_verbose && fault_type == DMA_REMAP )
print_vtd_entries(iommu, PCI_BUS(source_id), PCI_DEVFN2(source_id),
addr >> PAGE_SHIFT);
--
2.17.1
Re: [PATCH 1/1] x86/vtd: Mask DMAR faults for IGD devices
While it's described as errata for gen8/9, the previous reporting was from
2015 which predates those generations. I tested it on a gen 7 box (which
was causing me the grief of consuming my Xen console buffer). It could be
the case that the FPD code isn't implemented (which wouldn't matter for
gen8/9 because it is broken), and the original problem of faulty firmware
reporting bad ranges is the ultimate culprit. As far as the last two
questions, I was testing on an older version of Xen (4.9.x) and ported it
to master. Happy to hear a better approach to solving the original problem.
-Brendan
On Sun, Apr 19, 2020 at 11:28 PM Tian, Kevin wrote:
> > From: Brendan Kerrigan
> > Sent: Friday, April 17, 2020 9:36 PM
> >
> > From: Brendan Kerrigan
> >
> > The Intel graphics device records DMAR faults regardless
> > of the Fault Process Disable bit being set. When this fault
>
> Since this is an errata for specific generations, let's describe
> this way instead of stating it as a generic problem.
>
> > occurs, enable the Interrupt Mask (IM) bit in the Fault
> > Event Control (FECTL) register to prevent the continued
> > recording of the fault.
> >
> > Signed-off-by: Brendan Kerrigan
> > ---
> > xen/drivers/passthrough/vtd/iommu.c | 9 +
> > 1 file changed, 9 insertions(+)
> >
> > diff --git a/xen/drivers/passthrough/vtd/iommu.c
> > b/xen/drivers/passthrough/vtd/iommu.c
> > index 07d40b37fe..288399d816 100644
> > --- a/xen/drivers/passthrough/vtd/iommu.c
> > +++ b/xen/drivers/passthrough/vtd/iommu.c
> > @@ -41,6 +41,8 @@
> > #include "vtd.h"
> > #include "../ats.h"
> >
> > +#define IS_IGD(seg, id) (0 == seg && 0 == PCI_BUS(id) && 2 ==
> PCI_SLOT(id)
> > && 0 == PCI_FUNC(id))
> > +
> > struct mapped_rmrr {
> > struct list_head list;
> > u64 base, end;
> > @@ -872,6 +874,13 @@ static int iommu_page_fault_do_one(struct
> > vtd_iommu *iommu, int type,
> > printk(XENLOG_G_WARNING VTDPREFIX "%s: reason %02x - %s\n",
> > kind, fault_reason, reason);
> >
> > +if ( DMA_REMAP == fault_type && type && IS_IGD(seg, source_id) ) {
> > +u32 fectl = dmar_readl(iommu->reg, DMAR_FECTL_REG);
> > +fectl |= DMA_FECTL_IM;
> > +dmar_writel(iommu->reg, DMAR_FECTL_REG, fectl);
> > +printk(XENLOG_G_WARNING VTDPREFIX "Disabling DMAR faults for
> > IGD\n");
> > +}
> > +
>
> Several questions. First, I just note that FPD is not touched by any code
> today. then how is this errata being caught? Second, we already have
> pci_check_disable_device in place which stops DMAs from the problematic
> device if it triggers excessive fault reports. why doesn't it work for your
> case? Last, dma_msi_end just forces clearing the mask bit at end of
> handling
> the fault interrupt, making above fix meaningless. Those questions just
> make me wonder how you actually test this patch and whether it's
> necessary...
>
> Thanks
> Kevin
>
> > if ( iommu_verbose && fault_type == DMA_REMAP )
> > print_vtd_entries(iommu, PCI_BUS(source_id),
> PCI_DEVFN2(source_id),
> >addr >> PAGE_SHIFT);
> > --
> > 2.17.1
>
>
Design Session Notes - "How best to upstream Bareflank's implementation of the Xen VMM into the Xen Project"
# Design Session Notes - "How best to upstream Bareflank's implementation of the Xen VMM into the Xen Project" ## Design Session Description Assured Information Security, Inc. has been working on a new implementation of the Xen VMM (i.e., just the root component of the Xen hypervisor) using Bareflank/Boxy (https://github.com/Bareflank/hypervisor). The goals of this VMM include the ability to reload the hypervisor without having to reboot, support for a Windows Dom0 (or any Dom0 really), removal of the Xen scheduling and power management code and instead using the scheduler and power management logic built into the Dom0 kernel, and removal of PV support in favor of a pure PVH/HVM implementation. Although there is still a lot of work to do, we can demonstrate this capability today. The goal of this design session is to discuss the design of our new approach, ways in which we can improve it, and ultimately how best to upstream our work into the Xen Project. ## Current Status of Xen compatibility in Bareflank Bareflank has compatibility layer to work with the Xen PVH hypercalls, with the goal of re-using existing Xen VMs, improved performance, less complex scheduling and power management, and the ability to run non-traditional dom0s, such as Windows. * Prototype is up and running: * Removal of scheduling/power management code * Windows dom0 * Hotloading the hypervisor (optional late launch, is desirable by gwd) * Doesn't share any code directly with Xen at the moment, was referenced throughout the implementation * No legacy PV mode, PVH only (currently) though HVM is planned down the road * APIC not managed by the hypervisor, but by dom0 * Mostly MSIs are passed directly to dom0 * Minimal interference with power management, lets the operating system deal with it * Removal of possible schedule aliasing * libxl toolstack runs outside of dom0 ## Discussion of upstreamability (desirable by andyhhp) * How to do it organizationally: * Subproject (feedback indicated this was undesirable, there would be too much overhead) * Directly bring capabilities into the mainline (feedback indicated this was more desirable) * Potential technical requirements: * Add scheduling API to allow the use of a host OS scheduler rather than the built in Xen scheduler (may be difficult) ## Proposed Actions: * KConfig options to strip out undesirable code portions * Add ability to do the rootkit like loading of Xen (like hxen/bareflank support) * Addition of more hypercalls to support dom0 (or more generally non-hypervisor context) scheduling * Stay engaged with xen-devel to align changes with potential ABI changes that are coming down the road to support updating libxl and the hypervisor in stages and supporting encrypted memory schemes that are incorporated in newer architectures
