Hello all,
This is the fourth version of this patch series to add support for platform
device passthrough on ARM.
In order to passthrough a non-PCI device, the user will have to:
- Map manually MMIO/IRQ
- Describe the device in the newly partial device tree support
- Specify the list of device protected by an IOMMU to assign to the
guest.
While this solution is primitive, this is allow us to support more complex
device in Xen with an little additionnal work for the user. Attempting to
do it automatically is more difficult because we may not know the dependencies
between devices (for instance a Network card and a phy).
To avoid adding code in DOM0 to manage platform device deassignment, the
user has to add the property "xen,passthrough" to the device tree node
describing the device. This can be easily done via U-Boot. For instance,
if we want to passthrough the second network card of a Midway server to the
guest. The user will have to add the following line the u-boot script:
fdt set /soc/ethernet@fff51000 xen,passthrough
This series has been tested on Midway by assigning the secondary network card
to a guest (see instruction below). Though, it requires a separate patch as
we decide to not support the Midway SMMU within the new drivers.
I plan to do futher testing on other boards.
A working tree can be found here:
git://xenbits.xen.org/julieng/xen-unstable.git branch passthrough-v4
Major changes in v4:
- The partial device tree option can only be used with trusted
device tree
- Add more documentation
- Use DOMCTL_bind_pt_irq rather than PHYSDEVOP_map_pirq
- Add XSM support for non-PCI passthrough
Major changes in v3:
- Rework the approach to passthrough a device (xen,passthrough +
partial device tree).
- Extend the existing hypercalls to assign/deassign device rather than
adding new one.
- Merge series [4] and [5] in this serie.
Major changes in v2:
- Drop the patch #1 of the previous version
- Virtual IRQ are not anymore equal to the physical interrupt
- Move the hypercall to get DT informations for privcmd to domctl
- Split the domain creation in 2 two parts to allow per guest
VGIC configuration (such as the number of SPIs).
- Bunch of typoes, commit improvement, function renaming.
For all changes see in each patch.
Ian: I think, patch #1-#5 has been acked by all relevant people. Can you push
them to xen unstable?
Sincerely yours,
[1] http://lists.xen.org/archives/html/xen-devel/2014-07/msg04090.html
[2] http://lists.xenproject.org/archives/html/xen-devel/2014-12/msg01386.html
[3] http://lists.xenproject.org/archives/html/xen-devel/2014-12/msg01612.html
[4] http://lists.xen.org/archives/html/xen-devel/2014-11/msg01672.html
[5] http://lists.xenproject.org/archives/html/xen-devel/2014-07/msg02098.html
=========================================================================
Instructions to passthrough a non-PCI device
The example will use the secondary network card for the midway server.
1) Mark the device to let Xen knowns the device will be used for passthrough.
This is done in the device tree node describing the device by adding the
property "xen,passthrough". The command to do it in U-Boot is:
fdt set /soc/ethernet@fff51000 xen,passthrough
2) Create the partial device tree describing the device. The IRQ are mapped
1:1 to the guest (i.e VIRQ == IRQ). For MMIO will have to find hole in the
guest memory layout (see xen/include/public/arch-arm.h, noted the layout
is not stable and can change between 2 releases version of Xen).
/dts-v1/;
/ {
#address-cells = <2>;
#size-cells = <2>;
aliases {
net = &mac0;
};
passthrough {
compatible = "simple-bus";
ranges;
#address-cells = <2>;
#size-cells = <2>;
mac0: ethernet@10000000 {
compatible = "calxeda,hb-xgmac";
reg = <0 0x10000000 0 0x1000>;
interrupts = <0 80 4 0 81 4 0 82 4>;
/* dma-coherent can't be set because it requires platform
* specific code for highbank
*/
/* dma-coherent; */
};
foo {
my = <&mac0>;
};
};
};
3) Compile the partial guest device with dtc (Device Tree Compiler).
For our purpose, the compiled file will be called guest-midway.dtb and
placed in /root in DOM0.
3) Add the following options in the guest configuration file:
device_tree = "/root/guest-midway.dtb"
dtdev = [ "/soc/ethernet@fff51000" ]
irqs = [ 112, 113, 114 ]
iomem = [ "0xfff51,1@0x10000" ]
Cc: manish.ja...@caviumnetworks.com
Cc: suravee.suthikulpa...@amd.com
Cc: andrii.tseglyts...@globallogic.com
Cc: robert.vanvos...@dornerworks.com
Cc: josh.whiteh...@dornerworks.com
Julien Grall (33):
xen/arm: Divide GIC initialization in 2 parts
xen/dts: Allow only IRQ translation that are mapped to main GIC
xen/dts: Use unsigned int for MMIO and IRQ index
xen/arm: vgic: Introduce a function to initialize pending_irq
xen/arm: Map disabled device in DOM0
xen/arm: Introduce xen,passthrough property
xen: guestcopy: Provide an helper to safely copy string from guest
MAINTAINERS: move drivers/passthrough/device_tree.c in "DEVICE TREE"
xen: Extend DOMCTL createdomain to support arch configuration
xen/arm: Allow virq != irq
xen/arm: route_irq_to_guest: Check validity of the IRQ
xen/arm: gic: Add sanity checks gic_route_irq_to_guest
xen/arm: gic_route_irq_to_guest: Honor the priority given in parameter
xen/arm: vgic: Correctly calculate GICD_TYPER.ITLinesNumber
xen/arm: gic: GICv2 & GICv3 only supports 1020 physical interrupts
xen/arm: Let the toolstack configure the number of SPIs
xen/arm: vgic: Add spi_to_pending
xen/arm: Release IRQ routed to a domain when it's destroying
xen/arm: Implement hypercall DOMCTL_{,un}bind_pt_pirq
xen/dts: Provide an helper to get a DT node from a path provided by a
guest
xen/passthrough: Introduce iommu_construct
xen/passthrough: arm: release the DT devices assigned to a guest
earlier
xen/passthrough: iommu_deassign_device_dt: By default reassign device
to nobody
xen/iommu: arm: Wire iommu DOMCTL for ARM
xen/xsm: Add helpers to check permission for device tree passthrough
xen/passthrough: Extend XEN_DOMCTL_*assign_device to support DT device
tools/libxl: Create a per-arch function to map IRQ to a domain
tools/libxl: Check if fdt_{first,next}_subnode are present in libfdt
tools/(lib)xl: Add partial device tree support for ARM
tools/libxl: arm: Use an higher value for the GIC phandle
libxl: Add support for non-PCI passthrough
xl: Add new option dtdev
docs/misc: arm: Add documentation about non-PCI passthrough
MAINTAINERS | 2 +
docs/man/xl.cfg.pod.5 | 15 ++
docs/misc/arm/device-tree/passthrough.txt | 9 +
docs/misc/arm/passthrough.txt | 58 +++++++
tools/config.h.in | 6 +
tools/configure.ac | 5 +
tools/flask/policy/policy/modules/xen/xen.if | 2 +-
tools/libxc/include/xenctrl.h | 32 +++-
tools/libxc/xc_domain.c | 160 +++++++++++++++---
tools/libxl/Makefile | 2 +-
tools/libxl/libxl.h | 6 +
tools/libxl/libxl_arch.h | 10 ++
tools/libxl/libxl_arm.c | 236 +++++++++++++++++++++++++--
tools/libxl/libxl_create.c | 58 ++++++-
tools/libxl/libxl_dm.c | 3 +-
tools/libxl/libxl_dom.c | 2 +-
tools/libxl/libxl_fdt.c | 84 ++++++++++
tools/libxl/libxl_internal.h | 12 +-
tools/libxl/libxl_types.idl | 6 +
tools/libxl/libxl_x86.c | 23 +++
tools/libxl/xl_cmdimpl.c | 23 ++-
xen/arch/arm/device.c | 2 +-
xen/arch/arm/domain.c | 34 +++-
xen/arch/arm/domain_build.c | 114 +++++++++----
xen/arch/arm/domctl.c | 91 ++++++++---
xen/arch/arm/gic-v2.c | 86 +++++-----
xen/arch/arm/gic-v3.c | 91 ++++++-----
xen/arch/arm/gic.c | 97 +++++++++--
xen/arch/arm/irq.c | 162 ++++++++++++++++--
xen/arch/arm/mm.c | 6 +-
xen/arch/arm/platforms/omap5.c | 12 --
xen/arch/arm/platforms/xgene-storm.c | 2 +-
xen/arch/arm/setup.c | 10 +-
xen/arch/arm/vgic-v2.c | 2 +-
xen/arch/arm/vgic-v3.c | 2 +-
xen/arch/arm/vgic.c | 73 ++++++---
xen/arch/x86/domain.c | 3 +-
xen/arch/x86/mm.c | 6 +-
xen/arch/x86/setup.c | 8 +-
xen/common/Makefile | 1 +
xen/common/device_tree.c | 43 ++++-
xen/common/domain.c | 7 +-
xen/common/domctl.c | 3 +-
xen/common/guestcopy.c | 31 ++++
xen/common/schedule.c | 3 +-
xen/drivers/passthrough/arm/iommu.c | 7 +-
xen/drivers/passthrough/arm/smmu.c | 8 +-
xen/drivers/passthrough/device_tree.c | 136 +++++++++++++--
xen/drivers/passthrough/iommu.c | 35 +++-
xen/drivers/passthrough/pci.c | 69 ++++----
xen/include/asm-arm/domain.h | 2 +
xen/include/asm-arm/gic.h | 18 +-
xen/include/asm-arm/irq.h | 8 +-
xen/include/asm-arm/setup.h | 1 +
xen/include/asm-arm/vgic.h | 9 +-
xen/include/public/arch-arm.h | 10 ++
xen/include/public/arch-x86/xen.h | 4 +
xen/include/public/domctl.h | 46 +++---
xen/include/xen/device_tree.h | 32 +++-
xen/include/xen/domain.h | 3 +-
xen/include/xen/guest_access.h | 5 +
xen/include/xen/iommu.h | 7 +-
xen/include/xen/sched.h | 9 +-
xen/include/xsm/dummy.h | 47 ++++--
xen/include/xsm/xsm.h | 55 +++++--
xen/xsm/dummy.c | 10 +-
xen/xsm/flask/avc.c | 3 +
xen/xsm/flask/flask_op.c | 49 ++----
xen/xsm/flask/hooks.c | 142 +++++++++++-----
xen/xsm/flask/include/avc.h | 2 +
xen/xsm/flask/policy/access_vectors | 4 +-
71 files changed, 1880 insertions(+), 484 deletions(-)
create mode 100644 docs/misc/arm/device-tree/passthrough.txt
create mode 100644 docs/misc/arm/passthrough.txt
create mode 100644 tools/libxl/libxl_fdt.c
create mode 100644 xen/common/guestcopy.c
--
2.1.4
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
