On Mon, Jul 06, 2015 at 12:11:10PM +1000, Alexey Kardashevskiy wrote: > This adds support for Dynamic DMA Windows (DDW) option defined by > the SPAPR specification which allows to have additional DMA window(s) > > This implements DDW for emulated and VFIO devices. As all TCE root regions > are mapped at 0 and 64bit long (and actual tables are child regions), > this replaces memory_region_add_subregion() with _overlap() to make > QEMU memory API happy. > > This reserves RTAS token numbers for DDW calls. > > This implements helpers to interact with VFIO kernel interface. > > This changes the TCE table migration descriptor to support dynamic > tables as from now on, PHB will create as many stub TCE table objects > as PHB can possibly support but not all of them might be initialized at > the time of migration because DDW might or might not be requested by > the guest. > > The "ddw" property is enabled by default on a PHB but for compatibility > the pseries-2.3 machine and older disable it. > > This implements DDW for VFIO. The host kernel support is required. > This adds a "levels" property to PHB to control the number of levels > in the actual TCE table allocated by the host kernel, 0 is the default > value to tell QEMU to calculate the correct value. Current hardware > supports up to 5 levels. > > The existing linux guests try creating one additional huge DMA window > with 64K or 16MB pages and map the entire guest RAM to. If succeeded, > the guest switches to dma_direct_ops and never calls TCE hypercalls > (H_PUT_TCE,...) again. This enables VFIO devices to use the entire RAM > and not waste time on map/unmap later. This adds a "dma64_win_addr" > property which is a bus address for the 64bit window and by default > set to 0x800.0000.0000.0000 as this is what the modern POWER8 hardware > uses and this allows having emulated and VFIO devices on the same bus. > > This adds 4 RTAS handlers: > * ibm,query-pe-dma-window > * ibm,create-pe-dma-window > * ibm,remove-pe-dma-window > * ibm,reset-pe-dma-window > These are registered from type_init() callback. > > These RTAS handlers are implemented in a separate file to avoid polluting > spapr_iommu.c with PCI.
> diff --git a/trace-events b/trace-events
> index b300e94..a1234dd 100644
> --- a/trace-events
> +++ b/trace-events
> @@ -1302,6 +1302,8 @@ spapr_pci_lsi_set(const char *busname, int pin,
> uint32_t irq) "%s PIN%d IRQ %u"
> spapr_pci_msi_retry(unsigned config_addr, unsigned req_num, unsigned
> max_irqs) "Guest device at %x asked %u, have only %u"
> spapr_pci_dma_update(uint64_t liobn, long ret) "liobn=%"PRIx64" ret=%ld"
> spapr_pci_dma_realloc_update(uint64_t liobn, long ret) "liobn=%"PRIx64"
> tcet=%ld"
> +spapr_pci_vfio_init_window(int ps, uint64_t ws, uint64_t off)
> "pageshift=0x%x winsize=0x%"PRIx64" offset=0x%"PRIx64
> +spapr_pci_vfio_remove_window(uint64_t off) "offset=%"PRIx64
>
> # hw/pci/pci.c
> pci_update_mappings_del(void *d, uint32_t bus, uint32_t func, uint32_t slot,
> int bar, uint64_t addr, uint64_t size) "d=%p %02x:%02x.%x
> %d,%#"PRIx64"+%#"PRIx64
> @@ -1365,6 +1367,10 @@ spapr_iommu_pci_indirect(uint64_t liobn, uint64_t
> ioba, uint64_t tce, uint64_t i
> spapr_iommu_pci_stuff(uint64_t liobn, uint64_t ioba, uint64_t tce_value,
> uint64_t npages, uint64_t ret) "liobn=%"PRIx64" ioba=0x%"PRIx64"
> tcevalue=0x%"PRIx64" npages=%"PRId64" ret=%"PRId64
> spapr_iommu_xlate(uint64_t liobn, uint64_t ioba, uint64_t tce, unsigned
> perm, unsigned pgsize) "liobn=%"PRIx64" 0x%"PRIx64" -> 0x%"PRIx64" perm=%u
> mask=%x"
> spapr_iommu_alloc_table(uint64_t liobn, void *table, int fd)
> "liobn=%"PRIx64" table=%p fd=%d"
> +spapr_iommu_ddw_query(uint64_t buid, uint32_t cfgaddr, unsigned wa, uint64_t
> win_size, uint32_t pgmask) "buid=%"PRIx64" addr=%"PRIx32", %u windows
> available, max window size=%"PRIx64", mask=%"PRIx32
Turns out the dtrace trace backend barfs on the "long long" here :(
$ ./configure --target-list=ppc64-softmmu --enable-trace-backends=dtrace
[...]
$ make
GEN config-host.h
GEN trace/generated-tracers.h
GEN trace/generated-tracers-dtrace.dtrace
GEN trace/generated-tracers-dtrace.h
Warning: /bin/dtrace:trace/generated-tracers-dtrace.dtrace:2212: syntax error
near:
probe spapr_iommu_ddw_query
Warning: Proceeding as if --no-pyparsing was given.
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.h
CC trace/generated-events.o
GEN trace/generated-tracers-dtrace.o
Warning: /bin/dtrace:trace/generated-tracers-dtrace.dtrace:2212: syntax error
near:
probe spapr_iommu_ddw_query
Warning: Proceeding as if --no-pyparsing was given.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
pgpViS7FxhasU.pgp
Description: PGP signature
