The fix is currently prepared for focal based on
LP 1913487 - "Focal update: v5.4.90 upstream stable release"
https://bugs.launchpad.net/bugs/1913487
Hence aligning the status of the focal entry of this bug to LP1913487
and updating the status to In Progress.
** Changed in: linux (Ubuntu Focal)
Status: Confirmed => In Progress
** Changed in: ubuntu-z-systems
Status: Confirmed => In Progress
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1907421
Title:
[UBUNTU 21.04] vfio: pass DMA availability information to userspace
Status in Ubuntu on IBM z Systems:
In Progress
Status in linux package in Ubuntu:
Fix Committed
Status in linux source package in Focal:
In Progress
Status in linux source package in Groovy:
Fix Committed
Status in linux source package in Hirsute:
Fix Committed
Bug description:
Description: vfio: pass DMA availability information to userspace
Symptom: vfio-pci device on s390 enters error state
Problem: Commit 492855939bdb added a limit to the number of concurrent
DMA requests for a vfio container. However, lazy unmapping in
s390 can in fact cause quite a large number of outstanding DMA
requests to build up prior to being purged, potentially the
entire guest DMA space. This results in unexpected errors seen
in qemu such as 'VFIO_MAP_DMA failed: No space left on device'
Solution: The solution requires a change to both kernel and qemu - For
the kernel, add the ability to provide the number of allowable
DMA requests via the VFIO_IOMMU_GET_INFO ioctl.
Reproduction: Put a vfio-pci device on s390 under I/O load
Upstream-ID: a717072007e8aedd3f951726d8cf55454860b30d
7d6e1329652ed971d1b6e0e7bea66fba5044e271
Need also to be integrated into 20.10 and 20.04.
OK, just to clarify we don't need to fix bionic for this one, but rather
focal (20.04) and groovy (20.10). Furthermore, for 20.04, 20.10 and 21.04 ONLY
commit 7d6e1329652ed971d1b6e0e7bea66fba5044e271 is needed, the other was a
pre-req that is already present.
__________
SRU Justification:
[Impact]
* In case a vfio-pci device on s390x is under I/O load, vfio-pci
device may end up in error state.
* The commit 492855939bdb added a limit to the number of concurrent
DMA requests for a vfio container.
* However, lazy unmapping in s390x can in fact cause quite a large number of
outstanding DMA requests
to build up prior to being purged - potentially the entire guest DMA space.
* This results in unexpected errors seen in qemu such as 'VFIO_MAP_DMA
failed: No space left on device'.
* The solution requires a change to both kernel and qemu.
* The kernel side of things is addressed by this SRU.
* The fix adds the ability to provide the number of allowable DMA
requests via VFIO_IOMMU_GET_INFO ioctl.
* The actual fix comes with commit 7d6e1329652e, but another fix
ccd59dce1a21 is needed to get it cleanly applied.
[Fix]
* ccd59dce1a21f473518bf273bdf5b182bab955b3 ccd59dce1a21 "vfio/type1:
Refactor vfio_iommu_type1_ioctl()"
* 7d6e1329652ed971d1b6e0e7bea66fba5044e271 7d6e1329652e "vfio iommu:
Add dma available capability"
[Test Case]
* IBM Z or LinuxONE hardware with Ubuntu Server 20.10 installed.
* PCIe adapters in place that provide vfio, like RoCE Express 2.
* A KVM host needs to be setup and a KVM guest (use again 20.10) that
uses vfio.
* Generate I/O that flows through the vf and watch out for error like
'VFIO_MAP_DMA failed: No space left on device' in the log.
[Regression Potential]
* The first patch ccd59dce1a21 modifies the common code file
drivers/vfio/vfio_iommu_type1.c quity significantly.
* But the reason is not that it introduces a lot of new things, it's a
refactoring patch.
* Nevertheless if done in a bad way it can significantly harm the IO
memory management of virtual function adapters.
* In worst case it may break them entirely, instead of 'just' exeeding
the entire DMA space.
* Things could also go wrong while doing the mapping and unmapping of
DMA, that may even have an impact beyond vf adapters - harming other
DMA devices.
* The handling of dirty pages is also touched and the ioctl itself -
which is important to keep the control of the devices.
* But as said before, it re-factoring work, it's upstream accepted
since 5.9 and the provenance shows that many engineers had an eye on
these changes.
* The second patch 7d6e1329652e - that inclides the needed fix - comes
with far less modifications.
* It also tounches drivers/vfio/vfio_iommu_type1.c but in a way that
it mainly add one new function (vfio_iommu_dma_avail_build_caps) and
an add. if statement in vfio_iommu_type1_get_info.
* That allows to provide the current allowable number of DMA mappings to the
userspace via the IOMMU info chain,
so that the userspace can take appropriate mitigation.
* Potential problems here can be that the current allowable number of
DMA mappings are wrong and in best case just mappings are wasted and
in worst case there are more reported than available in reality, which
could have a severe impact.
* What happens in such a case is a bit depending on the userspace.
* This patch got upstream accepted with kernel 5.10 and pre-tested by
IBM.
* In addition a PPA was created with a patched groovy kernel that was
shared for further testing.
[Other]
* The patch got upstream accepted with kernel v5.10, hence it will
land in Hirsute once the target kernel 5.10 is in.
* For 5.4 this will come as upstream stable update 5.4.90/91.
* For 5.8 (and 5.9) upstream stable support already ended, hence this
SRU is only needed for groovy.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-z-systems/+bug/1907421/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
