** Description changed:
+ [ Impact ]
+
+ iommu/vt-d: Optimize iotlb_sync_map for non-caching/non-RWBF modes
+
+ The iotlb_sync_map iommu ops allows drivers to perform necessary cache
+ flushes when new mappings are established. For the Intel iommu driver,
+ this callback specifically serves two purposes:
+
+ - To flush caches when a second-stage page table is attached to a device
+ whose iommu is operating in caching mode (CAP_REG.CM==1).
+ - To explicitly flush internal write buffers to ensure updates to memory-
+ resident remapping structures are visible to hardware (CAP_REG.RWBF==1).
+
+ However, in scenarios where neither caching mode nor the RWBF flag is
+ active, the cache_tag_flush_range_np() helper, which is called in the
+ iotlb_sync_map path, effectively becomes a no-op.
+
+ Despite being a no-op, cache_tag_flush_range_np() involves iterating
+ through all cache tags of the iommu's attached to the domain, protected
+ by a spinlock. This unnecessary execution path introduces overhead,
+ leading to a measurable I/O performance regression. On systems with NVMes
+ under the same bridge, performance was observed to drop from approximately
+ ~6150 MiB/s down to ~4985 MiB/s.
+
+ Introduce a flag in the dmar_domain structure. This flag will only be set
+ when iotlb_sync_map is required (i.e., when CM or RWBF is set). The
+ cache_tag_flush_range_np() is called only for domains where this flag is
+ set. This flag, once set, is immutable, given that there won't be mixed
+ configurations in real-world scenarios where some IOMMUs in a system
+ operate in caching mode while others do not. Theoretically, the
+ immutability of this flag does not impact functionality.
+
+ [ Fix ]
+
+ Backport the following commit:
+ - 12724ce3fe1a iommu/vt-d: Optimize iotlb_sync_map for non-caching/non-RWBF
modes
+ from linux-next to Plucky.
+
+ [ Test Plan ]
+
+ Run fio against two NVMEs under the same pci bridge (dual port NVMe):
+
+ $ sudo fio --readwrite=randread --blocksize=4k --iodepth=32 --numjobs=8
+ --time_based --runtime=40 --ioengine=libaio --direct=1 --group_reporting
+ --new_group --name=job1 --filename=/dev/nvmeXnY --new_group --name=job2
+ --filename=/dev/nvmeWnZ
+
+ verify that the speed reached with the two NVMEs under the same bridge
+ is the same that would have been reached if the two NVMEs were not under
+ the same bridge.
+
+ [ Regression Potential ]
+
+ This fix affects the Intel IOMMU (VT-d) driver.
+ An issue with this fix may introduce problems such as
+ incorrect omission of required IOTLB cache or write buffer flushes
+ when attaching devices to a domain.
+ This could result in memory remapping structures not being visible
+ to hardware in configurations that actually require synchronization.
+ As a consequence, devices performing DMA may exhibit data corruption,
+ access violations, or inconsistent behavior due to stale or incomplete
+ translations being used by the hardware.
+
+ ---
+
[Description]
A performance regression has been reported when running fio against two NVMe
devices under the same pci bridge (dual port NVMe).
The issue was initially reported for 6.11-hwe kernel for Noble.
The performance regression was introduced in the 6.10 upstream kernel and is
still present in 6.16 (build at commit e540341508ce2f6e27810106253d5).
Bisection pointed to commit 129dab6e1286 ("iommu/vt-d: Use
cache_tag_flush_range_np() in iotlb_sync_map").
In our tests we observe ~6150 MiB/s when the NVMe devices are on
different bridges and ~4985 MiB/s when under the same brigde.
Before the offending commit we observe ~6150 MiB/s, regardless of NVMe
device placement.
[Test Case]
We can reproduce the issue on gcp on Z3 metal instance type
(z3-highmem-192-highlssd-metal) [1].
You need to have 2 NVMe devices under the same bridge, e.g:
# nvme list -v
...
Device SN MN FR
TxPort Address Slot Subsystem Namespaces
-------- -------------------- ----------------------------------------
-------- ------ -------------- ------ ------------ ----------------
nvme0 nvme_card-pd nvme_card-pd (null)
pcie 0000:05:00.1 nvme-subsys0 nvme0n1
nvme1 3DE4D285C21A7C001.0 nvme_card
00000000 pcie 0000:3d:00.0 nvme-subsys1 nvme1n1
nvme10 3DE4D285C21A7C001.1 nvme_card
00000000 pcie 0000:3d:00.1 nvme-subsys10 nvme10n1
nvme11 3DE4D285C2027C000.0 nvme_card
00000000 pcie 0000:3e:00.0 nvme-subsys11 nvme11n1
nvme12 3DE4D285C2027C000.1 nvme_card
00000000 pcie 0000:3e:00.1 nvme-subsys12 nvme12n1
nvme2 3DE4D285C2368C001.0 nvme_card
00000000 pcie 0000:b7:00.0 nvme-subsys2 nvme2n1
nvme3 3DE4D285C22A74001.0 nvme_card
00000000 pcie 0000:86:00.0 nvme-subsys3 nvme3n1
nvme4 3DE4D285C22A74001.1 nvme_card
00000000 pcie 0000:86:00.1 nvme-subsys4 nvme4n1
nvme5 3DE4D285C2368C001.1 nvme_card
00000000 pcie 0000:b7:00.1 nvme-subsys5 nvme5n1
nvme6 3DE4D285C21274000.0 nvme_card
00000000 pcie 0000:87:00.0 nvme-subsys6 nvme6n1
nvme7 3DE4D285C21094000.0 nvme_card
00000000 pcie 0000:b8:00.0 nvme-subsys7 nvme7n1
nvme8 3DE4D285C21274000.1 nvme_card
00000000 pcie 0000:87:00.1 nvme-subsys8 nvme8n1
nvme9 3DE4D285C21094000.1 nvme_card
00000000 pcie 0000:b8:00.1 nvme-subsys9 nvme9n1
...
For the output above, drives nvme1n1 and nvme10n1 are under the same
bridge, and looking the SN it seems it is a dual port NVMe.
- Under the same bridge
Run fio against nvme1n1 and nvme10n1, observe 4897MiB/s after a short spike
in the beginning at ~6150MiB/s.
# sudo fio --readwrite=randread --blocksize=4k --iodepth=32 --numjobs=8
--time_based --runtime=40 --ioengine=libaio --direct=1 --group_reporting
--new_group --name=job1 --filename=/dev/nvme1n1 --new_group --name=job2
--filename=/dev/nvme10n1
...
Jobs: 16 (f=16): [r(16)][100.0%][r=4897MiB/s][r=1254k IOPS][eta 00m:00s]
...
- Under different bridge
Run fio against nvme1n1 and nvme11n1, observe
# sudo fio --readwrite=randread --blocksize=4k --iodepth=32 --numjobs=8
--time_based --runtime=40 --ioengine=libaio --direct=1 --group_reporting
--new_group --name=job1 --filename=/dev/nvme1n1 --new_group --name=job2
--filename=/dev/nvme11n1
...
Jobs: 16 (f=16): [r(16)][100.0%][r=6153MiB/s][r=1575k IOPS][eta 00m:00s]
...
** So far, we haven't been able to reproduce it on another machine, but
we suspect will be reproducible with any machine with a dual port NVMe.
[Other]
In spreadsheet [2], the are some profiling data for different kernel
versions, showing consistent performance difference between kernel
versions.
Offending commit :
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=129dab6e1286525fe5baed860d3dfcd9c6b4b327
-
Report issue upstream [3].
-
[1]
https://cloud.google.com/compute/docs/storage-optimized-machines#z3_machine_types
[2]
https://docs.google.com/spreadsheets/d/19F0Vvgz0ztFpDX4E37E_o8JYrJ04iYJz-1cqU-j4Umk/edit?gid=1544333169#gid=1544333169
[3]
https://lore.kernel.org/regressions/[email protected]/
** Changed in: linux (Ubuntu Plucky)
Status: Confirmed => In Progress
** Changed in: linux (Ubuntu Questing)
Status: Confirmed => In Progress
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2115738
Title:
I/O performance regression on NVMes under same bridge (dual port nvme)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2115738/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
