Public bug reported:
Summary
On kernel 7.0 (Ubuntu 7.0.0-27 … 7.0.0-34-generic) with Intel VT-d in DMA-FQ
mode, an HP Smart Array P440 (hpsa) logical volume livelocks 1–2 minutes after
boot under bursty concurrent I/O. `iommu_dma_map_sg()` fails because
`vtdss_map_range()` returns -EADDRINUSE for an IOVA that `alloc_iova_fast()`
has just handed out. The IOMMU page table still holds a mapping for an IOVA
range the allocator considers free. hpsa turns the mapping failure into
`SCSI_MLQUEUE_HOST_BUSY`, so blk-mq retries the same request (same IOVA, same
failure) forever and the whole device stalls. No errors are logged. Booting
with `iommu=pt` avoids it completely (same workload, repeated tests).
Hardware / software
- HP ProLiant ML30 Gen9, Xeon E3-1220 v6 (Kaby Lake), 16 GB, System ROM U23
v2.50 (2017-10-02)
- Smart Array P440 (PCI 0000:03:00.0, [103c:3239], fw 7.20), hpsa 3.4.20-200,
one RAID 5 LV, mq-deadline, max_sectors_kb 1024, max_segments 543
- Kernel: Ubuntu 7.0.0-34-generic (traced); the same symptom has occurred since
at least 7.0.0-27 (July 2026). No iommu cmdline options; IOMMU group type
`DMA-FQ`. 7.0.0-38 and mainline not yet tested without the workaround (the
system now runs 7.0.0-38 with iommu=pt).
- Also present: Intel C236 SATA (ahci, RAID mode) with a SATA SSD (unaffected),
BCM5720 NIC
Reproduction
Boot, mount ext4-on-LVM on the P440 LV, start ~20 Docker containers whose data
lives on it (burst of reads/writes, load ~9). Within 1–2 minutes all I/O to the
LV stops. Happens on most boots; occasionally a boot survives.
Observed state during the hang
- `/sys/kernel/debug/block/sdX/sched/queued: 0 240 0`, `owned_by_driver: 0 1
0`, `hctx0/state: TAG_ACTIVE|SCHED_RESTART`, `hctx0/flags: TAG_QUEUE_SHARED`,
`dispatch_busy: 9`, driver tags busy=0/1013, sched_tags busy=237/256
- `device_busy 0`, `host_busy 0`, hpsa `commands_outstanding 0`,
`lockup_detected ld=0`, `iorequest_cnt == iodone_cnt`
- Hung tasks: jbd2, writeback kworkers (`do_get_write_access`,
`__wait_on_buffer`, `blk_mq_get_tag`, `rq_qos_wait`)
Trace 1: blk-mq retry loop (function_graph on blk_mq_run_hw_queue + events, 2 s)
block_rq_issue: 8,0 RA 1048576 () 3562792960 + 2048 be,0,4 [kworker/1:1H]
scsi_dispatch_cmd_error: host_no=8 channel=1 id=0 lun=0 data_sgl=16 ...
driver_tag=897 scheduler_tag=141
cmnd=(READ_16 lba=3562792960 txlen=2048 ...) rtn=SCSI_MLQUEUE_HOST_BUSY
... same request repeated 54 times in 2 s
Trace 2: inside hpsa_scsi_queue_command (function_graph, funcgraph-retval)
hpsa_scsi_queue_command() {
hpsa_ioaccel_submit(); /* ret=0x1 */
hpsa_ciss_submit() {
hpsa_scatter_gather() {
scsi_dma_map() {
dma_map_sg_attrs() {
__dma_map_sg_attrs() {
iommu_dma_map_sg() {
iommu_get_dma_domain();
iommu_dma_sync_sg_for_device(); /* ret=0x0 */
iommu_dma_alloc_iova() {
alloc_iova_fast() /* ret=0x7bf00 */
} /* ret=0x7bf00000 */
iommu_map_sg() {
iommu_map_nosync() {
vtdss_map_range(); /* ret=-98 */
iommu_unmap(); /* ret=0x23000 */
} /* ret=-98 */
iommu_unmap(); /* ret=0x0 */
} /* ret=-98 */
iommu_dma_free_iova() { free_iova_fast(); }
} /* iommu_dma_map_sg ret=-22 */
}
}
} /* scsi_dma_map ret=-12 */
} /* hpsa_scatter_gather ret=-12
*/
} /* hpsa_ciss_submit ret=0x1055
*/
} /* SCSI_MLQUEUE_HOST_BUSY */
Every retry gets the same IOVA (0x7bf00000) and the same -EADDRINUSE, so it
never recovers. Note that the partial-map rollback unmapped 0x23000 bytes, i.e.
part of the range mapped before hitting the occupied PTEs.
Workaround
`iommu=pt` (verified with a Docker-only start and a full normal boot, each 30
min clean, where the same start-up previously hung within ~2 min; also stable
on 7.0.0-38 since) (group type `identity`): the identical workload has run
clean through start-up and beyond on several boots.
Possibly related
- April 2026 report of stale-PTE warnings on IOVA reuse in the lazy-flush
(`__iommu_dma_unmap_sg` / `free_iova_fast`) path under ATA/SCSI load on 6.12.y
(series rejected; root cause disputed):
https://ratatoskr.run/linux-iommu/2026/04/3525711/t
- Our case differs: the stale PTE is persistent (dozens of retries over
minutes), not a transient TLB race. That suggests a mapping whose IOVA was
freed without its PTEs ever being cleared.
- Separately, hpsa turning any `scsi_dma_map()` failure into HOST_BUSY (with no
log message) means a deterministic mapping failure becomes a silent permanent
livelock. Logging it, or failing the command after N retries, would make this
kind of fault visible.
Full traces and debugfs dumps available on request.
ProblemType: Bug
DistroRelease: Ubuntu 26.04
Package: linux-image-7.0.0-38-generic 7.0.0-38.38
ProcVersionSignature: Ubuntu 7.0.0-38.38-generic 7.0.14
Uname: Linux 7.0.0-38-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Oct 1 21:04 seq
crw-rw---- 1 root audio 116, 33 Oct 1 21:04 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.34.1-0ubuntu0.1
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
'/dev/snd/timer'] failed with exit code 1:
CRDA: N/A
CasperMD5CheckResult: pass
CurrentDmesg: Error: command ['dmesg'] failed with exit code 1: dmesg: read
kernel buffer failed: Operation not permitted
Date: Thu Oct 1 21:13:10 2026
InstallationDate: Installed on 2026-05-26 (128 days ago)
InstallationMedia: Ubuntu-Server 26.04 "Resolute Raccoon" - Release amd64
(20260420.1)
Lsusb:
Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 0424:2660 Microchip Technology, Inc. (formerly SMSC) Hub
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Lsusb-t:
/: Bus 001.Port 001: Dev 001, Class=root_hub, Driver=uhci_hcd/2p, 12M
/: Bus 002.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/16p, 480M
|__ Port 003: Dev 002, If 0, Class=Hub, Driver=hub/2p, 480M
/: Bus 003.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/10p, 5000M
MachineType: HP ProLiant ML30 Gen9
PciMultimedia:
ProcEnviron:
LANG=en_US.UTF-8
PATH=(custom, no user)
SHELL=/bin/bash
TERM=xterm-256color
XDG_RUNTIME_DIR=<set>
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-7.0.0-38-generic
root=/dev/mapper/ssd--vg-root ro iommu=pt
crashkernel=2G-4G:320M,4G-32G:512M,32G-64G:1024M,64G-128G:2048M,128G-:4096M
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
acpidump:
dmi.bios.date: 10/02/2017
dmi.bios.release: 2.50
dmi.bios.vendor: HP
dmi.bios.version: U23
dmi.board.name: ProLiant ML30 Gen9
dmi.board.vendor: HP
dmi.chassis.type: 7
dmi.chassis.vendor: HP
dmi.ec.firmware.release: 2.77
dmi.modalias:
dmi:bvnHP:bvrU23:bd10/02/2017:br2.50:efr2.77:svnHP:pnProLiantML30Gen9:pvr:rvnHP:rnProLiantML30Gen9:rvr:cvnHP:ct7:cvr:sku823403-B21:pfaProLiant:
dmi.product.family: ProLiant
dmi.product.name: ProLiant ML30 Gen9
dmi.product.sku: 823403-B21
dmi.sys.vendor: HP
** Affects: linux (Ubuntu)
Importance: Undecided
Status: New
** Tags: amd64 apport-bug dma hang hpsa intel-iommu iommu resolute storage vt-d
** Attachment added: "Traces from affected system"
https://bugs.launchpad.net/bugs/2169238/+attachment/6004668/+files/hpsa-iommu-traces.tar.gz
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2169238
Title:
hpsa P440 livelocks: VT-d iommu_map -EADDRINUSE on fresh IOVA (DMA-
FQ), works with iommu=pt
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2169238/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
