Public bug reported:
== Comment: #2 - Mete Durlu <[email protected]> - 2026-06-01 08:59:07 ==
---Problem Description---
Ubuntu 26.04 shows massive performance degradation.
On large machines with more than 20 COREs (40 CPUs with SMT)
CPU bound workloads suffer greatly.
Ex: linux kernel compilation takes >10x more time
Resource utilization shows up to 100% system time during the
workload.
perf top output indicates excessive lock contention in the kernel.
$ make -j$(nproc)
$ perf top
52.41% [kernel] [k] arch_spin_trylock_retry
8.76% [kernel] [k] _raw_spin_lock_irqsave
2.03% [kernel] [k] arch_spin_relax
1.09% cc1 [.] ht_lookup_with_hash(ht*, unsigned char
0.97% [kernel] [k] diag49c
0.95% [kernel] [k] lru_gen_add_folio
0.80% [kernel] [k] post_alloc_hook.localalias
0.77% [kernel] [k] lru_gen_del_folio.constprop.0
0.63% cc1 [.] htab_find_slot_with_hash
0.60% [kernel] [k] folios_put_refs
0.49% [kernel] [k] arch_vcpu_is_preempted
0.48% cc1 [.] ggc_internal_alloc_no_dtor(unsigned lo
0.44% cc1 [.] _cpp_lex_direct
...
The lock contention seems to be linked directly to the thread count
on the workload;
# on a system with 34 COREs (68 CPUs w SMT)
$ make -j20
perf top shows no arch_spin_trylock_retry
$ make -j25
perf top shows ~2% arch_spin_trylock_retry
$ make -j30
perf top shows ~5% arch_spin_trylock_retry
$ make -j34 # thread count = core count
perf top shows ~15% arch_spin_trylock_retry
$ make -j40 # thread count > core count
perf top shows >30% arch_spin_trylock_retry
There has also been hints of delays on workqueue execution in dmesg output:
...
[10600.136975] workqueue: vmstat_update hogged CPU for >10000us 4 times,
consider switching to WQ_UNBOUND
[10806.428576] workqueue: delayed_vfree_work hogged CPU for >10000us 4 times,
consider switching to WQ_UNBOUND
[10819.822422] workqueue: delayed_vfree_work hogged CPU for >10000us 5 times,
consider switching to WQ_UNBOUND
[10885.381900] workqueue: delayed_vfree_work hogged CPU for >10000us 7 times,
consider switching to WQ_UNBOUND
[10915.209117] workqueue: pcpu_balance_workfn hogged CPU for >10000us 4 times,
consider switching to WQ_UNBOUND
[11059.719121] workqueue: pcpu_balance_workfn hogged CPU for >10000us 5 times,
consider switching to WQ_UNBOUND
[20223.529295] workqueue: inode_switch_wbs_work_fn hogged CPU for >10000us 4
times, consider switching to WQ_UNBOUND
[22584.374168] workqueue: mmput_async_fn hogged CPU for >10000us 4 times,
consider switching to WQ_UNBOUND
[22602.115559] workqueue: delayed_vfree_work hogged CPU for >10000us 11 times,
consider switching to WQ_UNBOUND
[22817.328172] workqueue: vmstat_update hogged CPU for >10000us 5 times,
consider switching to WQ_UNBOUND
[22840.202092] workqueue: delayed_vfree_work hogged CPU for >10000us 19 times,
consider switching to WQ_UNBOUND
[26834.512017] workqueue: delayed_vfree_work hogged CPU for >10000us 35 times,
consider switching to WQ_UNBOUND
[26883.480296] workqueue: vmstat_update hogged CPU for >10000us 7 times,
consider switching to WQ_UNBOUND
...
Systems with less COREs don't seem to be effected. The limit seems to be
around 15 COREs (30 CPUs)
---uname output---
Linux localhost 7.0.0-15-generic #15-Ubuntu SMP PREEMPT Wed Apr 22 15:04:00 UTC
2026 s390x GNU/Linux
** Affects: linux (Ubuntu)
Importance: Undecided
Assignee: Skipper Bug Screeners (skipper-screen-team)
Status: New
** Tags: architecture-s39064 bugnameltc-219695 severity-high
targetmilestone-inin---
** Tags added: architecture-s39064 bugnameltc-219695 severity-high
targetmilestone-inin---
** Changed in: ubuntu
Assignee: (unassigned) => Skipper Bug Screeners (skipper-screen-team)
** Package changed: ubuntu => linux (Ubuntu)
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2154748
Title:
[Ubuntu 26.04] Severe Performance Degradation on kernel 7.0.0-15
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2154748/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
