Package: src:linux Version: 6.1.115-1 Severity: important Tags: upstream Hi,
I'm seeing regularly kernel logs of the form below, combined with the system feeling terribly slow. This is a regression as I didn't have these problems with linux-image-6.1.0-26-amd64. The issue occurs on a VPS. The problems seem to be fixed with the linux-image-6.1.0-28-amd64-unsigned 6.1.118-1~test1 from the repo at https://people.debian.org/~carnil/tmp/linux/1086447/ (i.e., with the affected version, everything is terribly slow from the beginning while the previous version and test version don't show this symptom + nothing related yet in the kernel log). I thought about adding my report to #1087602 as it is also a regression from linux-image-6.1.0-26-amd64 on virtualized hardware and sounds related, but I decided to file as a different bug because the backtrace look different. thanks Willi [462022.185732] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [462022.186313] rcu: 0-...0: (1 GPs behind) idle=313c/1/0x4000000000000000 softirq=1652560/1652574 fqs=976 [462022.186876] (detected by 1, t=5255 jiffies, g=2935025, q=288 ncpus=2) [462022.186880] Sending NMI from CPU 1 to CPUs 0: [462037.911283] NMI backtrace for cpu 0 [462037.911288] CPU: 0 PID: 12664 Comm: systemd-journal Tainted: G L 6.1.0-27-amd64 #1 Debian 6.1.115-1 [462037.911292] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [462037.911293] RIP: 0010:asm_sysvec_reschedule_ipi+0x0/0x20 [462037.911301] Code: e9 15 07 00 00 0f 1f 44 00 00 0f 01 ca fc 6a ff e8 d5 05 00 00 48 89 c4 48 89 e7 e8 fa f9 e2 ff e9 f5 06 00 00 0f 1f 44 00 00 <0f> 01 ca fc 6a ff e8 b5 05 00 00 48 89 c4 48 89 e7 e8 fa 1b e3 ff [462037.911302] RSP: 0000:fffffe0000002fd8 EFLAGS: 00000002 [462037.911304] RAX: 0000000000000001 RBX: 0000000000fbafb0 RCX: 0000000000fbafb0 [462037.911306] RDX: 0000000000000006 RSI: 00007efd91dbafb0 RDI: 000055c938078790 [462037.911306] RBP: 000055c938078790 R08: 000055c938078970 R09: 000055c9380787c8 [462037.911307] R10: 0000000000000002 R11: 00007efd93d6f288 R12: 00007ffecfbc6918 [462037.911308] R13: 00007ffecfbc6890 R14: 0000000000000006 R15: 00007efd91dbafb0 [462037.911311] FS: 00007efd9573b5c0(0000) GS:ffff9201f9c00000(0000) knlGS:ffff9201f9c00000 [462037.911312] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [462037.911313] CR2: 00007efd93d6f1f4 CR3: 00000001022b8000 CR4: 00000000003506f0 [462037.911316] Call Trace: [462060.884093] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [462060.884626] rcu: 1-...!: (1 ticks this GP) idle=065c/1/0x4000000000000000 softirq=1599519/1599519 fqs=18 [462060.885310] (detected by 0, t=5742 jiffies, g=2935029, q=224 ncpus=2) [462060.885314] Sending NMI from CPU 0 to CPUs 1: [462073.753390] NMI backtrace for cpu 1 [462073.753396] CPU: 1 PID: 85627 Comm: kworker/1:1 Tainted: G L 6.1.0-27-amd64 #1 Debian 6.1.115-1 [462073.753401] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014 [462073.753403] Workqueue: events drm_fb_helper_damage_work [drm_kms_helper] [462073.753421] RIP: 0010:asm_sysvec_apic_timer_interrupt+0x0/0x20 [462073.753427] Code: e9 55 07 00 00 0f 1f 44 00 00 0f 01 ca fc 6a ff e8 15 06 00 00 48 89 c4 48 89 e7 e8 ba 1f e3 ff e9 35 07 00 00 0f 1f 44 00 00 <0f> 01 ca fc 6a ff e8 f5 05 00 00 48 89 c4 48 89 e7 e8 6a 1e e3 ff [462073.753429] RSP: 0018:ffffbc1a46d5fdc8 EFLAGS: 00000016 [462073.753430] RAX: 0000000000000000 RBX: 0000000000001400 RCX: 0000000000000200 [462073.753431] RDX: 0000000000000000 RSI: ffffbc1a40829000 RDI: ffffbc1a4240c000 [462073.753432] RBP: ffffbc1a4240b400 R08: ffff920162cfd848 R09: ffff920147dc1074 [462073.753433] R10: 0000000000000007 R11: 0000000000000002 R12: ffffbc1a40828400 [462073.753433] R13: ffffbc1a40828400 R14: ffffbc1a4240b400 R15: ffff920144f7bcd0 [462073.753437] FS: 0000000000000000(0000) GS:ffff9201f9d00000(0000) knlGS:0000000000000000 [462073.753438] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [462073.753439] CR2: 00007f85dd5ab0d8 CR3: 0000000122694000 CR4: 00000000003506e0 [462073.753442] Call Trace: [462073.753444] <TASK> [462073.753445] RIP: 0010:memcpy_toio+0x6d/0xb0 [462073.753448] Code: 5c e9 b7 ce 88 00 48 85 db 74 f2 40 f6 c5 01 75 45 48 83 fb 01 76 06 40 f6 c5 02 75 25 48 89 d9 48 89 ef 4c 89 e6 48 c1 e9 02 <f3> a5 f6 c3 02 74 02 66 a5 f6 c3 01 74 01 a4 5b 5d 41 5c e9 7b ce [462073.753449] RSP: 0018:ffffbc1a46d5fdf8 EFLAGS: 00000216 [462073.753452] ? drm_fb_helper_damage_work+0x2df/0x3a0 [drm_kms_helper] [462073.753463] ? process_one_work+0x1c7/0x380 [462073.753467] ? worker_thread+0x4d/0x380 [462073.753469] ? rescuer_thread+0x3a0/0x3a0 [462073.753471] ? kthread+0xda/0x100 [462073.753473] ? kthread_complete_and_exit+0x20/0x20 [462073.753475] ? ret_from_fork+0x22/0x30 [462073.753479] </TASK> [462073.753757] rcu: rcu_preempt kthread timer wakeup didn't happen for 5649 jiffies! g2935029 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [462073.754323] rcu: Possible timer handling issue on cpu=0 timer-softirq=4828143 [462073.754819] rcu: rcu_preempt kthread starved for 5650 jiffies! g2935029 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [462073.755256] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [462073.755759] rcu: RCU grace-period kthread stack dump: [462073.756240] task:rcu_preempt state:I stack:0 pid:15 ppid:2 flags:0x00004000 [462073.756245] Call Trace: [462073.756247] <TASK> [462073.756251] __schedule+0x34d/0x9e0
