** Description changed:
+ SRU Justification
+ =================
+
+ [Impact]
+ Ubuntu installer randomly fails when rsync calls getdents64() on
overlayfs-mounted
+ directories. The overlay filesystem module returns a bogus non-zero error
code that
+ propagates up to rsync, causing the transfer to abort.
+
+ Error log:
+
+ BUG: unable to handle page fault for address: ffffffff825c9290
+ Oops: Oops: 0000 [#2] SMP NOPTI
+ ovl_iterate_merged+0x1d8/0x2b0 [overlay]
+
+
+ rsync then fails with:
+
+ rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
+ rsync error: error in rsync protocol data stream (code 12) at io.c(232)
+
+
+ Affects installation on QEMU VMs and physical hardware with various CPU/GPU
configs
+ (AMD, Intel iGPU). Hits at random intervals.
+
+ [Fix]
+ Only compute PTR_ERR(cache) on the error path in ovl_iterate_merged().
+ The bug was storing PTR_ERR(cache) before checking IS_ERR(cache), so on
+ success the truncated pointer value was returned as a false error.
+
+ Upstream in v7.1-rc7.
+ Commit: 1711b6ed6953cee5940ca4c3a6e77f1b3798cee2
+
+ [Test Plan]
+ Mount an overlayfs and run rsync:
+
+ $ mkdir -p /tmp/lower /tmp/upper /tmp/work /tmp/mnt
+ $ mount -t overlay overlay -o lowerdir=/tmp/lower,upperdir=/tmp/upper,\
+ workdir=/tmp/work /tmp/mnt
+ $ echo "test" > /tmp/lower/file.txt
+ $ rsync -aXHAS /tmp/mnt/ /tmp/dest/
+
+ Without patch: rsync fails with connection error or page fault in dmesg
+ With patch: rsync completes successfully
+
+ [Where problems could occur]
+ Could break overlayfs directory iteration on any read.
+
+ If the fix has a logic error, overlay readdir could return false errors
+ or skip valid entries. Applications using rsync or other tools that do
+ directory traversal would fail unexpectedly.
+
+ ============================================
+
During install, subiquity calls RSync. But sometimes, at random, RSync
fails with a kernel page error:
Apr 28 14:22:11 ubuntu subiquity_log.3304[7458]: Running command ['rsync',
'-aXHAS', '--one-file-system', '/tmp/tmpmfrhpjlr/mount/', '.'] with allowed
return codes [0] (capture=False)
Apr 28 14:22:11 ubuntu kernel: BUG: unable to handle page fault for address:
ffffffff825c9290
Apr 28 14:22:11 ubuntu kernel: #PF: supervisor read access in kernel mode
Apr 28 14:22:11 ubuntu kernel: #PF: error_code(0x0000) - not-present page
- Apr 28 14:22:11 ubuntu kernel: PGD 1cfe47067 P4D 1cfe47067 PUD 1cfe48063 PMD
0
+ Apr 28 14:22:11 ubuntu kernel: PGD 1cfe47067 P4D 1cfe47067 PUD 1cfe48063 PMD 0
Apr 28 14:22:11 ubuntu kernel: Oops: Oops: 0000 [#2] SMP NOPTI
- Apr 28 14:22:11 ubuntu kernel: CPU: 8 UID: 0 PID: 7495 Comm: rsync Tainted: P
D O 7.0.0-14-generic #14-Ubuntu PREEMPT(lazy)
+ Apr 28 14:22:11 ubuntu kernel: CPU: 8 UID: 0 PID: 7495 Comm: rsync Tainted: P
D O 7.0.0-14-generic #14-Ubuntu PREEMPT(lazy)
Apr 28 14:22:11 ubuntu kernel: Tainted: [P]=PROPRIETARY_MODULE, [D]=DIE,
[O]=OOT_MODULE
Apr 28 14:22:11 ubuntu kernel: Hardware name: QEMU Ubuntu 24.04 PC v2 (i440FX
+ PIIX, arch_caps fix, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
Apr 28 14:22:11 ubuntu kernel: RIP: 0010:ovl_iterate_merged+0x1d8/0x2b0
[overlay]
Apr 28 14:22:11 ubuntu kernel: Code: 41 08 48 89 ce e8 28 49 ff ff 48 8b 4d
d0 41 89 ce 48 81 f9 00 f0 ff ff 0f 87 f5 fe ff ff 48 89 4b 08 49 8b 55 08 48
83 c1 10 <4c> 8b 39 48 85 d2 7e 21 49 39 cf 74 1c 31 c0 eb 0c 0f 1f 80 00 00
Apr 28 14:22:11 ubuntu kernel: RSP: 0018:ffffd4ec8278bca8 EFLAGS: 00010286
Apr 28 14:22:11 ubuntu kernel: RAX: 0000000000000000 RBX: ffff8de0a4a7a780
RCX: ffffffff825c9290
Apr 28 14:22:11 ubuntu kernel: RDX: 0000000000000000 RSI: 0000000000000000
RDI: 0000000000000000
Apr 28 14:22:11 ubuntu kernel: RBP: ffffd4ec8278bce8 R08: 0000000000000000
R09: 0000000000000000
Apr 28 14:22:11 ubuntu kernel: R10: 0000000000000000 R11: 0000000000000000
R12: ffff8de110301ec0
Apr 28 14:22:11 ubuntu kernel: R13: ffffd4ec8278bdc0 R14: 00000000825c9280
R15: ffff8de0821af238
Apr 28 14:22:11 ubuntu kernel: FS: 00007fe1b1fd9100(0000)
GS:ffff8de224a00000(0000) knlGS:0000000000000000
Apr 28 14:22:11 ubuntu kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033
Apr 28 14:22:11 ubuntu kernel: CR2: ffffffff825c9290 CR3: 00000001cb64f000
CR4: 0000000000750ef0
Apr 28 14:22:11 ubuntu kernel: PKRU: 55555554
Apr 28 14:22:11 ubuntu kernel: Call Trace:
Apr 28 14:22:11 ubuntu kernel: <TASK>
Apr 28 14:22:11 ubuntu kernel: ovl_iterate+0xd3/0x120 [overlay]
Apr 28 14:22:11 ubuntu kernel: ? __pfx_ovl_iterate+0x10/0x10 [overlay]
Apr 28 14:22:11 ubuntu kernel: wrap_directory_iterator+0x4f/0x80
Apr 28 14:22:11 ubuntu kernel: shared_ovl_iterate+0x15/0x30 [overlay]
Apr 28 14:22:11 ubuntu kernel: iterate_dir+0xc1/0x2a0
Apr 28 14:22:11 ubuntu kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Apr 28 14:22:11 ubuntu kernel: __x64_sys_getdents64+0x76/0x140
Apr 28 14:22:11 ubuntu kernel: ? __pfx_filldir64+0x10/0x10
Apr 28 14:22:11 ubuntu kernel: x64_sys_call+0x100b/0x2390
Apr 28 14:22:11 ubuntu kernel: do_syscall_64+0x115/0x5a0
Apr 28 14:22:11 ubuntu kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Apr 28 14:22:11 ubuntu kernel: ? handle_mm_fault+0x1c0/0x2e0
Apr 28 14:22:11 ubuntu kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Apr 28 14:22:11 ubuntu kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Apr 28 14:22:11 ubuntu kernel: ?
arch_exit_to_user_mode_prepare.isra.0+0xd/0x100
Apr 28 14:22:11 ubuntu kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Apr 28 14:22:11 ubuntu kernel: ? irqentry_exit+0x97/0x5a0
Apr 28 14:22:11 ubuntu subiquity_log.3304[7458]: rsync: connection
unexpectedly closed (0 bytes received so far) [Receiver]
Apr 28 14:22:11 ubuntu kernel: ? srso_alias_return_thunk+0x5/0xfbef5
Apr 28 14:22:11 ubuntu kernel: ? exc_page_fault+0x94/0x1e0
Apr 28 14:22:11 ubuntu kernel: entry_SYSCALL_64_after_hwframe+0x76/0x7e
Apr 28 14:22:11 ubuntu kernel: RIP: 0033:0x7fe1b20d07d7
Apr 28 14:22:11 ubuntu kernel: Code: 11 00 64 c7 00 16 00 00 00 31 c0 eb 9f
e8 61 ec 04 00 90 f3 0f 1e fa b8 ff ff ff 7f 48 39 c2 48 0f 47 d0 b8 d9 00 00
00 0f 05 <48> 3d 00 f0 ff ff 77 01 c3 48 8b 15 01 b6 11 00 f7 d8 64 89 02 48
Apr 28 14:22:11 ubuntu kernel: RSP: 002b:00007ffc5f3161c8 EFLAGS: 00000293
ORIG_RAX: 00000000000000d9
Apr 28 14:22:11 ubuntu kernel: RAX: ffffffffffffffda RBX: 0000000000000000
RCX: 00007fe1b20d07d7
Apr 28 14:22:11 ubuntu kernel: RDX: 0000000000008000 RSI: 00005d8aff7c8d80
RDI: 0000000000000003
Apr 28 14:22:11 ubuntu kernel: RBP: 00007ffc5f316200 R08: 00005d8aff7c8d54
R09: 00007fe1b21ecac8
Apr 28 14:22:11 ubuntu kernel: R10: 0000000000008040 R11: 0000000000000293
R12: 0000000000000001
Apr 28 14:22:11 ubuntu kernel: R13: 00005d8aff7c8d50 R14: 00007ffc5f316300
R15: 00007fe1b1fd90b0
Apr 28 14:22:11 ubuntu kernel: </TASK>
Apr 28 14:22:11 ubuntu kernel: Modules linked in: bcache btrfs libblake2b ufs
qnx4 hfsplus hfs minix msdos jfs nls_ucs2_utils xfs snd_seq_dummy snd_hrtimer
qrtr binfmt_misc zfs(PO) spl(O) snd_hda_codec_generic intel_rapl_msr
snd_hda_intel intel_rapl_common snd_hda_codec snd_hda_core snd_intel_dspcfg
snd_intel_sdw_acpi snd_hwdep snd_pcm snd_seq_midi snd_seq_midi_event
snd_rawmidi kvm_amd ccp snd_seq snd_seq_device kvm snd_timer irqbypass snd
soundcore i2c_piix4 qxl i2c_smbus drm_ttm_helper ttm input_leds joydev drm_exec
mac_hid sch_fq_codel lp ppdev parport_pc parport msr efi_pstore nfnetlink
dmi_sysfs qemu_fw_cfg virtiofs vmw_vsock_virtio_transport
vmw_vsock_virtio_transport_common vsock autofs4 overlay isofs raid10 raid456
async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1
raid0 linear 8139too psmouse floppy ghash_clmulni_intel vga16fb serio_raw
8139cp vgastate mii pata_acpi aesni_intel
Apr 28 14:22:11 ubuntu kernel: CR2: ffffffff825c9290
Apr 28 14:22:11 ubuntu kernel: ---[ end trace 0000000000000000 ]---
Apr 28 14:22:11 ubuntu kernel: RIP: 0010:ovl_iterate_merged+0x1d8/0x2b0
[overlay]
Apr 28 14:22:11 ubuntu kernel: Code: 41 08 48 89 ce e8 28 49 ff ff 48 8b 4d
d0 41 89 ce 48 81 f9 00 f0 ff ff 0f 87 f5 fe ff ff 48 89 4b 08 49 8b 55 08 48
83 c1 10 <4c> 8b 39 48 85 d2 7e 21 49 39 cf 74 1c 31 c0 eb 0c 0f 1f 80 00 00
Apr 28 14:22:11 ubuntu kernel: RSP: 0018:ffffd4ec88763b88 EFLAGS: 00010286
Apr 28 14:22:11 ubuntu kernel: RAX: 0000000000000000 RBX: ffff8de082540940
RCX: ffffffff825c9290
Apr 28 14:22:11 ubuntu kernel: RDX: 0000000000000000 RSI: 0000000000000000
RDI: 0000000000000000
Apr 28 14:22:11 ubuntu kernel: RBP: ffffd4ec88763bc8 R08: 0000000000000000
R09: 0000000000000000
Apr 28 14:22:11 ubuntu kernel: R10: 0000000000000000 R11: 0000000000000000
R12: ffff8de13c0df080
Apr 28 14:22:11 ubuntu kernel: R13: ffffd4ec88763ca0 R14: 00000000825c9280
R15: ffff8de0821af0f8
Apr 28 14:22:11 ubuntu kernel: FS: 00007fe1b1fd9100(0000)
GS:ffff8de224a00000(0000) knlGS:0000000000000000
Apr 28 14:22:11 ubuntu kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033
Apr 28 14:22:11 ubuntu kernel: CR2: ffffffff825c9290 CR3: 00000001cb64f000
CR4: 0000000000750ef0
Apr 28 14:22:11 ubuntu kernel: PKRU: 55555554
Apr 28 14:22:11 ubuntu kernel: note: rsync[7495] exited with irqs disabled
Apr 28 14:22:11 ubuntu subiquity_log.3304[7458]: rsync error: error in rsync
protocol data stream (code 12) at io.c(232) [Receiver=3.4.1]
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2150640
Title:
Installer fails internally with a RSync error due to page fault
To manage notifications about this bug go to:
https://bugs.launchpad.net/hwe-next/+bug/2150640/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
