And now a crash with the special 4.15.0-48-generic #51~lp1824981 kernel. So the buffer overflow is NOT fixed with any 4.15 or 4.18 kernel currently available with Ubuntu.
We started some IO-intense tasks lately that didn't run the last 2 weeks. That seems to be the trigger for the bug buffer overflow. We're consider to switch to NFS or store more data locally to workaround that issue. Recommendations and alternatives welcome. I've seend that there are some cifs related patches for 5.1RC7 - are those gonna be backportet to 4.x-generic? Apr 30 00:57:23 kernel: [106134.709084] detected buffer overflow in strcat Apr 30 00:57:23 kernel: [106134.709123] ------------[ cut here ]------------ Apr 30 00:57:23 kernel: [106134.709124] kernel BUG at /home/ubuntu/Sources/linux-lp1824981/lib/string.c:1052! Apr 30 00:57:23 kernel: [106134.709149] invalid opcode: 0000 [#1] SMP PTI Apr 30 00:57:23 kernel: [106134.709162] Modules linked in: cmac(E) arc4(E) md4(E) nls_utf8(E) cifs(E) ccm(E) fscache(E) ufs(E) qnx4(E) hfsplus(E) hfs(E) minix(E) ntfs(E) msdos(E) jfs(E) xfs(E) cpuid(E) mpt3sas raid_class scsi_transport_sas mptctl mptbase dell_rbu bonding nls_iso8859_1 intel_rapl skx_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass input_leds joydev dell_smbios dcdbas ipmi_ssif dell_wmi_descriptor intel_cstate intel_rapl_perf wmi_bmof ipmi_si ipmi_devintf mei_me shpchp mac_hid mei lpc_ich acpi_power_meter ipmi_msghandler sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 raid0 multipath linear Apr 30 00:57:23 kernel: [106134.709360] hid_generic crct10dif_pclmul mgag200 crc32_pclmul usbhid i2c_algo_bit ghash_clmulni_intel ttm pcbc drm_kms_helper hid uas syscopyarea bnx2x sysfillrect aesni_intel usb_storage sysimgblt aes_x86_64 ptp fb_sys_fops crypto_simd pps_core glue_helper mdio ahci megaraid_sas cryptd drm libcrc32c libahci wmi Apr 30 00:57:23 kernel: [106134.709444] CPU: 36 PID: 23957 Comm: perl Tainted: G E 4.15.0-48-generic #51~lp1824981 Apr 30 00:57:23 kernel: [106134.709466] Hardware name: Dell Inc. PowerEdge R740/0923K0, BIOS 1.6.11 11/20/2018 Apr 30 00:57:23 kernel: [106134.709491] RIP: 0010:fortify_panic+0x13/0x22 Apr 30 00:57:23 kernel: [106134.709504] RSP: 0018:ffffac87f5ba7940 EFLAGS: 00010286 Apr 30 00:57:23 kernel: [106134.709519] RAX: 0000000000000022 RBX: 0000000000000004 RCX: 0000000000000000 Apr 30 00:57:23 kernel: [106134.709537] RDX: 0000000000000000 RSI: ffff90fc00c96498 RDI: ffff90fc00c96498 Apr 30 00:57:23 kernel: [106134.709555] RBP: ffffac87f5ba7940 R08: 0000000000000000 R09: 00000000000006a5 Apr 30 00:57:23 kernel: [106134.709572] R10: ffffac87f5ba79f0 R11: 00000000ffffffff R12: ffff9123571e6408 Apr 30 00:57:23 kernel: [106134.709591] R13: 0000000000000001 R14: 0000000000000003 R15: ffff90fbf676da00 Apr 30 00:57:23 kernel: [106134.709609] FS: 000014abed58dfc0(0000) GS:ffff90fc00c80000(0000) knlGS:0000000000000000 Apr 30 00:57:23 kernel: [106134.709629] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Apr 30 00:57:23 kernel: [106134.709643] CR2: 000014abec7ed110 CR3: 0000002f1b022002 CR4: 00000000007606e0 Apr 30 00:57:23 kernel: [106134.709661] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Apr 30 00:57:23 kernel: [106134.709679] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Apr 30 00:57:23 kernel: [106134.709696] PKRU: 55555554 Apr 30 00:57:23 kernel: [106134.710224] Call Trace: Apr 30 00:57:23 kernel: [106134.710756] smb21_set_oplock_level+0x147/0x1a0 [cifs] Apr 30 00:57:23 kernel: [106134.711262] smb3_set_oplock_level+0x22/0x90 [cifs] Apr 30 00:57:23 kernel: [106134.711758] smb2_set_fid+0x76/0xb0 [cifs] Apr 30 00:57:23 kernel: [106134.712260] cifs_new_fileinfo+0x259/0x390 [cifs] Apr 30 00:57:23 kernel: [106134.712765] ? smb2_get_lease_key+0x40/0x40 [cifs] Apr 30 00:57:23 kernel: [106134.713276] ? cifs_new_fileinfo+0x259/0x390 [cifs] Apr 30 00:57:23 kernel: [106134.713790] cifs_open+0x3db/0x8d0 [cifs] Apr 30 00:57:23 kernel: [106134.714308] do_dentry_open+0x1c2/0x310 Apr 30 00:57:23 kernel: [106134.714831] ? cifs_uncached_writev_complete+0x3f0/0x3f0 [cifs] Apr 30 00:57:23 kernel: [106134.715364] ? do_dentry_open+0x1c2/0x310 Apr 30 00:57:23 kernel: [106134.715899] ? __inode_permission+0x5b/0x160 Apr 30 00:57:23 kernel: [106134.716451] ? cifs_uncached_writev_complete+0x3f0/0x3f0 [cifs] Apr 30 00:57:23 kernel: [106134.717004] vfs_open+0x4f/0x80 Apr 30 00:57:23 kernel: [106134.717561] path_openat+0x66e/0x1770 Apr 30 00:57:23 kernel: [106134.718123] ? mem_cgroup_commit_charge+0x82/0x530 Apr 30 00:57:23 kernel: [106134.718693] do_filp_open+0x9b/0x110 Apr 30 00:57:23 kernel: [106134.719267] ? _cond_resched+0x19/0x40 Apr 30 00:57:23 kernel: [106134.719844] ? __kmalloc+0x19b/0x220 Apr 30 00:57:23 kernel: [106134.720433] ? security_prepare_creds+0x9c/0xc0 Apr 30 00:57:23 kernel: [106134.721013] do_open_execat+0x7e/0x1e0 Apr 30 00:57:23 kernel: [106134.721585] ? prepare_creds+0xd5/0x110 Apr 30 00:57:23 kernel: [106134.722149] ? do_open_execat+0x7e/0x1e0 Apr 30 00:57:23 kernel: [106134.722721] do_execveat_common.isra.34+0x1c7/0x810 Apr 30 00:57:23 kernel: [106134.723394] SyS_execve+0x31/0x40 Apr 30 00:57:23 kernel: [106134.724072] do_syscall_64+0x73/0x130 Apr 30 00:57:23 kernel: [106134.724716] entry_SYSCALL_64_after_hwframe+0x3d/0xa2 Apr 30 00:57:23 kernel: [106134.725321] RIP: 0033:0x14abec8b0e37 Apr 30 00:57:23 kernel: [106134.725911] RSP: 002b:00007ffcd32a6998 EFLAGS: 00000202 ORIG_RAX: 000000000000003b Apr 30 00:57:23 kernel: [106134.726508] RAX: ffffffffffffffda RBX: 0000561fccc9d050 RCX: 000014abec8b0e37 Apr 30 00:57:23 kernel: [106134.727087] RDX: 0000561fc7902cf0 RSI: 0000561fccc9d050 RDI: 0000561fcdd634e0 Apr 30 00:57:23 kernel: [106134.727655] RBP: 00007ffcd32a6a30 R08: 00007ffcd32a6a50 R09: 000014abecdd72b0 Apr 30 00:57:23 kernel: [106134.728205] R10: 0000000000000008 R11: 0000000000000202 R12: 0000561fc7902cf0 Apr 30 00:57:23 kernel: [106134.728746] R13: 0000561fc6158c00 R14: 0000561fce1e58e0 R15: 0000561fcdd634e0 Apr 30 00:57:23 kernel: [106134.729300] Code: e0 4c 89 e2 e8 41 6a 00 00 42 c6 04 20 00 48 89 d8 5b 41 5c 5d c3 0f 0b 55 48 89 fe 48 c7 c7 a8 a3 fa 85 48 89 e5 e8 bf 4b 76 ff <0f> 0b 90 90 90 90 90 90 90 90 90 90 90 90 90 55 31 c9 48 89 fa Apr 30 00:57:23 kernel: [106134.730455] RIP: fortify_panic+0x13/0x22 RSP: ffffac87f5ba7940 Apr 30 00:57:23 kernel: [106134.731050] ---[ end trace 2c5ad441b9fcf798 ]--- -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1824981 Title: cifs set_oplock buffer overflow in strcat Status in linux package in Ubuntu: Confirmed Bug description: Ubuntu 18.04.2 LTS Linux SRV013 4.15.0-47-generic #50-Ubuntu SMP Wed Mar 13 10:44:52 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux DELL R740, 2 CPU (40 Cores, 80 Threads), 384 GiB RAM top - 12:39:53 up 3:41, 4 users, load average: 66.19, 64.06, 76.90 Tasks: 1076 total, 1 running, 675 sleeping, 12 stopped, 1 zombie %Cpu(s): 28.2 us, 0.3 sy, 0.0 ni, 71.5 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st KiB Mem : 39483801+total, 24077185+free, 57428284 used, 96637872 buff/cache KiB Swap: 999420 total, 999420 free, 0 used. 33477683+avail Mem We've seen the following bug many times since we introduced new machines running Ubuntu 18. Wasn't an issue older machines running Ubuntu 16. Three different machines are affected, so it's rather not a hardware issue. | detected buffer overflow in strcat | ------------[ cut here ]------------ | kernel BUG at /build/linux-6ZmFRN/linux-4.15.0/lib/string.c:1052! | invalid opcode: 0000 [#1] SMP PTI | Modules linked in: [...] | Hardware name: Dell Inc. PowerEdge R740/0923K0, BIOS 1.6.11 11/20/2018 | RIP: 0010:fortify_panic+0x13/0x22 | [...] | Call Trace: | smb21_set_oplock_level+0x147/0x1a0 [cifs] | smb3_set_oplock_level+0x22/0x90 [cifs] | smb2_set_fid+0x76/0xb0 [cifs] | cifs_new_fileinfo+0x259/0x390 [cifs] | ? smb2_get_lease_key+0x40/0x40 [cifs] | ? cifs_new_fileinfo+0x259/0x390 [cifs] | cifs_open+0x3db/0x8d0 [cifs] | [...] (Full dmesg output attached) After hitting this bug there are many cifs related dmesg entries, processes lock up and eventually the systems freezes. The share is mounted using: //server/share /mnt/server/ cifs defaults,auto,iocharset=utf8,noperm,file_mode=0777,dir_mode=0777,credentials=/root/passwords/share,domain=myDomain,uid=myUser,gid=10513,mfsymlinks Currently we're testing the cifs mount options "cache=none" as the bug seems to be oplock related. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1824981/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
