Public bug reported: This bug has been introduced by an Oracle engineer, Chuck Lever, in "net/sunrpc/svcsock.c" as a series of optimizations in commit 2b877fc53e975bdc5614c0a7d64047cfdbae3894 , on 2023-07-19. It breaks NFS-RPC over TCP calls on clients under high load. In our case it is a bunch of rsync daemons, syncing multiple mailboxes content. As noted, we are using TCP protocol for NFS shares on Ubuntu 24.04, kernel 6.8, and that's why we've run into that bug.
Here is the client DMESG log snippet: ===================== [Fri Mar 21 05:26:02 2025] nfs: server ___________ not responding, still trying [Fri Mar 21 05:26:02 2025] nfs: server ___________ not responding, still trying [Fri Mar 21 05:26:02 2025] nfs: server ___________ not responding, still trying [Fri Mar 21 05:26:02 2025] call_decode: 12 callbacks suppressed [Fri Mar 21 05:26:02 2025] nfs: server ___________ OK [Fri Mar 21 05:26:02 2025] nfs: server ___________ OK [Fri Mar 21 05:26:02 2025] nfs: server ___________ OK ===================== Fortunately Chuck Lever has rolled his patch back in 966a675da844f1a764bb44557c21561cc3d09840 on 2025-01-03. Can we please cherry-pick this commit to kernel 6.8, since we've run into a time-window when the fix is unavailable yet? We are currently using Ubuntu 24.04 on both client and server, and the kernel sources, cloned with "apt source linux" still have this commit un-reverted. I have also un-patched these changes, and got my problem fixed, so sharing the solution with you. Thanks in advance! ProblemType: Bug DistroRelease: Ubuntu 24.04 Package: linux-image-6.8.0-55-generic 6.8.0-55.57 ProcVersionSignature: Ubuntu 6.8.0-55.57-generic 6.8.12 Uname: Linux 6.8.0-55-generic x86_64 NonfreeKernelModules: zfs AlsaDevices: total 0 crw-rw---- 1 root audio 116, 1 Mar 21 11:55 seq crw-rw---- 1 root audio 116, 33 Mar 21 11:55 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.28.1-0ubuntu3.3 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/timer', '/dev/snd/seq'] failed with exit code 1: CRDA: N/A CasperMD5CheckResult: pass Date: Fri Mar 21 12:06:45 2025 InstallationDate: Installed on 2025-02-13 (36 days ago) InstallationMedia: Ubuntu-Server 24.04.1 LTS "Noble Numbat" - Release amd64 (20240827) IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 001 Device 002: ID 1d6b:0107 Linux Foundation USB Virtual Hub Bus 001 Device 003: ID 0557:9241 ATEN International Co., Ltd SMCI HID KM Bus 001 Device 004: ID 0b1f:03ee Insyde Software Corp. RNDIS/Ethernet Gadget Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub MachineType: Supermicro Super Server PciMultimedia: ProcFB: 0 astdrmfb ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-6.8.0-55-generic root=/dev/mapper/vg0-root ro RelatedPackageVersions: linux-restricted-modules-6.8.0-55-generic N/A linux-backports-modules-6.8.0-55-generic N/A linux-firmware 20240318.git3b128b60-0ubuntu2.10 RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 07/23/2024 dmi.bios.release: 5.32 dmi.bios.vendor: American Megatrends International, LLC. dmi.bios.version: 2.3 dmi.board.asset.tag: Base Board Asset Tag dmi.board.name: X13SEB-TF dmi.board.vendor: Supermicro dmi.board.version: 1.01 dmi.chassis.asset.tag: Chassis Asset Tag dmi.chassis.type: 17 dmi.chassis.vendor: Supermicro dmi.chassis.version: 0123456789 dmi.modalias: dmi:bvnAmericanMegatrendsInternational,LLC.:bvr2.3:bd07/23/2024:br5.32:svnSupermicro:pnSuperServer:pvr0123456789:rvnSupermicro:rnX13SEB-TF:rvr1.01:cvnSupermicro:ct17:cvr0123456789:skuTobefilledbyO.E.M.: dmi.product.family: Family dmi.product.name: Super Server dmi.product.sku: To be filled by O.E.M. dmi.product.version: 0123456789 dmi.sys.vendor: Supermicro ** Affects: linux (Ubuntu) Importance: Undecided Status: New ** Tags: amd64 apport-bug noble -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2103802 Title: NFS-over-TCP fix needs backported from master To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2103802/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs