Then I did the same on 5.19. There is a huge difference. I am fairly certaan that everything from "zio_execute" to "SHA256Tra..." is time spent in ZFS. To the right right side of `SHA256Tra...` is the compiler, I checked this when looking at it in more detail. The compiler graph is basically the same thing as in the 5.15 flamegraph, just relatively much smaller, because so much CPU is wasted otherwise. To the left the huge red block is kernel overhead as far as I can tell.
** Attachment added: "flamegraph-19.png" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2007717/+attachment/5649867/+files/flamegraph-19.png -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2007717 Title: Kernel linux-image-5.19.0-32 regression, compile speed down by 33% Status in linux package in Ubuntu: Incomplete Bug description: I just yesterday and the day before made some benchmark tests for a comparison between ext4 and zfs. I ran repeated builds of a large software package and it always came out to 45s. Now today I ran the same test a few times and it was always 60s. I noticed that today an update ran and that kernel 5.19.0-32 was installed. So just now I booted back to 5.15.0-60-generic:amd64 and ran the test again: back to 45s. I am running Ubuntu 22.04.1 LTS on a AMD Ryzen 3970X with 128GB RAM otherwise there is nothing very special going on. I ran the tests on ZFS and ext4, it makes no difference. To reproduce, I would think that compiling any decent sized software package one on 5.15 and then on 5.19 whould yield the same results. --- ProblemType: Bug ApportVersion: 2.20.11-0ubuntu82.3 Architecture: amd64 CRDA: N/A CasperMD5CheckResult: pass CurrentDesktop: ubuntu:GNOME DistroRelease: Ubuntu 22.04 InstallationDate: Installed on 2022-07-15 (217 days ago) InstallationMedia: Ubuntu 22.04 LTS "Jammy Jellyfish" - Release amd64 (20220419) MachineType: To Be Filled By O.E.M. To Be Filled By O.E.M. NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair nvidia_modeset nvidia Package: linux (not installed) ProcFB: 0 EFI VGA ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.19.0-32-generic root=UUID=c1b17af6-e10d-4836-a5d0-aa0c25802bca ro mitigations=off quiet splash vt.handoff=7 ProcVersionSignature: Ubuntu 5.19.0-32.33~22.04.1-generic 5.19.17 RelatedPackageVersions: linux-restricted-modules-5.19.0-32-generic N/A linux-backports-modules-5.19.0-32-generic N/A linux-firmware 20220329.git681281e4-0ubuntu3.10 Tags: wayland-session jammy Uname: Linux 5.19.0-32-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: adm audio cdrom davfs2 dip docker libvirt lpadmin lxd plugdev sambashare sudo _MarkForUpload: True dmi.bios.date: 06/01/2020 dmi.bios.release: 5.15 dmi.bios.vendor: American Megatrends Inc. dmi.bios.version: P1.70 dmi.board.name: TRX40 Taichi dmi.board.vendor: ASRock dmi.chassis.asset.tag: To Be Filled By O.E.M. dmi.chassis.type: 3 dmi.chassis.vendor: To Be Filled By O.E.M. dmi.chassis.version: To Be Filled By O.E.M. dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvrP1.70:bd06/01/2020:br5.15:svnToBeFilledByO.E.M.:pnToBeFilledByO.E.M.:pvrToBeFilledByO.E.M.:rvnASRock:rnTRX40Taichi:rvr:cvnToBeFilledByO.E.M.:ct3:cvrToBeFilledByO.E.M.:skuToBeFilledByO.E.M.: dmi.product.family: To Be Filled By O.E.M. dmi.product.name: To Be Filled By O.E.M. dmi.product.sku: To Be Filled By O.E.M. dmi.product.version: To Be Filled By O.E.M. dmi.sys.vendor: To Be Filled By O.E.M. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2007717/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : [email protected] Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp

