On Sat, Dec 10, 2022 at 08:48:56AM -0800, Adam Williamson wrote: > On Sat, 2022-12-10 at 11:59 +0000, Zbigniew Jędrzejewski-Szmek wrote: > > On Fri, Dec 09, 2022 at 03:08:19PM -0700, Chris Murphy wrote: > > > > > > > > > On Fri, Dec 9, 2022, at 7:30 AM, Ray Strode wrote: > > > > Hi, > > > > > > > > On Thu, Dec 8, 2022 at 2:55 PM Adam Williamson > > > > <adamw...@fedoraproject.org> wrote: > > > > > This is the direction Daniel was thinking down. I'm waiting for > > > > > someone > > > > > with more expertise to reply, but I suspect the reply is going to be > > > > > along the lines of "yes, we *can* do that, but it's somewhat tricky > > > > > work that involves thinking about lots of paths that aren't obvious, > > > > > and somebody would need to dedicate their time to working on that". > > > > Presumably we could package the firmware separately and just unpack it > > > > into place from a udev rule when the hardware is detected? > > > > > > > > But first, do we actually know this is a problem? > > > > I think you're saying squashfs loads the whole decompressed image into > > > > memory, but my expectation prior to your mail was that it performs I/O > > > > on the usb stick (with a cache in between). If my intuition was right > > > > and files only hit ram when accessed, then it seems like this is > > > > pretty much not an issue, right? > > > > > > From a certain point of view there's a potential inefficiency with > > > squashfs reads in that there's a minimum block size that it needs to read > > > in order decompress its 128 KiB block. It's possible quite a lot of > > > what's decompressed isn't (immediately) needed. But it's still a random > > > access file system. It's not necessary to read the whole image into RAM. > > > > > > Repo metadata is the big hit for netinstall because it's downloaded into > > > /tmp which is tmpfs. And DVD already has repomd on it, and only downloads > > > more if you enable some other repo. Live doesn't need repomd. > > > > > > So initially netinstaller uses less memory up until the the Anaconda > > > language selection screen appears, at which point it starts background > > > downloading repomd. It quickly catches up to, and surpasses, Live media > > > memory consumption. > > > > We *could* do something about repo metadata: only install the "main" > > metadata, > > and not the "filepath" metadata. This would reduce the metadata size by > > ~80%. > > It'd also have huge benefits for speed: on small dnf operations a > > significant > > portion of time is spent just loading metadata. > > > > We had some discussions on this a few years ago, but this never went > > anywhere. > > (I did a quick search, but can't find the ticket now.) But the idea was > > that > > dnf would load "main" metadata by default, and then load "filepath" metadata > > if needed and restart the transaction. Our Packaging Guidelines forbid > > filepath > > dependencies (except for a small set of directories which are in the "main" > > part), > > so normal rpm transactions don't need the "filepath" metadata. It is only > > needed for non-confirming rpms and when the user does something like > > 'dnf install /some/strange/path'. > > AIUI this is already a part of dnf5.
I would love to hear more about this. A quick test suggests that on F37 dnf5 at least downloads the same metadata that dnf4: $ sudo dnf upgrade Fedora 37 - x86_64 - Updates 46 kB/s | 12 kB 00:00 Fedora 37 - x86_64 - Updates 1.3 MB/s | 2.4 MB 00:01 ... $ sudo dnf5 upgrade Fedora 37 - x86_64 - Updates 100% | 44.4 KiB/s | 12.4 KiB | 00m00s Fedora 37 - x86_64 - Updates 100% | 724.6 KiB/s | 2.0 MiB | 00m03s ... Zbyszek _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
