On Thu, Jan 16, 2025 at 8:28 PM Gao Xiang <hsiang...@linux.alibaba.com> wrote: > > > > On 2025/1/17 09:14, Neal Gompa wrote: > > On Thu, Jan 16, 2025 at 8:00 PM Gao Xiang <hsiang...@linux.alibaba.com> > > wrote: > >> > >> (resend this due to lack of subscription. and I just quote some reply to > >> express my basic ideas on this stuff) > >> > >>> There's one thing the EROFS developers never talk about, and that's > >>> metadata compression. EROFS doesn't do it, but Squashfs does, and it has > >>> done since 2002. > >> > >> Did you notice that https://erofs.docs.kernel.org/en/latest/faq.html ? > >> > >>> The tests are all done in way to flater EROFS and put Squashfs in the > >>> worst light. For example see > >>> https://erofs.docs.kernel.org/en/latest/comparsion/dedupe.html. They > >>> deliberately disable Squashfs metadata compression because EROFS doesn't > >>> support it, and use 128K blocks despite Squashfs supporting 1Mbyte blocks > >>> (look for the line [1] SquashFS uses –b 131072 by default, -noI will > >>> disable its metadata compression.). > >> > >> It just shows that SquashFS doesn't support extent-based deduplication > >> (because SquashFS on-disk indexes just record compressed sizes and cannot > >> be randomly indexed), since EROFS doesn't support metadata compression so > >> I benchmarked against SquashFS and explicitly point out the metadata > >> compression is off on the SquashFS side to show the benefits of > >> deduplication. Why it's called unbiased? > >> > >> Don't make me wrong, if -b 1048576 (1m chunks) is used, then the chunk > >> deduplication possibility is low. > >> But not anyone cares image size (especially for smartphone) just by using > >> large extent sizes (like 1MiB) and real-time performance is important for > >> them, decompress 1MiB to get 4KiB random data is unacceptable for them. > >> > >>> By doing this they deliberately reduce the compression and speed of > >>> Squashfs in their tests. This IMHO is biased and unethical. But that > >>> is how it is. > >> > >> Also if I were a random filesystem author, I will never argue just due to > >> lack of development resource. > >> > >> Whether EROFS will use for Fedora LiveOS finally, honestly I don't care > >> (although I wish it could be used more widely), I will focus on new > >> features that end users really care and it's really an input. I only care > >> EROFS can be totally usable on RHEL/CentOS since we have customers to use > >> these OSes and I will serve for their detailed use cases. > >> > >> Please note, EROFS was once proposed to resolve SquashFS unacceptable > >> random runtime performance issues on the high-performance Android > >> scenerios, and almost all smartphone vendors on the market use EROFS now. > >> As for other scenarios, I would just call it as a plus. > >> > >> If users think metadata compression is useful to minimize their images, > >> and I will consider to add this in my spare time (Because it's never > >> useful for my paid job too, they even don't care about multithreaded > >> compression). Also I did not ask random vendors to contribute EROFS, just > >> because Bytedance or other vendors need it and they develop new features > >> for their use cases. > >> > > > > I certainly would appreciate features like metadata compression, BCJ > > filters, and anything to improve image creation time when -Ededupe is > > used. I also wanted to use zstd compression instead of lzma (since it > > decompresses way faster), but the compression performance isn't quite > > there yet. > > Since it's not my paid job, I will try my best to work them out in > my spare time. But really, for our own use cases we don't care much > about them, it's also called "a labor of love". >
I understand this very well. Most of my FOSS work would fall in that category too. > > > > Runtime speed is really valuable for us because typical media that > > Linux live environments are running from are slower than even most > > mobile device flash storage modules. At worst, we're talking about > > actually booting from a DVD (which is still a thing people do in some > > places), but we also need maximal space savings because of download > > bandwidth and fitting on small media. > > Anyway, EROFS was once specifically designed for Android devices > initially, I will try to make it work better for other useful cases, > questions, issues, and features are much helpful for me to improve > the filesystem itself. > Thank you, I appreciate whatever you can do to make things better for our use-cases. :) -- 真実はいつも一つ！/ Always, there's only one truth! -- _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
