On 7/1/20 9:24 AM, Josef Bacik wrote: > On 7/1/20 7:49 AM, Steven Whitehouse wrote: >> Hi, >> >> On 01/07/2020 12:09, Zbigniew Jędrzejewski-Szmek wrote: >>> On Wed, Jul 01, 2020 at 11:28:10AM +0100, Steven Whitehouse wrote: >>>> Hi, >>>> >>>> On 01/07/2020 07:54, Zbigniew Jędrzejewski-Szmek wrote: >>>>> On Mon, Jun 29, 2020 at 03:15:23PM -0400, Solomon Peachy wrote: >>>>>> So yes, I think an explicit "let's all test btrfs (as anaconda >>>>>> configures it) before we make it default" period is warranted. >>>>>> >>>>>> Perhaps one can argue that Fedora has already been doing that for the >>>>>> past two years (since 2018-or-later-btrfs is what everyone with positive >>>>>> results appears to be talking about), but it's still not clear that >>>>>> those deployments utilize the same feature set as Fedora's defaults, and >>>>>> how broad the hardware sample is. >>>>> Making btrfs opt-in for F33 and (assuming the result go well) opt-out for >>>>> F34 >>>>> could be good option. I know technically it is already opt-in, but it's >>>>> not >>>>> very visible or popular. We could make the btrfs option more prominent and >>>>> ask people to pick it if they are ready to handle potential fallout. >>>>> >>>>> Normally we just switch the default or we don't, without half measures. >>>>> But >>>>> the fs is important enough and complicated enough to be extra careful >>>>> about >>>>> any transitions. >>>>> >>>>> Zbyszek >>>> Indeed, it is an important point, and taking care is very important >>>> when dealing with other people's data, which is in effect what we >>>> are discussing here. >>>> >>>> When we looked at btrfs support in RHEL, we took quite a long time >>>> over it. In fact I'm not quite sure how long, since the process had >>>> started before I was involved, but it was not a decision that was >>>> made quickly, and a great deal of thought went into it. It was >>>> difficult to get concrete information about the stability aspects at >>>> the time. Just like the discussions that have taken place on this >>>> thread, there was a lot of anecdotal evidence, but that is not >>>> always a good indicator. Since time has passed since then, and there >>>> is now more evidence, this part of the process should be easier. >>>> That said to get a meaningful comparison then ideally one would want >>>> to compare on the basis of user populations of similar size and >>>> technical skill level, and look not just at the overall number of >>>> bugs reported, but at the rate those bugs are being reported too. >>> Yeah. I have no doubt that the decision was made carefully back then. >>> That said, time has passed, and btrfs has evolved and our use cases >>> have evolved too, so a fresh look is good. >>> >>> We have https://fedoraproject.org/wiki/Changes/DNF_Better_Counting, >>> maybe this could be used to collect some statistics about the fs type >>> too. >> >> Yes, and also the questions that Fedora is trying to answer are different >> too. So I don't think that our analysis for RHEL is applicable here in >> general. The method that we went through, in general terms, may potentially >> be helpful. >> >> >>>> It is often tricky to be sure of the root cause of bugs - just >>>> because a filesystem reports an error doesn't mean that it is at >>>> fault, it might be a hardware problem, or an issue with volume >>>> management. Figuring out where the real problem lies is often very >>>> time consuming work. Without that work though, the raw numbers of >>>> bugs reported can be very misleading. >>>> It would be worth taking that step here, and >>>> asking each of the spins what are the features that they would most >>>> like to see from the storage/fs stack. Comparing filesystems in the >>>> abstract is a difficult task, and it is much easier against a >>>> context. I know that some of the issues have already been discussed >>>> in this thread, but maybe if someone was to gather up a list of >>>> requirements from those messages then that would help to direct >>>> further discussion, >>> Actually that part has been answered pretty comprehensively. The split >>> between / and /home is hurting users and we completely sidestep it >>> with this change. The change page lists a bunch of other benefits, >>> incl. better integration with the new resource allocation mechanisms >>> we have with cgroups2. So in a way this is a follow-up to the >>> cgroupsv2-by-default change in F31. Snapshots and subvolumes also give >>> additional powers to systemd-nspawn and other tools. I'd say that the >>> huge potential of btrfs is clear. It's the possibility of the loss of >>> stability that is my (and others') worry and the thing which is hard >>> to gauge. >>> >>> Zbyszek >> >> If the / and /home split is the main issue, then dm-thin might be an >> alternative solution, and we should check to see if some of the issues >> listed on the change page have been addressed. I'm copying in Jon for >> additional comment on that. Are those btrfs benefits which are listed on the >> change page in priority order? >> >> File system resize is mentioned there, but pretty much all local filesystems >> support grow. Also, no use cases are listed for that benefit. Shrink is more >> tricky, and can easily result in poor file layouts, particularly if there >> are repeated grow/shrink operations, not to mention potential complications >> with NFS if the fs is exported. So is there some specific use case there >> that cannot be supported easily with the existing tools? There are a few >> other features listed that are available in other fs/volume management tools >> as well. >> >> Eric has already pointed out that XFS has cgroups2 support, so the statement >> that btrfs is the only fs with that is incorrect. It would help to make >> things a bit clearer if that list was updated, with the information gathered >> so far, >> > > Yeah that should be changed. > > There's a big gap between having cgroups2 support and it actually working.
Well, that's why dchinnner pushed back on the first patches from FB; there was no way for us or any other filesystem to validate that it worked, or would continue to work. > The thing that I've said consistently is that there's nothing keeping XFS > from working with cgroups2, it's just that we (Facebook) haven't tested it, > because at the time we were rolling it out it didn't have writeback support. > > Even btrfs with writeback support enabled still required a few investigations > and follow up work to get everything working properly, because you don't ever > know what's going to break until you actually use it. So while XFS > technically has support, Btrfs is the only fs that we use cgroup2 with IO > isolation in production, so it's the only thing we're comfortable with. XFS > may work perfectly fine, but AFAIK nobody has ever tested it or used it in > production. Thanks, The work was done by Christoph and was sponsored and tested by Profihost AG, who presumably use it in production. -Eric _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
