Partially top-posting because there are some assumptions here that need cleared up.
Hi Johannes, and thanks for your feedback. I would like to point out that I explicitly said in the very start of my email, that this was something I was "interested in potentially contributing code to fix." I see this part got omitted from what you quoted of my email, so it may have gone unnoticed. I'm fully intend to implement and test much or even most of the code require to make this happen should it be considered desirable. So in reply to your "Do you plan to send patches?" questions, yes, I intended to send patches from the moment I started writing the email. It's generally understood and oftentimes spelled out in open-source projects that one should discuss and get feedback on a large change before even starting to implement it, which is exactly what I'm doing here. I know that my proposal, as given, is likely to not be accepted as is and may even be rejected altogether, which is why I said "potentially". I would appreciate if you would look at my suggestion in this light rather than assuming I'm just coming up with a "great idea" that I expect everyone else to work on without any further help from me. It may also help to look at this as an initial "testing the waters" conversation starter, and not as a proposal in its entirety. That's what the DEP process is for. In your reply you mention "any good proposal includes disadvantages", which is true. This doesn't have those because it's not meant to be a formal proposal. The only reason I sent this on the mailing lists rather than simply starting a conversation on IRC is because I wanted it to get wider visibility. The rest of my replies are inline. On Wed, 06 Nov 2024 02:28:33 +0100 Johannes Schauer Marin Rodrigues <jo...@debian.org> wrote: > Hi, > > Quoting Aaron Rainbolt (2024-11-06 00:35:59) > > According to the Debian Policy Manual, section 7.2, the Recommends > > field in Debian packages "declares a strong, but not absolute, > > dependency. The Recommends field should list packages that would be > > found together with this one in all but unusual installations." > > While this is a very useful definition, the actual way in which > > Recommends are used seems to differ substantially from this. > > then you should file bugs against the packages that violate this part > of policy. > > > If this was just a diffoscope problem, it would be easy to just > > file a bug asking that most of these packages be demoted to > > Suggests, but this is a much more pervasive issue, as evidenced by > > the fact that the live-build manual has special instructions for > > how to disable the installation of *all* recommended packages when > > building a live image[1]. I have built live images that ended up > > with all sorts of weird packages installed on them, which issue was > > resolved by disabling the installation of recommended packages. > > On my system, I have apt configured to not install Recommends by > default because I want to manually pick what to install when I > install new things. Oftentimes I find the extra things that get > installed too bloated for my taste, so I sympathize with your quest, > but I do not agree with your solution. > > > Furthermore, the current (ab)use of Recommends in Debian packages > > illustrates something important - there is a real need for > > packagers to specify packages that should automatically be > > installed alongside another package, but that aren't necessarily > > strong dependencies. Using diffoscope again as an example, it's > > reasonable that the diffoscope maintainers want *all* of > > diffoscope's functionality to "just work" out of the box, even if > > that means installing over three and a half gigabytes of packages > > to do it.[2] This may not be policy-compliant, but demoting these > > packages to the "Suggests" field doesn't feel right. Should a user > > who just wants to compare things have to figure out the right combo > > of packages to make diffoscope work for their particular use > > case?[3] There's also the question of logistics - going through and > > "fixing" all of the packages with overkill Recommends could be a > > massive undertaking, and it's more than likely that there will be > > some packagers who won't be thrilled with the changes. > > This argument goes both ways. Your proposed solution has the same > drawbacks. To properly implement your solution, you still have to go > through and fix all of the packages with overkill Recommends which is > a massive undertaking and it's not unlikely that there will be some > packagers who won't be thrilled with the change (as it usually is the > case for any kind of change you propose to somebody's package). Specifically for this point, the reason I thought that my proposed solution would be superior to refactoring packages or demoting things to Suggests is because both of those solutions require somewhat invasive or significantly functionality-modifying changes to packages. The suggested Weak-Depends field's end goal was to allow resolving the problem with a very small change to each package that needed help, which change would have almost no functional effect on the package but would alleviate the problem. If it was my package someone wanted to modify, I know I'd be much more amicable to a solution that worked that way than to a functionality-changing modification or a packaging overhaul. > > Weak-Depends would basically just be a stronger "Recommends". > > I don't think that the solution to your problem is to make the > Recommends/Suggests logic more granular. You still have to convert > all the packages which you think have too many Recommends. Even > assuming that their maintainers agree, instead of doing that work, > why not invest that time in finding a solution using the existing > mechanisms instead? You mentioned that meta-packages are besides the > point. Why? Have one package diffoscope and one package > diffoscope-full and you could even have a package diffoscope-minimal > and there you have user-selectable granularity. This gets into what I just mentioned about packaging overhauls. Sure, diffoscope *could* be refactored so that there was a base "engine" package, a number of metapackages for different use cases, and a "full" metapackage that would pull in all 3.5 GB of packages like it does today. That would even be a great solution, and for this particular package it would work even better than a Weak-Depends field. So why did I consider it was beside the point? Because of what I just mentioned - metapackage-based solutions are for cases that are too complicated for Weak-Depends to handle perfectly. It would do the job, it just wouldn't achieve a "shining city on the hill" result the way a metapackage-based solution would. > > Some of the advantages of this solution include: > > Any good proposal also includes disadvantages. Any suggestion to > change something should come with an attempt to weigh the benefits > against the costs. In your mail you do not make an argument about the > costs of your proposed change. I'd argue, that the cost of your > proposal are too high. Especially when comparing that cost to using > the existing packaging mechanisms to achieve essentially the same > thing. See my note at the top about this being a conversation starter, and not a formal proposal. > > * It requires comparatively few changes to initially implement. > > I strongly disagree. I think you are estimating the number of > packages which attempt messing around with Debian packages and their > dependencies and which would have to receive a patch to support a new > field. > > > All > > existing packages in the Debian repository will be compliant with > > a Debian Policy Manual update that adds Weak-Depends, without > > changes. > > I thought your proposal was to make Weak-Depends effectively what > Recommends is today? Yes. My point is that Recommends will still be able to function the way it does today even with Weak-Depends in existence. But packages can also migrate to using Weak-Depends in addition. > > Packagers can start using Weak-Depends if they want to or if a bug > > report requests that they do. Some of the packages that would > > need to change to implement this would be dpkg, apt, possibly the > > libraries they depend on, and live-build. > > Yes. And sbuild, python-apt, aptitude, mmdebstrap, debootstrap, > cdebootstrap, debhelper, cdbs, lintian, pbuilder, dose3, wanna-build, > dak, python-debian, libconfig-model-dpkg-perl, augeas, > haskell-debian, dh-exec, autopkgtest and very many tools in > devscripts like build-rdeps, debrebuild, wrap-and-sort... And then > there is all the work that our downstream distros need to do for > their tooling as well. *This* is the kind of feedback I was hoping for. This is very useful info to have, thank you for pointing this out. > Do you plan to send patches? Yes, as noted above. > > There's probably others here, but > > nonetheless it wouldn't require a massive overhaul of the whole > > archive to make it start working, the way a mass-"demote to > > Suggests" operation would. > > I disagree. The change you are proposing requires core changes to > very many tools, potentially creating new bugs and lots of friction > going forward. A mass-demote, if you want to drive that, can be done > package-by-package at your own time without breaking the archive, our > tooling and our downstreams. > > > * Change the definition of the "Recommends" field to match the way > > the field is oftentimes used in the Debian archive. > > Please do it the other way round. File bugs where packages do not > follow policy and use workarounds like your meta-package idea in > other cases. > > > * Suggest that all active package maintainers in Debian review the > > packages they maintain and see if they feel there are some > > packages that should be promoted from "Recommends" to > > "Weak-Depends". > > Your proposal would be much simpler if it would just be this item but > about demoting Recommends to Suggests. I get the feeling, from how > you are writing this, that you expect others to do the work for you? > I fear that little change will happen unless you are putting in the > work yourself. File bugs and attach patches. > > > * (Potentially?) Scan the Debian archive and see if there are > > dependencies that should be promoted from "Recommends" to > > "Weak-Depends". This would probably be something that interested > > Debian Developers and other volunteers could slowly chip away at, > > as they had time and the desire to do so. > > Why am I so negative? I was one of the main driver behind the > introduction of the Build-Profiles field. It was a lot of pain to do > this and it took a couple of years. But to get the desired > functionality, there was not much else that could be done than to add > a new field. In your case, there are alternatives using the existing > mechanisms. I suggest you instead think about how you can use the > existing machinery to achieve the desired effect. I fear you are > vastly underestimating the crazy amount of work that introducing a > new dependency relationship field incurs. Please try having a more > realistic look at the cost/benefit ratio of what you are proposing > here. I think I can summarize much of what you're saying as "why not make it work with the existing infrastructure?" To be clear, I fully agree that the existing infra *can* be used to fix the issue without further features. Should it? I explained the reasons I thought perhaps it shouldn't in my initial email. I don't expect people will want to change their packages to make their Recommends fields policy-compliant, and while I certainly can use the policy to force people to "do it right", it seemed to me like the policy violations were an indicator that the policy was wrong and the existing machinery insufficient for packagers to easily express what they were trying to express in their packaging code. I gather from your email that this is a larger of a job than I made it sound like (which makes sense since I didn't spend a huge amount of time considering what would need changes since this is, after all, just a conversation starter), so maybe that means that my suggestion is not one that is ultimately worth it. The fact that you've been down this road already is also very good to know. I would like to point out that, as you are well aware of, developers don't appreciate feedback that is intentionally worded in a highly negative fashion. I'm no different from most others in this respect. I think the information you've shared is valuable, especially the feedback on the amount of work it would take to implement this (something I did not fully consider when writing this since this was just intended to be a "hey here's an idea I'm interested in, I wonder what everyone else thinks" post). But the harsh and somewhat hurtful assumptions scattered throughout aren't appreciated, and I don't think they're necessary to make your point at least to me. (Certainly if I were a random person who just wanted to dump work on others, this response would be potentially warranted and very effective at getting said random person to just go away. But seeing as I'm actually intending to do the work needed to make things happen, I don't *think* you want me to just go away. :P) I've made a conscious effort to look past those assumptions as much as possible and just get to the technical details, and I am thankful for the insight you've provided. Thanks again for taking a look at this! Aaron > Thanks! > > cheers, josch
pgpbFGjt950lC.pgp
Description: OpenPGP digital signature
