I also want to state for the record that all of my criticism is directed towards the article, and not towards you Nils.
-- Jason Liu On Sat, Aug 30, 2025 at 8:46 PM Jason Liu <jason...@umich.edu> wrote: > I very strongly disagree with almost everything in the article that you > linked. I believe that the author's entire premise is wrong. They somehow > conflate reproducible builds with the ability to specify version ranges in > the build system. Yes of course you can use the ability to specify version > ranges to create non-reproducible builds, but even right now in MacPorts, > you can easily write Tcl code in a Portfile that can result in > non-reproducible builds. Having version ranges doesn't automatically mean > that all builds are now suddenly non-deterministic and non-reproducible. > > In addition, I would argue that the two examples that they provide, the > one with the conversation between Bob and Alice and the second one with the > trailing comma bug, don't actually have any relation to version ranges at > all. Both of these scenarios occur quite often during the course of both > software development and systems administration work... it can happen _any_ > time software gets upgraded. It even happens in MacPorts right now, as > occasional messages on this very mailing list can attest to. > > I do agree that "1.8.0 - Infinity is a bad version range". But this is > why, if you look back to my examples, I nearly always use a range that has > limits on both ends of the range. In each and every version range example > the author provides, it always shows a "blahblah version AND ABOVE". The > author completely ignores ranges that have limits on both ends. > > I also find their list of suggested solutions to range from (yes, pun > intended) unhelpful to downright bewildering. Don't allow version range? > Well, that is the current situation of MacPorts, so that's unhelpful. Use > version control outputs and copy the binaries for dependencies into your > repo? Wait, what?! Seriously? Look, I know that MacPorts is usually used on > macOS, and Apple loves to "sandbox" everything by always including compiled > dependencies (e.g. libraries) in each and every app bundle, but MacPorts, > and FreeBSD ports collection, and every other *nix-style package management > system, are specifically designed to _avoid_ having copies of compiled > binaries all over the place in each and every project's source code tree. > Copy repos? Why would anyone do this, when the MacPorts project uses git, > which has the ability to reference other repos through the use of git > submodules? > > I also find their statement that "Any dissenting opinion lacks imagination > (and experience)" to be incredibly condescending. Because even what this > statement is referring to, that "there's no such thing as a 100% innocuous > change", doesn't inherently have anything to do with the ability to specify > version ranges. Upgrading 1.8.0 to 1.8.1 causing months of stored data to > be lost has everything to do with managing your own dev group's software > update scheduling, it has nothing to do with being able to specify version > ranges. Software upgrades causing data loss is the very reason why most > professional-level software engineering teams have dev, staging, and prod > servers that are completely separate, in order to prevent this very > scenario from happening. > > The author is even forced to admit, in smaller text as a footnote, that: > "FYI, there actually is one valid reason for using version ranges: to > document/enforce compatibility. This is the intent in package managers like > apt and yum." This is precisely the applicable case for MacPorts: it's a > package manager. > > -- > Jason Liu > > > On Sat, Aug 30, 2025 at 12:17 PM Nils Breunese <br...@macports.org> wrote: > >> I agree that version awareness would be a very helpful concept. All other >> dependency mechanisms that I’ve encountered (Maven, NPM, etc.) have this >> ability. In MacPorts for some ports there are separate ports for different >> major versions, but sadly in 2025 backwards compatibility is still very >> often broken in non-major updates, so being able to depend on a specific >> version would be very welcome. >> >> I do believe supporting version *ranges* for dependencies is a mistake, >> because they break reproducible builds. See e.g. >> https://lucid.co/techblog/2017/03/15/package-management-stop-using-version-ranges >> . >> >> Nils. >> >> Op 30 aug 2025 om 17:38 heeft Jason Liu <jason...@umich.edu> het >> volgende geschreven: >> >> >> [Warning: Apologies ahead of time. This reply message is super long, but >> hopefully everything I've said is relevant.] >> >> Please correct me if I'm wrong here, but it seems like the "mutating >> port" that you talk about would probably be referred to by various *nix >> package management systems as a "metapackage" or a "virtual package". For >> example, a couple of years ago I worked with @Gcenx to modify the MoltenVK >> port to behave like a virtual package, which attempts to select the correct >> versioned subport based on the user's OS version. The unversioned MoltenVK >> port doesn't install any binary itself, it only mutates which versioned >> subport gets specified as the 'depends_run'. >> >> ========== >> >> I have long advocated for moving MacPorts to be a fully version-aware >> package management system. All modern PMSes allow specifying version >> requirements when listing dependencies. I particularly like the robust way >> that Debian implemented it when they rewrote Apt from the ground up for >> version 2.0 ( >> https://www.debian.org/doc/debian-policy/ch-relationships.html), e.g.: >> >> Package: mutt >> Version: 1.3.17-1 >> Depends: libc6 (>= 2.2.1), default-mta | mail-transport-agent >> >> (Here's a more complicated example involving ranges: >> https://askubuntu.com/questions/766846/version-range-in-debian-control) >> >> Even FreeBSD's Ports has implemented the ability to specify dependency >> version requirements ( >> https://docs.freebsd.org/en/books/porters-handbook/makefiles/#makefile-version-dependency >> ): >> >> p5-Spiffy>=0.26:devel/p5-Spiffy >> >> I'm fully cognizant that this would involve a major re-architecting of >> macports-base and probably also the server-side infrastructure, but I >> continue to feel that, at some point, the MacPorts project _must_ implement >> this sort of a feature in order to continue to be viable as a modern PMS >> far into the future. >> >> ========== >> >> Sometimes the single mutating port strategy doesn't work so well. The go >>> port is an example. Many go modules fail to build on older systems because >>> they require a newer go than the latest one that works on the older system. >>> But this wouldn't be solved by switching to multiple versioned go ports. >>> Each go module port would still need to know its minimum go version which >>> it currently doesn't. >> >> >> I believe this problem would be completely solved by having a >> version-aware PMS. >> >> First, let's assume that all ports are versioned, because we have a >> version-aware PMS. When an older system is asked to install a particular go >> module, it will automatically know that it needs to install an older >> version of the module, because the module itself is an unversioned virtual >> package that selects the go module version based on OS version. The >> versioned (sub)port of the module will attempt to install the go compiler >> as a dependency; this dependency line in the Portfile has a version range >> specified, i.e. 'depends_build: go (>= 2.6.1 && <= 3.1.7)', which means >> that the newest version of go within that version range will get added to >> the dependency tree. Thus, an old version of go that is compatible with the >> old version of the module will get installed, followed by the old version >> of the go module that is compatible with the old OS version. >> >> The separate versioned ports strategy also doesn't always work well >>> because it takes continual effort to keep them in sync. >> >> >> Not necessarily. If you consider the versioned ports to basically be >> immutable after they have been fully packaged, then you can consider them >> to be historical archived snapshots of each version of the port. (Okay, >> immutable is not really the right word, because Portfiles might need to be >> altered with minor fixes and revbumps.) Really the only thing that would >> take continual effort to keep in sync would be the Portfiles for the >> top-level "unversioned" virtual packages. >> >> If we go back to the go and go-module example, then this is what the >> various Portfiles would look like (I'm going to use pseudocode here, please >> don't hate me): >> >> In the go module's unversioned virtual package >> (ports/go/go-module/Portfile): >> >> if {${os.major} <= 15} { >> depends_run go-module (>= 14.1.7 && <= 16.2.2) >> } elseif {${os.major} == 16} { >> depends_run go-module (>= 16.3.0 && <= 17.4.4) >> } else { >> depends_run go-module (>= 18.0.0) >> } >> >> In the go module's OLD versioned (sub)port >> (ports/go/go-module/16.2.2/Portfile): >> >> depends_build go (>= 3.0.0 && <= 4.3.7) >> >> In the module's NEW versioned (sub)port >> (ports/go/go-module/18.0.0/Portfile): >> >> depends_build go (>= 6.0.0 && <= 7.1.4) >> >> As you can see, because the Portfile for go-module v16.2.2 exists at the >> same time as the Portfile for go-module v18.0.0, and the valid version >> ranges for each version of the go module will always be specified in the >> versioned (sub)port's Portfile, installing that particular version of the >> go module will always specify a valid version of the go compiler. >> >> Note that this also elegantly takes care of your example of >> >> For example, if I later figure out a way to offer version 5 to some older >>> OS versions, how does the user who installed cliclick4 learn of that? >> >> >> With a version-aware PMS, let's say that you figure out how to get v5.0.0 >> of the go compiler to work with Darwin 15. Then in the go module's old >> versioned (sub)port (ports/go/go-module/16.2.2/Portfile), you would update >> it to be this: >> >> revision 5 # revbump >> depends_build go (>= 3.0.0 && <= 5.0.0) >> >> It wouldn't disturb go-module v18.0.0 in any way, because both of >> their Portfiles exist at the same time in the ports tree. >> >> I know what you're thinking. The logical next question would be: "What >> about all of the Portfiles for go-module from versions 14.1.7 through >> 16.2.1? Would those now also need to be changed so that the 'depends_build' >> line for go is also 5.0.0?" And my answer to that would be: You could if >> you wanted to, but I don't think it would be absolutely necessary. Because >> what's the likelihood that someone would want to use go-module v16.2.1, or >> v15.y.z, or v14.1.7, when v16.2.2 is available? >> >> On the one hand, in the scenario where you don't change those other >> Portfiles, they would still work fine, it's just that they would still be >> dependent on the go compiler being (>= 3.0.0 && <= 4.3.7). It wouldn't >> break anything, it's just that anyone who specifically wants to install one >> of those particular older versions of the module wouldn't receive the >> benefit of your efforts on getting go-compiler v5.0.0 to work on Darwin 15. >> On the other hand, if you wanted to risk it, you could mass update all of >> the Portfiles for go-module v14.1.7 through v16.2.2 to use go-compiler >> v5.0.0 by using sed or awk commands. >> >> ========== >> >> I think one trade-off of a fully version-aware PMS might be that it could >> explode the amount of storage needed, if you were to start retaining all of >> the pre-compiled binaries for versioned ports. But, other PMSes seem to be >> able to handle this somehow, so I'm sure there must be a way to balance the >> trade-offs. Perhaps you could have some way to specify which versions of >> old pre-compiled binaries to keep on the servers, and leave the other >> unused versioned ports with only the control files (i.e. the Portfile) >> present, forcing the user to build no-longer used versioned ports from >> source locally on their own machines. >> >> -- >> Jason Liu >> >> >> On Fri, Aug 29, 2025 at 10:27 AM Ryan Carsten Schmidt < >> ryandes...@macports.org> wrote: >> >>> Should we continue to allow changing the port version based on OS >>> version? Currently we do this in several ports e.g. to offer the last >>> compatible version to old OS versions. Josh brought up in a ticket that we >>> should consider not doing that, and instead create separate versioned >>> ports. I thought we should discuss because there are pros and cons to each >>> approach and currently we have a mix of strategies. >>> >>> For example a user who wants cliclick just installs the port. It picks >>> the right version for the OS version. This is simple for the user. The >>> alternative is that a user of an old OS would have to know to install a >>> hypothetical cliclick4 instead. And the situation becomes complicated if >>> the criteria change. For example, if I later figure out a way to offer >>> version 5 to some older OS versions, how does the user who installed >>> cliclick4 learn of that? >>> >>> But such ports that mutate to fit the environment are more complex to >>> write and maintain. And we have to remember that we can only change the >>> version (and any other property that goes into the portindex) based on the >>> segmentation of our server-side portindexes. We maintain separate indexes >>> for each OS version/arch combination so we can change a port version based >>> on OS version or arch but not based on other criteria like Xcode version. >>> >>> Sometimes the single mutating port strategy doesn't work so well. The go >>> port is an example. Many go modules fail to build on older systems because >>> they require a newer go than the latest one that works on the older system. >>> But this wouldn't be solved by switching to multiple versioned go ports. >>> Each go module port would still need to know its minimum go version which >>> it currently doesn't. Or, depending on how common it is for modules to >>> require newer go versions, we could dispense with the idea of go supporting >>> old systems entirely. >>> >>> The separate versioned ports strategy also doesn't always work well >>> because it takes continual effort to keep them in sync. The postgresql >>> ports for example are all a little different from one another because at >>> various times a fix or a reformat only got applied to one of them but not >>> the others. >>> >>> >>>
