Katherine Cox-Buday <cox.katherin...@gmail.com> writes: > Hello again, François! I've redirected this thread to guix-devel, and > the bug since we've begun discussing implementation details. > > JOULAUD François <francois.joul...@radiofrance.com> writes: > >> First is to use vendored dependencies (when upstream provides them). This >> one has the merits of simplicity (as we just have to download the >> source and launch `go build` or whatever is needed) and less risks of >> bugs. The downsides are known: difficult to audit Licences, duplication >> of code source, difficulty to follow security bugs, etc. > > -1 to this approach (explanation below).
Seems like there could be two sub-scenarios here: the code could use go modules or not. >> Second is to re-vendor package from go.mod (coming directly from code >> source or synthetized from source) by creating a source derivation >> including all dependencies. This is the strategy followed by Nixpkgs >> as explained in [1]. (It seems to me this is the route followed in >> the patches to from Helio in [2] but I did not take the time to read >> them.) With this approach we still have some of the downsides of using >> vendored upstream but it is at least easier to verify the existence >> of upstream. > > I don't fully understand this approach, so I don't understand how this > is different to approach three? Does this mean we create a pseudo, > non-public package which states all the dependencies as inputs? If so, > why wouldn't we just go with option three? I think this approach is like saying you git clone the upstream repo, run go mod vendor or go mod download, then go build. Or whatever buildGoModule does in nix (https://github.com/NixOS/nixpkgs/issues/84826) I read through that issue and would personally vote for this approach of using `go` to restore the code. I think trying to reimplement go module restore process with Guix packages is bordering on Not Invented Here. There would be a never ending battle of trying to reimplement the go module restore process and the amount of source packages would really clutter things up. I think some of the issues with the distro wanting to change a package could be solved with a feature to patch go.mod before calling `go` to restore. >> Third is to package every dependencies. This is the most transparent way >> and the one that leads to less code duplication but the downside is the >> difficulty to scale with the number of packages (even if the objective of >> patch at [3] is to automate it) and the need to have only one reference >> version for each package in the distribution which have its own share of >> problems (one of them being that we don't use those tested by upstream >> as stated in [4]). > > I think this is the eternal conflict between distributions and > code-bases. As a software engineer, I am a fan of vendoring. It removes > entire classes of issues: > > - Dependency on remote servers to be up to fetch dependencies and > perform builds > - Requiring some kind of system to pin versions > - Needing to stay on top of releases even if that doesn't meet your > schedule or needs > > As a packager for a distribution, I dislike vendoring because of the > reasons you outlined above, _but_ I also dislike building upstream > software with versions of dependencies that weren't approved, tested, > and verified, upstream. It seems to me like that's a recipe for > unstable, maybe even insecure, software. > > Normally, distributions are forced to do this because their packaging > and build systems are only set up to support one version at a time. Guix > can theoretically support all versions, but doesn't want to take on the > dependency graph explosion this approach would cause. > > If we go on historical precedent, and how Guix handles the other > language ecosystems, we should only have one version of each dependency, > and build all software which relies on that dependency on the version we > have packaged. Guix has already begun taking on the difficult challenges > with this approach, including patching upstream to work with versions of > dependencies upstream was not built for. That sounds like a pretty difficult challenge. Seems like it could quickly become untenable if a commonly used library had some sort of breaking change between versions and different versions of it were used by different packages. I don't think anything is stopping Guix from having multiple versions, but hand-assigning them would feel pretty manual and error-prone. On the other hand, like you said, the dependency graph explosion from importing everything would be overwhelming. > I dislike it, but I also don't think we should try to solve the broader > class of issues while trying to implement an importer. It should be a > larger discussion within the Guix community across _all_ language > packages about how we might achieve upstream parity while still > maintaining our goals as a distribution, all while not crippling our > infrastructure and people :) > > That's my viewpoint, but I think we should all defer to some of the > longer-term maintainers who have helped guide Guix. You wrote up both sides pretty well, but I couldn't tell if you had a strong opinion on having go restore dependencies vs Guix packaging dependencies. You had a lot of strong points for vendoring, but you're writing an importer... > Thanks for writing up these options. +1
