I'll answer your questions in a different thread to limit the noise in this one. Look for the "Xiden/Guix design comparison" subject line within the next 24h.
On 8/24/21 12:42 PM, Maxime Devos wrote: > Sage Gerard schreef op ma 23-08-2021 om 18:24 [+0000]: >> Thank you for the links! >> >>> I miss which problem Xiden is solving and how it does. >> You are not the first to say so, and I'm happy to learn more about why. >> >> I'll try to explain in a different way here, so forgive the text wall. >> I'll incorporate any feedback here to improve the white paper. I'd also like >> to hear more about Guix's philosophy on what I'm going to talk about, because >> I had reasons to end up with a different model. > Maybe the publications at <https://guix.gnu.org/en/publications/> are > informative. > >> -- >> >> If we each use a program to distribute software, then those programs are >> probably >> incompatible in some way (e.g. Cargo vs. Nexus' Vortex). Xiden addresses >> this by >> decoupling the subjective elements of software distribution from the >> objective ones. >> That means modeling software distribution as (partly) a semantics >> problem. Racket didn't >> do this with its own package managers. Which is ironic, because it's hard >> to guarantee >> that that an arbitrary Racket program will get the dependencies they mean >> to use. >> >> I've written about this at length [1][2][3], > About polyglot: packages in Guix can specify which version of a package they > use. > E.g., Guix has multiple versions of 'mozjs', and 'mozjs@78.10.1' uses a > specific > version of 'autoconf' (2.13) and 'rust' (1.41) > > (native-inputs > `(("autoconf" ,autoconf-2.13) > ("automake" ,automake) > ("llvm" ,llvm) ;for llvm-objdump > ("perl" ,perl) > ("pkg-config" ,pkg-config) > ("python" ,python-3) > ("rust" ,rust-1.41) > ("cargo" ,rust-1.41 "cargo"))) > > Is something missing from Guix here? About [3]: this doesn't seem a problem > in Guix, > if you're referring to the ability to define multiple versions of a package > at the same > time (it's an ‘official policy’ to usually try to only keep the latest > version of a > package though, but channels can ignore this completely). > >> and spoke about it last year [4]. Xiden has changed a bit since the speech >> to be more flexible, and it is no longer limited for use in Racket projects. >> Apologies if you see something that looks contradictory. >> What does that mean? Let's say Alice and Bob each write a Xiden launcher. >> >> Alice >> >> trusts SHA-384, but only if its implementation uses Keccak. > SHA-384 and Keccak are separate things? SHA-384: a version of SHA-2, > SHA-3: a version of Keccak. By definition, implementations of SHA-384 > have no reason to use Keccak. > >> defines "sha384" as the canonical name of the CHF, and treat variants like >> "SHA-384" as aliases. >> delegates trust in artifact signatures to GPG subprocesses. > Guix allows changing the hash algorithm used, search for ‘content-hash’ in > the guix manual. > Changing the hash algorithm globally unavoidably requires modifying every > package definition > though, but that could be automated. > >> downloads content from her employer's S3 buckets. > The ‘official’ substitute servers can be replaced: > "guix build --substitute-urls=http://s3.bucket.employer"; > (after authorising that substitute server). > >> unconditionally prefers cached installations, such that every defined >> artifact >> has exactly one stored copy on disk. > Coming from Guix, I don't know what this means. Does this mean only one > version > of a package can be in the store at the same time, and building a newer > version > deletes the older version? How does this compare with ‘content > deduplication’? > >> All dependents access the stored artifact >> using symbolic links or Windows junctions. > >> prefers declaring versions with edition and revision names. >> Bob trusts SHA-1, > See ‘content-hash’ above. Also, why should Xiden let Bob trust SHA-1 in the > first place? > (See > <https://en.wikipedia.org/wiki/SHA-1#SHAttered_%E2%80%93_first_public_collision>.) > >> but only for artifacts from services authenticated by his operating >> system's certificates > This is rather vague to me, unless you're referring to substitutes. > What does this mean for origin specifications? > > (source (origin > (method url-fetch) > (uri (string-append "mirror://gnu/hello/hello-" version > ".tar.gz")) > (sha256 ; imagine this was SHA-1 instead. > (base32 > "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i")))) > > Bob got the origin specification from Xiden (or something like a Guix channel > but for Xiden). How would this ‘artifact’ authenticated by ‘operating system > certificates’? I mean, when you let your operating system build that origin > (via guix (or Xiden?), which you could consider part of the OS), it will check > that hash, and it will be considered valid (‘authenticated?’) if the hash > matches. > There are no certificates at play here. > > I suppose the OS (guix or Xiden?) could then ‘sign’ it, but what value does > that > provide to Bob? (Unless Bob runs a substitute server on their computer.) > >> . >> trusts digest creation and signature verification as implemented by his >> host's dynamically--linked `libcrypto` library for use in the same process. >> downloads content from GitHub packages > AFAIK there is no such thing a ‘GitHub packages’. What is a ‘GitHub package’, > precisely? When I hear ‘X package’, I think of "X = python, haskell, ...’ > which > have a standardised build sytem (represented by python-build-system > and haskell-build-system in Guix), a standardised location to download them > from > (pypi.org and hackage.haskell.org, represented by the 'pypi' and 'hackage' > importers > an refreshers in Guix) and are separate languages. > > The only thing things on GitHub have in common are: > > * they have git repositories on GitHub > * some repositories on GitHub have ‘releases’, which can automatically be > discovered > > Notice dependency information and build system information are absent. > >> unconditionally prefers SxS installations, > I searched for ‘SxS’, and found this: <https://en.wikipedia.org/wiki/SxS>. > I don't think you meant that. What is ‘SxS’? > >> such that packages cannot conflict in a shared namespace > ‘packages cannot conflict in a shared namespace’ is a bit vague. > Are you referring to ‘profile collisions’? About having multiple versions > of the same package on the same system? Guix (and nix) can handle the latter. > When the ‘propagated-inputs’ are limited, installing multiple packages each > using > a different version of a package is possible. > > Is something missing from Guix here (please be very concrete)? > >> at the cost of defeating an internal cache. > I'm not sure what you mean here, can you illustrate this very concretely? > And is something missing in Guix here? > >> prefers Semantic Versions > If upstream doesn't do ‘semver’, and instead uses a versioning system like > used by TeX (3.14, 3.141, 3.1415, 3.14159, ...), then Xiden cannot somehow > let the package use ‘semver’ instead. Unless you want to teach Xiden to > convince upstream to change their versioning system? > >> These preferences are valid in Xiden's DSL for writing launchers. > What is a launcher (compared to packages in Guix), and why would I want one? > Why shouldn't I just install packages and run them ("guix package -i texmacs", > start texmacs from some application chooser menu of the desktop environment). > >> The invariants for each launcher can differ as much as Cargo and Vortex >> differ. >> What I wanted to do was allow users to declare their own invariants >> explicitly, >> but only when those invariants are wanted for subjective reasons. > What are these invariants you're speaking of? I don't think you're referring > to, say, translation invariance (‘the laws of physics remain the same under a > translation of the coordinate system’). > > Or do you mean something like ‘Invariant (computer science), an expression > whose > value doesn't change during program execution’ > (<https://en.wikipedia.org/wiki/Invariant>)? > But what are the ‘expressions’ in this case? > >> I can't just write an overly-opinionated tool in this space because >> opinions >> have shelf-lives, and they can't interrupt our ability to get the content >> we >> need on demand, with our own safety guarentees. > Is Guix too opinionated about something in particular? > >> Xiden launchers have advantages over shell scripts in that we can both still >> download >> and install dependencies from the same servers, and create reproducible >> builds in terms >> of the same (bit-identical) state on disk. We just differ on details that >> shouldn't impact >> how we work. It also helps us patch holes faster, since if (say) a catalog >> maintainer tells > What is a ‘catalog maintainer’ (maybe it's like someone with commit access to > a ‘guix channel’?) > Also, what ‘shell scripts’ are you referring to? I've been doing well > without writing any shell > scripts. > >> us that a CHF was compromised, we can immediately revoke trust in the CHF >> independently > Maybe guix could gain a command "guix style transition-hash OLD NEW" to > automagically > rewrite package definitions to use the new hash algorithm instead of the old. > It will > need to download plenty of source code however, and it will entail a > world-rebuild. > > Note that, in practice, signs of the brokenness of hashes appear well in > advance of > actual exploits. > >> in our own clients. > What are these ‘clients’ you're speaking of? I haven't been needing any > ‘clients’ lately. > >> The package definitions are all expressed in terms of what the launchers >> allow. > If 'package definition' = (package definition as in Guix), then this is false, > Guix doesn't have a notion of ‘launchers’ and doesn't depend on ‘Xiden’. > What are you meaning, exactly? Also, I haven't found any package definitions > in Xiden, > could you point me to any? > >> So what problem does this solve? I'm trying to preserve an element of >> walk-away power >> in any tech community. Maybe your community goes in a questionable >> direction. Maybe your >> desired software is in a different ecosystem. > Guix channels can function independently of the main ‘guix’, augmenting > ‘guix’ with > additional packages, even if they contravene official Guix policies (e.g. > non-free, > bad quality ...), or simply if putting it in ‘guix proper’ takes to much time. > There exist a few relatively well-known channels like that. > >> Maybe the maintainers are abusive jerks. > Technically, Guix has ‘maintainers’, but I wouldn't know where I could find > the list. > Guix (the package manager) doesn't have its own infrastructure. > It does have official substitute servers, but you can ignore them. > So I don't see ‘maintainers are abusive jerks’ as a plausible threat to Guix. > If they become existent, they can easily be replaced. > >> Maybe you have a `leftpad`/`event-stream` incident and the developer is >> unable/unwilling >> to patch the problem. > In Guix, if someone pushed malware (*) to, say, a package on pypi.org, then > we (Guix) > would simply stop using new versions of that package from pypi.org. If > there's a healthy > fork somewhere, we could switch over to that for new versions. > > (*) I could have confused the ‘`leftpad`/`event-stream` incident’ for another > incident. > >> You'd probably want a way to escape to a new community or distribution model >> without > ‘Escape to a new community’ --> only if the community as a whole are ‘abusive > jerks’. > If it's merely some maintainers, they can be removed or replaced. If it's > all the > maintainers, I'm sure a few well-regarded members of the community could come > together > for a ‘coup’. If it's the community as a whole, I don't see how Xiden could > help, > unless you mean I would move from Guix to Xiden? > > Why would I want to ‘escape to a new distribution model’? Writing something > like Guix > from scratch seems a lot of work to me, I would rather fork Guix (with a few > likewise-minded collaborators) than to start over again. Or move to Debian > maybe. > I don't see what ‘Xiden’ could offer here. > >> giving up content or doing complex integration work. If Xiden can do that, >> users >> don't have to depend on the same middlemen to share work. > Is ‘middlemen’ = guix committers here? Do you have concrete scenario to > illustrate this, > >> This is also better for >> developers because it doesn't oblige them to do everything their users >> want, > Are we talking about upstream developers here, or guix(/Xiden) committers? > How does this ‘obligation’ exist in the first place? > >> while knowing users will adapt to inevitable distribution problems. > I would rather not have to ‘adapt to problems’, I would rather that upstream > has something > that works (though I can understand if occasionally mistakes are made). > >> Everyone's consent >> becomes explicit, such that Xiden's "real" invariant is that consent >> between any >> two parties be mutually compatible. > Consent about what? > >> I hope to improve user freedom from this angle, >> and my approach is biased by my experience with Racket's limitations in >> this space. > Why are you comparing it with ‘racket’ instead of, say, Debian or Guix? > ‘racket’ is a Scheme implementation, not a package manager. > >> The subjective parts I talk about go further into details like what name >> canons, >> versioning schemes, trusted certificate chains, and even specific approaches >> to an exact >> diamond dependency. The objective parts include integrity checking, >> signature verification, >> safety limits, automatic dependency resolution, patches, protocols, and >> other concerns that >> a decent package manager normally cares about. By keeping these separate, >> all of the below >> features become cheap to write, without sacrificing trust & safety in the >> abstract. >> >> Download archived GitHub repositories (source on Racket Discord atm) > It can download GitHub repositories. Why are you writing ‘archived’ here? > Guix supports git repositories (see 'git-reference' in the guix manual), so > I don't see what additional value Xiden brings here. Can it also > automagically build > the software? Also, why ‘GitHub repositories’ specifically? Why not, say, > the git repositories at <https://git.ngyro.com/>? > >> Manage Racket installations like how `nvm` manages Node.js installations [5] > It would seem that your ‘racket-installation-manager’ [5] cannot resist a > compromise > of https://downloads.racket-lang.org/releases/. Also, I don't see how it > would be > possibe to use ‘racket-installation-manager’ offline. And how does it > compile the > racket packages? If we're talking about ‘trust’ and ‘escaping to a new > community’, > maybe provide an option to override 'host-url-prefix'? Currently, it's > hardcoding > the ‘official’ racket community. > >> Produce a self-hosted Xiden installations, with implicit trust for the >> running Xiden instance [6] >> Control which exact artifacts mnust be produced deterministically [7] > Can I reproduce the ‘artifact’ (= store item?) on a separate machine? > >> Force generated bindings to be `eq?` (This one haunts Racket's package >> managers today) [8] > That seems like a Scheme thing to me, not a package manager thing. > >> Source package definitions from Pastebin to download content, digests, >> and signatures from a public S3 bucket (This does not exist today. I'm >> spitballing because I know >> it would work) > How can this be secure? > >> I hypothesize that Xiden can "abstract over" package managers in the same >> way that Racket can >> abstract over languages. I think people are used to package managers >> deciding things for them. > Please provide some sane defaults. I would rather not have to decide > everything myself, > and be able to trust Guix(/Xiden?) to do something reasonable. > > Also, if you ‘abstract over package managers’, wouldn't you just end up with > one ‘super > package manager’? Why not ‘manage’ everything with a single package manager, > like e.g. guix? > >> So what is Xiden for? Well, it's not for deciding an SOP for you. > Searching for SOP, I find <https://en.wikipedia.org/wiki/Same-origin_policy>. > What were you referring to here? > >> It's for giving everyone the option to change the SOP for themselves with >> minimal politics. > What are these changes you're referring to? > >>> what you would like to bootstrap? >> All binaries related to creating a GNU/Linux distribution, such that I can >> reproduce an exact >> OS, Racket installation, and Xiden instance. I want a trusted initial state >> on GNU/Linux. >> >> To be clear, I don't need to do this for functional reasons. Xiden's already >> operational. I >> just can't claim that you can trust a given Xiden instance with the same >> confidence as a >> Guix instance right now. > If you want to get package definitions from guix, take a look at (guix > inferior). > Maybe you could port the relevant code of (guix inferior) to Racket, to let > Xiden > talk to Guix? In particular, 'inferior-eval' sends an S-exp to the inferior > Guix, > which is evaluated there, and the result is sent back. > > Greetings, > Maxime.
