Hi Maxime (it's been some time, welcome back!) Maxime Devos <maximede...@telenet.be> writes:
[...] > I think nar stuff should be kept outside SWH. It doesn't seem > scalable to me for SWH to support the format of every distribution. > Likewise, I think that SWH identifiers should _not_ become an > intrinsic identifier that is recorded in package definitions -- if > there are other archives that are somewhat SWH-like archives, then > Guix should support them too even if they don't use SWH identifiers > for whatever reason, and including the identifier of every single > archive seems unscalable to me. > > I believe I have a solution on how to solve the ‘everyone uses > different identifiers, how to map between them’ problem, but it will > take some paragraphs: > > At some point in the past, when thinking about downloading source code > over GNUnet File-sharing (FS), I had the problem that Guix and GNUnet > uses different intrinsic identifiers -- Guix uses the NAR hash for > querying substitute servers, whereas FS has a system of its own that's > more convenient for P2P file-sharing stuff. > > The problem then was to somehow map the NAR hash to the FS identifier. > I couldn't do this the Disarchive way, because the point was to be > _P2P_ and Disarchive ... isn't. > > A straightforward solution would be to just replace the https:// by > gnunet:// in the origin (like in https://issues.guix.gnu.org/44199, > except that patch doesn't support fallbacks to other URLs like > url-fetch does). > > The problem was that people demanded that gnunet:// should only be > supported once there is actually source code on GNUnet and GNUnet is > stable, but why would people put source code on GNUnet when no > distribution supports it and how would GNUnet become stable without > any users? > > To work-around these circular demands, I started 'rehash': > <https://lists.gnu.org/archive/html/guix-patches/2021-01/msg01067.html> > (current location: https://notabug.org/maximed/rehash). It is a > (P2P!) GNUnet service that maintains a 'SHA1512<->GNUnet FS URI' > mapping, or more generally, a 'this hash type<->that hash type' > mapping. > > (It is just a service on top of the DHT, so the same could easily be > done for BitTorrent or IPFS.) > > It's rather incomplete at the moment (there is no verification or > reputation mechanism at all so the network could be flooded with bogus > mappings, mappings are only in DHT, not stored on disk, so they are > lost on reboot, the POC Guix integretation is a bit limited), but the > basics are there -- the POC successfully downloaded a substitute over > GNUnet _without_ having to include FS URI in the narinfo (*)!. > > I'm writing about substitutes here, but the exact same approach could > be done for plain source code. > > (*) I might have misremembered; I can't find the POC on > issues.guix.gnu.org again, and I'm not sure if the POC used rehash or > if it just included the FS URI in the narinfo. > > (TBC, I haven't been working on Rehash lately, but rather > Scheme-GNUnet: a Scheme port of the GNUnet libraries that's less > limited than Guile-GNUnet. Idea is to make GNUnet-FS and rehash more > convenient to use from Scheme, and in particular, in Guix.) Thanks for sharing your efforts on the P2P in Guix/GNUnet front! P2P seems like it'd make substitutes mirroring easy and improve robustness as the network gets populated. It's very interesting; it'd definitely make an interesting summer internship :-). Keep up the good and inspiring hacks! -- Thanks, Maxim
