Re: Move switch-symlinks to (guix build utils)

[Maxime Devos]

b...@bokr.com schreef op za 04-06-2022 om 01:55 [+0200]:
> I am not expert on kernel link internals, but if
> you need/prefer atomic change to a specific link,
> does my log [1] below suggest a way?
> [...]

Only the replacing of an old by a new symlink needs to be atomic, and
this is already the case.  The only non-atomicity is that if
interrupted, an extra symlink [...].new can be created, but except for
tidyness this is not really a problem AFAICT.

Speaking of atomicity, I wonder if 'fsync'ing the symlink or the
directory is required?  E.g., is it possible after a power interruption
for the contents (= target) of the new symlink to be truncated without
doing a 'fsync' before the rename?

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Teams

[Maxime Devos]

Ricardo Wurmus schreef op za 04-06-2022 om 14:07 [+0200]:
> As a first step I’d suggest collecting teams, setting up the email
> aliases, and updating the website to show the existing teams.  Here’s
> a draft of three teams:
> 
> [...]

You can add me to the Rust team and to a new Minetest team.
Maybe Vivien Kraus would be interested in joining the Minetest team.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Teams

[Maxime Devos]

Tobias Geerinckx-Rice schreef op za 04-06-2022 om 14:50 [+]:
> I think we should also have (natural) language 'teams' who can be pinged 
> when, e.g., a news item lands, through a single guix-translators@ meta-alias, 
> and who can co-ordinate before releases.
> 
> I'll take -nl.  Maxime?

Ok sure.


signature.asc
Description: This is a digitally signed message part

Re: Move switch-symlinks to (guix build utils)

[Maxime Devos]

Arun Isaac schreef op ma 06-06-2022 om 16:57 [+0530]:
> Could we implement symlink/remove-old without catch and throw? Something
> like:
> 
> --8<---cut here---start->8---
> (define (symlink/remove-old target link)
>   "Make a symbolic link named LINK pointing to TARGET. If LINK already
> exists, it will be removed first.  This is not an atomic operation."
>   (when (file-exists? link)
>     (delete-file link))
>   (symlink target link))
> --8<---cut here---end--->8---

file-exists? does not check if the file exists -- for a broken symlink,
it will return #false instead of #true.

Also, this introduces additional non-atomicity (though in this
particular case, this does not seem to be a problem).  It would also
make things a tiny bit slower, because file-exists? makes an additional
syscall.

For these three reasons, I would go for the more robust catch+throw.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Repology and outdated packages

[Maxime Devos]

kias...@disroot.org schreef op di 07-06-2022 om 18:39 [+]:
> - we package their dependencies separately (eg Rust crates, Go modules), 
> these are a significant portion of Guix that cannot be constantly updated 
> easily
> - the Guix 1.4 release is coming soon so more time may be spent debugging 
> that instead of updating the package archives
[...]
> However I hope that faster package reviews can mean we stay updated faster 
> without compromising package quality.

FWIW, antioxidant removes most of the complexity of Rust's dependency
system, which should help with packaging and reviewing.  No need to
package old versions to satisfy Cargo and making sure the old version
builds, has an appropriate synopsis/description, doesn't introduce
malware, ... Theoretically (currently untested!) faster build times, no
need to package Windows or redox crates.

The cost is that, as you seem to write, that the dependent might not
support the old version of the dependency, but updating is something
upstream needs to do eventually anyway, and (antioxidant) in theory
(untested) supports working with multiple versions.  There's also the
option of making some compatibility changes in the Guix version of the
dependency (undoing a removal of a function, or adding some aliases,
etc.).

Greetings,
Maxime


signature.asc
Description: This is a digitally signed message part

Re: Repology and outdated packages

[Maxime Devos]

kias...@disroot.org schreef op di 07-06-2022 om 18:39 [+]:
> So without demanding more maintainer time, for now I just convince myself 
> that:
> - key toolchains such as Rust and Go are not always up to date, thus blocking 
> the
>  upgrade of several packages

In my experience with antioxidant, not-up-to-date Rust toolchains do
not prevent upgrading packages -- in the process of resolving build
failures, I updated some crates.  This never resulted in a ‘you need an
new toolchain error’.  At worst, I had to disable the "unstable",
"nightly" or "generic-simd" features of the crate.

There are a lot of outdated crates in Guix, but AFAICT this is
unrelated to the newness of the toolchain.

I don't know about Go.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Repology and outdated packages

[Maxime Devos]

Maxime Devos schreef op di 07-06-2022 om 23:32 [+0200]:
> In my experience with antioxidant, not-up-to-date Rust toolchains do
> not prevent upgrading packages -- in the process of resolving build
> failures, I updated some crates.  This never resulted in a ‘you need
> an
> new toolchain error’.  At worst, I had to disable the "unstable",
> "nightly" or "generic-simd" features of the crate.

Correction (adding the rust-git-path crate to update another crate to
avoid non-building crate due to updated crates with a different API):

Saving crate informtion in 
/gnu/store/fr292pqsfas2i9nwc3bijx5skb8g3ayg-rust-git-path-0.1.3/lib/guixcrate/git-path.crate-info
error[E0658]: use of unstable library feature 'is_symlink'
  --> src/realpath.rs:50:34
   |
50 | if real_path.is_symlink() {
   |  ^^
   |
   = note: see issue #85748 <https://github.com/rust-lang/rust/issues/85748> 
for more information


Looks like it is possible to work-around avoid though:

https://github.com/rust-lang/cargo/commit/c6745a3d7fcea3a949c3e13e682b8ddcbd213add

Alternatively, I could set RUSTC_BOOTSTRAP=1.  From
<https://rustc-dev-guide.rust-lang.org/building/bootstrapping.html>

The build system sets RUSTC_BOOTSTRAP=1. This special variable means
to break the stability guarantees of rust: Allow using #![feature(...)]
with a compiler that's not nightly. This should never be used except
when bootstrapping the compiler.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: On commit access, patch review, and remaining healthy

[Maxime Devos]

Giovanni Biscuolo schreef op vr 10-06-2022 om 14:27 [+0200]:
> > - Our synopses and descriptions are not casually copy-pasted from
> the
> >    project website. We try to rewrite and improve on them if
> necessary.
> 
> AFAIK similar requirements are "enforced" by all other distributions

They aren't by the special-purpose crates.io and pypi.org distributions
AFACIT, though maybe including those distributions here is a bit too
nitpicky?  This rule has also been neglected for the Rust crates in
Guix.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: U.S. Midwest based build farm

[Maxime Devos]

jbra...@dismail.de schreef op za 11-06-2022 om 16:06 [+]:
> What's good and/or bad about this idea?

A positive point: extra resources, could be useful for reproducibility
testing, ...?

A negative point: extra points through with malware can be introduced
(->compromises).  Can be solved by reproducible builds and variation of
"guix challenge". Unfortunately, "guix challenge" is inherently racy.
"guix substitute" currently only checks that the narinfo has a _single_
authorised signature, maybe it can be adjusted to allow the user to
ask: ‘only consider a substitute to be authorised if the same hash is
signed by N different authorised keys’?

Other points: ...?

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: On commit access, patch review, and remaining healthy

[Maxime Devos]

Giovanni Biscuolo schreef op zo 12-06-2022 om 11:42 [+0200]:
> > or have packages with bundled dependencies (e.g. vendored jars).
> 
> bundling binaries it's (is it?) for sure against the definition of a
> reproducible build, but what about bundling (source) dependencies?
> 
> AFAIU not to bundle (source) dependencies is an additional Guix
> requirement (and it is a Good Thing™): do I miss something?

FWIW, sometimes the bundled ‘source’ dependencies contain bundled
binaries of their own.  So while AFAICT not strictly necessary for
reproducible builds, unbundling ‘source dependencies’ makes ensuring
reproducibility(*) much more convenient.

(*) i.e., the non-trivial kind of reproducibility, where things are
actually built from source instead of copying binaries.

> honestly I did not study all the reproducible-builds.org
> documentation,
> but it's impossible to me to understand how a packaged upstream jar
> can be considered reproducible (and bootstrappable); maybe distros
> like NixOS are still slowly transitioning to a full reproducible
> build workflow?

It's ‘reproducible’ in the trivial sense that you can ‘reproduce’ a
scientific paper by putting it a photocopier.  That way, you can
reproduce the results, but you cannot confirm whether these results
were correct.

Greetings,
Maxime.



signature.asc
Description: This is a digitally signed message part

Re: On commit access, patch review, and remaining healthy

[Maxime Devos]

Giovanni Biscuolo schreef op ma 13-06-2022 om 11:34 [+0200]:
> Maxime I have a question for you please: do you really think that in
> the NixOS community

Going by the Java example, yes, at least for some of the NixOS
community.  I've also seen this interpretation of reproducibility in
Clojure (there was some question on building things from source and
reproducibility, and the response was something along the line ‘using
upstream binaries is 100% reproducible’).

>  (or any other project mentioned in
> https://reproducible-builds.org/who/projects/) the term reproducible
> is
> interpreted in that way?

I don't know about all of them, but for Guix and Debian: no.

> IMVHO if we continue using the term reproducible in that trivial way
> [3]
> when talking about software (this include each and every scientific
> paper [4]), we will never get to any point; reproducible is what
> reproducible means: https://reproducible-builds.org/docs/definition/

Exactly, trivial interpretations aren't really the point, because
trivial.

> [...]
> I was just hoping that nowadays "reproducible" is perceived as "build
> reproducible" (as defined in the definition above) by all software
> developers and many users (including scientists)

I'd hope so, yes.

> P.S.: or you Maxime are just playng the devil's advocate? :-D

No, I'm not advocating that trivial reproducibility is useful or the
goal or such.  My response was some speculation on an answer to the
following question:

> but it's impossible to me to understand how a packaged upstream jar
> can be considered reproducible (and bootstrappable);


signature.asc
Description: This is a digitally signed message part

Re: how to write services (was: Re: Teams)

[Maxime Devos]

Blake Shaw schreef op wo 15-06-2022 om 17:01 [+]:
> Thats very good advice and will be a useful guide in refactoring the
> parts of the system services documentation. I think in general, we
> need to find a nice middle ground between the extremely general and
> the immediately sensible, as I remember when I first got into guix
> 1.5 years ago, arriving at services left me very confused. 

I don't doubt your confusal, though personally I'm confused on the
confusal and I think I would have been confused by ‘file AND file-like
object’.  Even more so since we both come from a mathematical
background, where AFAICT this kind of terminology and Guix'
interpretation is standard.

> mathematics I'm a fellow appreciator of the power of generality (the
> extreme genericity of scheme and guix is why I'm here!), I also think
> if it doesn't obey strict linguistic rules it can antithetical to its
> original purpose.

I don't see what linguistic rule the term ‘file-like object’ does not
follow.  

> For example, I remember being very confused about
> "file-like objects", for the simple reason that it wasn't "a file or
> file-like object". While this might come from a GNU terminological
> lineage i'm unaware of,

AFAIK no relation to GNU.

>  my immediate reaction to trying to understand
> file-likeness is the simple rule that a semblance is strictly not
> what it resembles, and likeness qualifies semblance. It would be
> improper to place phones in a category of "phone-like objects",
> because the likeness assumes a distinction from the thing itself.

An object being ‘X-like’ merely means that it is like an X.  This does
not imply it _isn't_ an X, only suggests that in some cases it might be
a non-X.

More concretely, to me phones resemble phones and are objects, so
phones are phone-like objects.

Summarised, to me semblance/similar/likeness is reflexive, I don't see
where the non-reflexivity would come from?

Something I dislike about the ‘file AND file-like objects’ construction
is that it suggests that files and file-like objects are separate and
are handled separately, whereas files (as in, 'local-file' or
'computed-file') are just another case of file-like objects to Guix
(next to 'file-append', 'package', 'git-checkout', ...).  Furthermore,
usually file-like objects aren't files but more often they are
packages.

For a comparison, suppose we have a hierarchy of concepts, e.g.

  {0}⊊ℕ⊊ℤ⊊ℚ⊊ℝ⊊ℂ

Whole numbers can (informally speaking) be considered to be natural-
like numbers. Yet, that doesn't make natural numbers non-whole. 
Compare:

File-like objects are objects that are like a file.  Yet, that doesn't
make files non-file-like.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: how to write services (was: Re: Teams)

[Maxime Devos]

Blake Shaw schreef op wo 15-06-2022 om 21:40 [+]:
> > Something I dislike about the ‘file AND file-like objects’
> > construction
> > is that it suggests that files and file-like objects are separate
> > and
> > are handled separately, whereas files (as in, 'local-file' or
> > 'computed-file') are just another case of file-like objects to Guix
> > (next to 'file-append', 'package', 'git-checkout', ...). 
> > Furthermore,
> > usually file-like objects aren't files but more often they are
> > packages.
> 
> Going off the packages example, are they not often handled 
> differently? While I may want to (load "gnu/packages/base.scm"),
> a file,

Files can be loaded by Guile, yes.

>  a that I can work with ,

That's not a file, it's a package record.  You ca

>  as a record I 
> will get an error if I've included (gnu packages base) in my module
> and then try to invoke (load coreutils), although its defined.
> 
> But I think i'm missing your point here.



signature.asc
Description: This is a digitally signed message part

Re: how to write services (was: Re: Teams)

[Maxime Devos]

Blake Shaw schreef op wo 15-06-2022 om 21:40 [+]:
> > AFAIK no relation to GNU.
> 
> I thought recalled hearing it used in relation to GNU/Linux. A quick
> search
> brings up this stackexchange discussion[1], which quotes the book
> "Linux
> Philosophy" with the following:
> 
> #+begin_example
> Whenever possible, Linux makes its components available via files or
> objects
> that look like files. Processes, devices, and network sockets are all
> represented 
> by file-like objects, and can often be worked with using the same
> utilities used
>  for regular files.
> #+end_example
> 
> So the contents of /proc are file-like objects, but AFAIK they
> strictly aren't files
> per-se, but representations of processes that take the shape of a
> file at a 
> particular instance in time.
> 
> [1]
> https://unix.stackexchange.com/questions/416778/what-is-file-like-objects-in-linux

From Guix and Guile's perspective, things in /proc are just (OS) files
that happen to be dynamically generated.  Devices are weird files that
need a special I/O API, but still files).  Some information on
processes is available in files, but the process itself isn't a file.

I would say that sockets (except for unix domain sockets) aren't files
and are rather unlike files (you cannot copy, hardlink, mv, symlink or
stat them, they don't have file names, they don't have an
owner/group/...) -- the only similarity seems to be the basis on file
descriptors and the possibility of read/write.

However, you cannot save sockets in the store, so from Guix perspective
even Unix domain sockets aren't file-like.

Though good point about the potential confusing with Linux' notion of
file-like objects!

Greetings
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: how to write services (was: Re: Teams)

[Maxime Devos]

Blake Shaw schreef op wo 15-06-2022 om 21:40 [+]:
> On the contrary, lets say I'm writing an intro book on CT. If I'm
> demonstrating something trivial, say the initial object, I'm not
> going to refer to it as "an initial-like object" for the sake of
> generality.

Neither does Guix?  If you're in a context where only the basic object
(in this case, your demonstration the initial object) is used, just
talk about the basic object.  But in a later section where you
generalize things to ‘initial-like objects’ (whatever that would be in
CT, I don't know any CT), you talk about ‘initial-like objects’, not
‘initial object and initial-like objects’.

For an example from another domain, consider groups in algebra.
In group theory, we have e.g. the fundamental theorem on homomorphisms.
Wikipedia formulates this as:

Given two groups G and H and a group homomorphism f : G → H, let K be a
normal subgroup in G and φ the natural surjective homomorphism G → G/K
(where G/K is the quotient group of G by K). If K is a subset of ker(f)
then there exists a unique homomorphism h: G/K → H such that f = h∘φ.

An equivalent statement could be made by replacing ‘given a group’ by
‘given an Abelian group or a group’:

Given two Abelian groups or groups G and H and a group homomorphism f :
G → H, let K be an Abelian normal subgroup or normal subgroup in G and
φ the natural surjective homomorphism G → G/K (where G/K is the
quotient group of G by K). If K is a subset of ker(f) then there exists
a unique homomorphism h: G/K → H such that f = h∘φ.’

But why do such a pointless thing, wouldn't just talking about groups
instead of ‘Abelian groups or groups’ be much simpler?

TBC: here ‘file-like object’ ≃ ‘group’ and ‘file’ = ‘Abelian group’.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: how to write services (was: Re: Teams)

[Maxime Devos]

Blake Shaw schreef op do 16-06-2022 om 06:20 [+0700]:
> But, perhaps it's just getting late and the matters are now & the
> details are slipping my mind, but im starting to realize im unsure of
> many examples of file-like objects that aren't a file? The email
> where you responded re: packages was cut short, but it seemed to be
> that you were saying that record-types *aren't* file-like,

I wrote that a package record # is not a
file (as it lacks a lot of operations and properties that would be
expected of a package, such as 'stat', 'read', 'write', and a file
name’.

Likewise, the  record type (!= an instance of the record type
) is not a file, but that's going rather meta.

However, packages (not the package _type_, but packages) are definitely
file-like, because when used in a G-exp with #$ or #+, Guix is able to
automatically ‘lower’ it in the store (resulting in a /gnu/store/.../
file name).

>  when I had thought they are; I thought anything with simple means of
> serialization could be considered file-like,
> [...]

It depends on the serialisation.  Not any serialisable object can do,
it must be an object that _Guix_ considers to be serialisable -- in
Guix terminology, this is called a ‘lowerable’ (low-level terminology)
or ‘file-like’ (high-level terminology, equivalent to ‘lowerable’
AFAICT) object.

> Would anyone care to share an explanation of what is/is not a
> file-like object in Guix? 
> Are fluids not considered file-like?

They aren't, as they do not implement lowering. (Technically: they
don't have a ‘define-gexp-ompiler’).

> I had thought that would be a use case where this geneticity becomes
> important.

Why would one put a fluid in a G-exp?  I suppose we could define what
lowering is for fluids (probably: get the value of what's inside and
lower that value), implement it in the Guix code and document it, and
hence consider fluids to be file-like.

I suppose that's all technically possible, though shouldn't it then be
extended to SRFI-111 boxes, parameter objects, variable objects,
promises and thunks as well?  Where would we stop?  And is this
behaviour actually useful?


> I remember when I first encountered gexps I thought, as FLOs didn't
> seem to be files, they were either records or fluids.

There is only a single mention of fluids in the manual (concerning
%guile-for-build).  The related concept of parameters is never used in
(guix)G-expressions, (except for 'with-parameters’).  So I fail to see
where this could have come from.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: FSDG-compatibility of APSL-2.0

[Maxime Devos]

TBC, did you see my previous mail about cherry-picking and power
assymetry.


> FWIW, I think that adopting a different (more stringent) license
> policy hits two issues:
> 
>  1. Where do you draw the line?  Based on which concrete principles
> to decide for this or for that?

I would like to refer to some blog article about something along the
lines ’free software is not about licenses, but about ???’, but I
cannot find it anymore.

zimoun schreef op vr 17-06-2022 om 11:06 [+0200]:
>  I think it is not affordable to adopt a different license policy than
>  the one listed by the GNU project [2].  It is a pragmatical line
>  because the Guix project does not have the manpower nor the structure
>  to do differently.
>
> And I miss what aim it would serve.

That page mentions:

> We classify> license according to certain criteria:
> [...]
> * Whether it causes any particular practical problems.
> [...]

Being forced to go to the US as a defendant seems like a very practical
problem to me.  E.g., apparently being imprisoned for 5 years or being
fined for $250 000 or such is a thing in the US[0], and the prison
situation in the US is reportedly bad.

The clause is also rather extra-territorial: what if $local_country
reforms copyright to make all sofware free, if we accepted ‘go to this
jurisdiction clauses’, then opponents could effectively block the
legally-enforced freeing of software by adding such a clause.   Such a
clause could potentially also be used to enforce a particular
jurisdiction that has forbidden modifying otherwise free software
entirely or somehting.

[0]
https://www.justice.gov/archives/jm/criminal-resource-manual-1852-copyright-infringement-penalties-17-usc-506a-and-18-usc-2319

>  2. The GNU project is already strict on what is accepted; for good
>  reasons.  The Guix project is a niche and being more stringent would
>  lead to be an even more niche.

Not being subject to the US seems worth some extra niche-ness to this
non-US person.  Also:

>  I think it is not affordable to adopt a different license policy
>  than the one listed by the GNU project [2].  It is a pragmatical
>  line because the Guix project does not have the manpower nor the
>  structure to do differently.

... currently Guix isn't using the APSL2.0 anywhere (according to git
grep -F aspl), so it seems quite practical and effortless to just
remove apsl2 from (guix licenses).

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: FSDG-compatibility of APSL-2.0

[Maxime Devos]

zimoun schreef op vr 17-06-2022 om 11:06 [+0200]:
> [...] How do we resolve the disagreements?

By talking on guix-devel.


signature.asc
Description: This is a digitally signed message part

Re: FSDG-compatibility of APSL-2.0

[Maxime Devos]

Liliana Marie Prikler schreef op vr 17-06-2022 om 12:00 [+0200]:
> Am Freitag, dem 17.06.2022 um 11:39 +0200 schrieb Maxime Devos:
> > The clause is also rather extra-territorial: what if $local_country
> > reforms copyright to make all sofware free, if we accepted ‘go to
> > this jurisdiction clauses’, then opponents could effectively block
> > the legally-enforced freeing of software by adding such a clause.
> No, they can't.  If $local_country makes all software free, such a
> clause would likely be illegal in $local_country and thus unenforcible.
> If Apple did try to sue a $local_country citizen, $local_country could
> sue Apple for breaking $local_country law.

It's unenforcable in $local_country, yes, but the due to the ‘Dispute
Resolution clause’, the sueing happens in California, not
$local_country, which is the potential issue I wanted to highlight. 
Though possibly ...

> While this clause exists, it might be unenforcible in practice. 
> Suppose you did violate the APSL, for instance, by linking to GPL
> code.
>
> If you are already in California, [...], but if you don't and simply
> decide to not heed their call, Apple needs to appeal to international
> law enforcement or your country in particular, whichever is easier. 
> I doubt that this will make any difference in the US [...], but if
> they have to cross either ocean, things become more difficult.

... yes, but I wouldn't know whether it will be to difficult or not for
Apple and whether $local_country would agree/disagree, so I'm inclined
to assume the worst for safety, unless informed by a reliable expert.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: FSDG-compatibility of APSL-2.0

[Maxime Devos]

Doesn't seem to reach some kind of consensus, or maybe there's actually
some consensus for considering APSL-2.0 acceptable (albeit suboptimal)
for Guix but I'm to biased to see it :p, so I suppose continue with
status quo (i.e.: allow APSL-2.0)?

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: FSDG-compatibility of APSL-2.0

[Maxime Devos]

(zimoun pointed out that I didn't actually send this mail, apparently
it never left ‘drafts’.  Anyway, just sending this e-mail for
completeness; unless someone comes with a new insight or something the
discussion appears to be done for now.)

Philip McGrath schreef op do 16-06-2022 om 02:21 [-0400]:
> Still, I'm in favor of the status quo. I think fragmentation over 
> license policies has a significant cost for the community, and this
> does not seem to be sufficiently problematic to be worth a schism.

Maybe, but I'm not aware of any method to revise the decisions of the
FSF.

Philip McGrath schreef op do 16-06-2022 om 02:21 [-0400]:
> I'm not a lawyer, so take this paragraph lease seriously, but I also 
> think the concrete impact is less than it might first seem. We accept 
> choice-of-forum provisions like the one in MPL-2.0 ("Any litigation 
> relating to this License may be brought only in the courts of a 
> jurisdiction where the defendant maintains its principal place of 
> business and such litigation shall be governed by laws of that 
> jurisdiction, without reference to its conflict-of-law provisions.") [8] 
> which would require you to sue Apple in California

I consider this to be a much milder clause than the clause in APSL-2.0:
also IANAL, but what it looks like to me:

1. APSL-2.0: Apple can legally drag you (*) to California to be sue you
   there under California's law and everything that entails.
2. APSL-2.0: Likewise, you can drag Apple to the California to sue
   Apple there.

I don't see any reason to do (2) here.
What I consider problematic here, is (1).

Contrast this to MPL-2.0 (for simplicity, this assumes Apple uses the
MPL, feel free to replace by Mozilla or whatever):

1. If Apple sues you, they have to sue you in _your_ country.
2. If you sue Apple, you have to sue in Apple's country.

This seems rather symmetric to me, and while sometimes I might disagree
with $foreign_country's or $local_country's laws, this seems a rather
reasonable system to me.

(*) unless $your_country's legal system disagrees on this choice of
forum provision.

> We also accept licenses like the GPL that don't have any choice-of-
> forum provisions:
>
> the law of "personal jurisdiction" and venue is complex, but I would
> not be shocked if Apple could sue you in California in this case. My
> impression is that it would be very difficult to require something
> like a "freedom not to litigate in California" (especially so for all
> possible values of "California") without rejecting many 
> currently-accepted licenses.

My problem is not a ‘freedom to not litigate in $foo’, but rather ‘no
cherry-picking jurisdictions to whatever is convenient for limiting the
freedom the most’.

Sure, if it comes to a conflict between party X and Y, the legal system
will need to somehow decide on a forum, but no need for this power
asymmetry.

In this case, MPL-2.0's clause seems acceptable to me, but APSL-2.0's
doesn't.

TBC, if two parties of about equal power choose a forum to avoid
potential future problems, ok, but this doesn't seem to be the case for
the APSL-2.0.


signature.asc
Description: This is a digitally signed message part

Re: PARI/GP and parallelism

[Maxime Devos]

zimoun schreef op vr 17-06-2022 om 16:36 [+0200]:
> Well, I do not know if the parallel computation of PARI/GP are
> reproducible.  But the building of PARI/GP is not; because some
> documentation…
> 
> --8<---cut here---start->8---
> $ diff -r --no-dereference 
> /gnu/store/fw2mckl5bn5mb6z79a1fyficvsclzdv3-pari-gp-2.13.4{,-check}
> Binary files 
> /gnu/store/fw2mckl5bn5mb6z79a1fyficvsclzdv3-pari-gp-2.13.4/share/pari/doc/develop.dvi
>  and 
> /gnu/store/fw2mckl5bn5mb6z79a1fyficvsclzdv3-pari-gp-2.13.4-check/share/pari/doc/develop.dvi
>  differ

Maybe solved by
!


signature.asc
Description: This is a digitally signed message part

Re: FSDG-compatibility of APSL-2.0

[Maxime Devos]

Felix Lechner schreef op vr 17-06-2022 om 13:11 [-0700]:
> > If you are already in California, [...], but if you don't and
> > simply
> > Apple needs to appeal to international
> > law enforcement or your country in particular, whichever is easier.
> 
> I generally think of license violations as civil matters. With some
> small exceptions, law enforcement focuses on criminal cases.

I thought that copyright violations were a criminal matter in the US,
but after some cursory searching, it appears to be more complicated. 
Apparently there's both a civil part and a criminal part, where
apparently the criminal part is for some cases of willful copyright
infringement.

> As a resident of Northern California, I personally like the local
> legal environment. You would have to talk to a lawyer to see if the
> Ninth Circuit (copyright claims are Federal) would honor the clause
> about ignoring the UN Convention after considering the relative
> strength of the parties, as a matter of sound public policy.

I don't know California but sounds nice.  Also I guess I'd need to do
the same to see if $local_country would do such a thing too.

> Perhaps the weighing of those factors played a role in why Apple's
> attempt to stipulate a venue did not render the license unfree.

Maybe, yes, I dunno.

Greetings,
Maxime.



signature.asc
Description: This is a digitally signed message part

Re: how to write services

[Maxime Devos]

indieterminacy schreef op za 18-06-2022 om 13:53 [+0200]:
> Additionally, based upon a decent demonstration on LMDB, I realised that 
> my annotation system makes it more feasible to adapt documents into LDIF 
> database-like-files (is that the correct terminology Maxime?) - 
> potentially turning each document into an LDAP ready database.

If your asking me, I don't know.  What's LDAP doing here?  Isn't LDAP
about authenticating users, which doesn't seem relevant to the
documentation effort?  If this is about databases: is the exact
database format relevant, or would any do -- e.g., Guix uses SQLite in
some places, will SQLite do?

And the text above seems about databases and RDF, but there appear to
be missing some things:

  * what's the RDF and database for?  As I understand it, it's for
something about documentation and terminology, but currently it's
super vague.

  * what stuff goes in the RDF and database, and what documents are you
speaking of?  The Guix manual?  All the package definitions, to use
them as examples?  The mails in the ML?  Manually written things?
Likewise, how is this database written or generated?

  * How will this RDF be used?  I mean, RDF can be flexible (see e.g.
Wikidata), but someone has to actually write some applications that
make use of the information, otherwise the fancy RDF is useless.

  * How is the RDF an improvement on the TeXinfo documentation?
I guess I'm missing something important here, but I prefer reading
TeXinfo documentation over RDF.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Teams: first draft list

[Maxime Devos]

Josselin Poiret schreef op wo 22-06-2022 om 14:30 [+0200]:
> Hello,
> 
> zimoun  writes:
> > Then maybe, we could hook Mumi and add regexps based on commit messages
> > for notifying a specific team.  Well, it is a rough approximation with
> > many false-positive but hey the aim is to deal with patches so these
> > false-positive do not matter, IMHO.
> 
> Maybe we could attribute files to teams?  It seems like the simplest and
> more robust way, since it easily grants 99% coverage (excluding new
> files, that is), and the structure of the Guix files seem well-amenable
> to such classification.

FWIW, I'm in the Rust team for the build system, not the individual
rust packages.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Time namespace for build sandbox (was Re: Set FORCE_SOURCE_DATE=1 by default)

[Maxime Devos]

Zhu Zihao schreef op wo 22-06-2022 om 23:16 [+0800]:
> 
> Can we make some experiments on the Linux time namespace for build
> sandbox?
> 
> This can mock the real system with our desired value, maybe a good
> solution for the reproduciblity on Linux machine.
> 
> Ref: https://man7.org/linux/man-pages/man7/time_namespaces.7.html

There are some comments on time namespace at
.  Unfortunately, it only accepts
offsets in a limited range, so you can't just reset the clock to 1970.

There was also some talk on avoiding the no-CLOCK_REALTIME problem at
 with syscall rewriting.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Experimental nar-herder support for serving fixed output files by hash

[Maxime Devos]

Christopher Baines schreef op vr 24-06-2022 om 09:10 [+0100]:
> [...]
> In terms of next steps, there's some things to do with improving the
> implementation, but it would be good to hear if this is actually
> worthwile?

I wouldn't know about the Guix part, but supporting streaming reading
responses in Guile(-Fibers) sounds useful outside Guix as well.  IIRC,
at some point in the past, I tried doing some streaming of reading
responses (or was it writing responses?) and it didn't work out ...

I'm not sure what ‘this’ refers to here: supporting fixed-output
derivations in general, or improving the implementation?  I don't know
the answer on the latter (except for a generic less memory/latency =
good answer), but I'd like to say that ci.guix.gnu.org's support for
fixed-output derivation makes it effectively acts like a mirror for all
source code used in Guix, so I'd like to keep that behaviour (*).
(There's always SWH, but avoiding a single points of failure would be
nice ...)

(*) I don't know if a lack of support for fixed-output derivations in
nar-herder would affect this ...

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: reader macros for hidden packages

[Maxime Devos]

jgart schreef op za 25-06-2022 om 22:27 [-0500]:
> Out of curiosity, is it possible to make reader macros like this with guile?
> 
> ```
> @hidden
> (define-public python-httplib2
>   (package
>     (name "python-httplib2")
> [...]
> The above would make the package hidden by using `hidden-package`.

FWIW, read-hash-extend things aren't macros, though that can be worked
around with some cleverness, see (guix gexp) where #~(...) -> (gexp
...) by the read syntax, and 'gexp' is a macro.

Also, @hidden does not start with #, so you can't extend the read
syntax that wat with read-hash-extend. (I suppose  you could go #hidden
instead).

However, I cannot recommend this, for read syntax is global (not per
module, like the module system) and there can only be a single read
syntax per first character (so high chance for collisions).

It's also rather magical, and we already have the less magical but no
less concise 'hidden-package', why not:

(define-public python-httplib2
  (hidden-package
(name "...")
...))

Or a new 'define-public-hidden':

(define-public-hidden python-httplib2
  (package
(name "...")
...))

or move the relevant packages upstream?  Also, the module system has a
#:replace, which may be useful to put in the (guixrus packages ...)
modules such that in case a module imports both the guixrus and guix
module, the guixrus wins (not 100% sure if that would works).

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Rust reprodubility -- .rmeta and shadow-rs

[Maxime Devos]

Hi,

There was some mail about irreproducibility in Rust, but I couldn't
find it anymore.  Anyway, I found a potential cause: rust-shadow-rs
embeds timestamps (even though it nominally respects
SOURCE_DATE_EPOCH???) and the ordering of definitions it generates is
based on a hash map (and hence, irreproducible).

The crate id is based on a hash over the source code, so this
irreproducibility can cause build failures if substitutes are used.

By removing the time stamp and sorting the definitions, 'nushell'
successfully built on ci.guix.gnu.org whereas it previously failed to
build on ci.guix.gnu.org but built successfully locally (with
antioxidant), and IIRC the (antioxidated) 'rust-nu-command' is now
reproducible:

Anyway, here the patch I used:

("rust-shadow-rs"
 ,#~((add-after 'unpack 'fixup-source-date-epoch
   (lambda _
 ;; TODO: it nominally supports SOURCE_DATE_EPOCH, yet something 
things go wrong,
 ;; as the shadow.rs still contains the unnormalised time stamp ...
 ;; For now, do a work-around.
 (substitute* '("src/lib.rs" "src/env.rs")
   (("BuildTime::Local\\(Local::now\\(\\)\\)\\.human_format\\(\\)")
(object->string "[timestamp expunged for reproducibility]"))
   (("time\\.human_format\\(\\)")
"\"[timestamp expunged for reproducibility]\".to_string()")
   (("time\\.to_rfc3339_opts\\(SecondsFormat::Secs, true)")
"\"[timestamp expunged for reproducibility]\".to_string()")
   (("time\\.to_rfc2822\\(\\)")
"\"[timestamp expunged for reproducibility]\".to_string()"
 (add-after 'unpack 'more-reproducibility ;; by default, it uses a 
hashmap, leading to an irreproducible ordering in shadow.rs and hence an 
irreproducible .rmeta (TODO: upstream?)
   (lambda _
 (substitute* "src/lib.rs" ; sort
   (("\\(k, v\\) in self\\.map\\.clone\\(\\)")
"(k, v) in 
std::collections::BTreeMap::from_iter(self.map.clone().iter())")
   (("self\\.write_const\\(k, v\\)") "self.write_const(k, 
v.clone())")
   (("self\\.map\\.keys\\(\\)") 
"std::collections::BTreeSet::from_iter(self.map.keys())"))

Maybe that was the cause?

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: reader macros for hidden packages

[Maxime Devos]

jgart schreef op zo 26-06-2022 om 09:43 [-0500]:
> > Or a new 'define-public-hidden':
> > 
> > (define-public-hidden python-httplib2
> >    (package
> >  (name "...")
> >  ...))
> 
> I like this idea. You'd implement that as a macro that inserts
> hidden-package for the user of define-public-hidden?

Yep.


signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

Ludovic Courtès schreef op ma 27-06-2022 om 12:10 [+0200]:
> Regarding the review process, I think we should strive for a predictable
> process—not requesting work from the submitter beyond what they can
> expect.  Submitters can be expected to follow the written rules¹ and
> perhaps a few more rules (for example, I don’t think we’ve documented
> the fact that #:tests? #f is a last resort and should come with a
> comment). 
> 
> However, following that principle, we reviewers cannot
> reasonably ask for work beyond that. [...]

We can ask to do send the notice upstream, if it's in the rules (I
consider this to be part of the unwritten rules).  And I don't often
have time for addressing the noticed issues and I have other things to
do as well -- I usually limit myself to just reviewing.  I do not
intend to take up work beyond that.

I also consider them to not be rules as in ‘if they are followed, it
WILL be accepted’ but more guidelines like ‘these things are important,
they usually need to be followed, but it's not exhaustive, at times it
might be discovered the list is incomplete’.

I don't think that patch submitters can reasonably expect reviewers to
do all the work.

Also, previously in discussions about the review process, weren't there
points about a reviewer not having to do everything all at once, they
could choose to review parts they know how to review and have time for
and leave the rest for others?

> Related to that, I think it’s important to have a consistent review
> process.  In other words, we should be equally demanding for all
> patches to avoid bad surprises or a feeling of double standard.

I think I've been consistent in asking to inform upstream of the issues
(*), with no distinction of whether it's a new submitter or an
established one or whatever.

(*) Except for when it's really basic downstream changes, like #:make-
flags #~(list ... #$(cc-for-target)) or a substitute* cc -> TARGET-gcc.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

Maxime Devos schreef op ma 27-06-2022 om 12:30 [+0200]:
> We can ask to do send the notice upstream, if it's in the rules (I
> consider this to be part of the unwritten rules). [...]
> I also consider them to not be rules as in ‘if they are followed, it
> WILL be accepted’ but more guidelines like ‘these things are
> important, they usually need to be followed, but it's not exhaustive,
> at times it might be discovered the list is incomplete’.

Also, some of those rules are incorrect -- "guix style" occasionally
makes things wrong and patch submitters had to be told to not follow
it.

Greetings,
Maxime.



signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

zimoun schreef op ma 27-06-2022 om 14:32 [+0200]:
> Hi Maxime,
> 
> On Mon, 27 Jun 2022 at 12:37, Maxime Devos  wrote:
> 
> > Also, some of those rules are incorrect -- "guix style" occasionally
> > makes things wrong and patch submitters had to be told to not follow
> > it.
> 
> Do you have concrete examples in mind?

E.g.: https://issues.guix.gnu.org/55606#16


signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

zimoun schreef op ma 27-06-2022 om 14:53 [+0200]:
> Maybe I misunderstand the point.  To me, the aim of the package
> submission is the inclusion in Guix.  AFAIK, the Guix project is not
> applying any standard audit on the upstream code before inclusion.
> 
> Therefore, if the upstream code is poor in some areas, then it is not
> blocking for the package adoption in Guix…

I think there should be some degree of standards (where I mean
standards in the non-shodyness sense of the word, not in the sense of
specs, though specs would be nice too) and of audit (bundling, malware,
non-free, bugs) -- some of which are blocking (bundling, malware, non-
free, some bugs), some of which aren't (some other bugs).

E.g., see removal of unmaintained Python, of some old SSL libraries

>>> However, AFAICT, none of that has happened yet.
> …and such adoption does not mean the upstream quality cannot be
> improved, by reporting or add a patch.

The problem is that this sometimes seems to be neglected unless people
are practically forced to make the issue be reported upstream.

> > My view is that such issues should be reported upstream but cannot
> alone
> > block package adoption in Guix.
> 
> I agree; we cannot fix the world. ;-) In the case of patch#55541, the
> issues of cross-compilation can be reported directly to upstream

Agreed -- I did not ask that explicitely in #55541, but the implied
question was to report it upstream (or fix local, that could be done
too).  But my point is that this should have been done _before_ merging
the patch.

> and another Debbugs number could be open.

Would be pointless.  Standard policy seems to be to leave the debbugs
issue unresolved forever, then someone does some cleanup in debbugs to
close the debbugs number due to lack of activity or such.  Things would
be delayed forever.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

zimoun schreef op ma 27-06-2022 om 17:06 [+0200]:
> Do you mean
> 
> --8<---cut here---start->8---
> > +(define-public harec
> > +  (let ((commit "bbabe09bddf74bd699f8ad2224fdd6e2eefbd35e")
> > (revision "0"))
> Despite what (guix style) may tell you, revision goes to an extra line.
> --8<---cut here---end--->8---
> 
> ? 

Yes, that comment (and some others, but that's the one I could find
quickly).

Greetings,
Mxaime.


signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

zimoun schreef op ma 27-06-2022 om 17:23 [+0200]:
> You are mixing unrelated topics, IMHO.
> 
> We have policies, not standard.
> 
> «A policy is a set of ideas or plans that is used as a basis for making
> decisions, especially in politics, economics, or business.» 
> 
> «A standard is a level of quality or achievement, especially a level
> that is thought to be acceptable.»

I think it's a reasonable policy to have standards, and that it's a bad
policy to not have standards.  Seems related to me.  Also, we do have
some standards -- e.g., bundling, malware, non-freeness and security
problems is not considered quality.

> > Agreed -- I did not ask that explicitely in #55541, but the implied
> > question was to report it upstream (or fix local, that could be
> done
> > too).  But my point is that this should have been done _before_
> merging
> > the patch.
> 
> So, what are you explicitly asking? :-)

Nothing, it was implicit.

> Other said, it cannot be asked to submitter to fix unrelated-to-Guix
> issue on upstream code.  Although cross-compilation issue is somehow
> related to Guix. ;-)

I never asked this.  All I asked for is to report the issue upstream,
such that upstream can fix it.  (Though fixing it would be nice bonus.)

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

zimoun schreef op ma 27-06-2022 om 17:23 [+0200]:
> Old unsolved bugs are still open

Sometimes they aren't:

 * https://issues.guix.gnu.org/45828
 * https://issues.guix.gnu.org/26705
 * https://issues.guix.gnu.org/25719 (exception handling hasn't been cleaned up 
before closing)
 * https://issues.guix.gnu.org/44199 (it's a WIP, not completed yet, but still 
closed!)

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

zimoun schreef op ma 27-06-2022 om 17:23 [+0200]:
> Old unsolved bugs are still open.  The cross-compilation of one package is
> an issue for sure, but:
> 
>  1. it is not an issue for inclusion in Guix
>  2. it has to be solved by people interested by cross-compilation

Unfortunately, systematic cross-compilation fixes tend to be ignored,
while individual new package patches can go through, so it is necessary
to do the fixes in the individual new package patches.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: guix refresh to a specific version?

[Maxime Devos]

Hartmut Goebel schreef op wo 15-06-2022 om 13:34 [+0200]:
> Hi,
> 
> I wonder whether this is a way to refresh to a specific version, like 
> one can import a specific version:
> 
> works:
> 
>  guix import pypi trytond@6.2.0

FWIW, this works for the 'crate' importer.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

zimoun schreef op di 28-06-2022 om 13:01 [+0200]:
> Well, from my understanding, the question is: should a perfectly working
> and fine submission be delayed because unrelated-to-Guix issues are in
> upstream code?

This is not the question.  The dispute is about:

Maxime Devos: https://issues.guix.gnu.org/55541#3
> AFAICT the issues have not been reported upstream yet, so I don't
> think we can close this entry on debbugs yet. 

zimoun:
> Ludo said these unrelated-to-Guix issues are not blocker, from my
> understandings.  And I agree.  Do you disagree?

I agree too.  What I disagree with, is ignoring the bug.  The blocker
for me is: appropriate parties need to be at least informed of bug if
it isn't fixed.

> You are commenting on “standard” which somehow asks about explicit
> criteria.  And, you are implicitly commenting on blocking while
> issues from upstream are not fixed.  Instead of trying to deduce
> myself (and probably the wrong way), could you please explicitly
> write down your arguments?

Reviewer noticed a $bug.  This kind of $bug has two accepted and
standard methods for addressing it:

 (1) fix it (by replacing the configure script or patching it or
 sufficient substitute*).
 (a) in Guix (often work-around-ish, though often a work-around is
 sufficient for these kind of cross-compilation problems)
 (b) upstream (more work, sometimes more fulfilling, sometimes not 
 worth it
 (2) report it upstream (because it's more complicated than a simple
 'substitute*'.

Why?  It's a bug, needs to be fixed somehow, and for (2): we can't
solve everything ourselves.

What happened:

  Committer pushed changed, ignoring (1) and (2).

/me: What?  Why ignore the bugs?

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

zimoun schreef op di 28-06-2022 om 13:01 [+0200]:
>  Do you think that patch#55541 should be
> delayed while submitter has not open an upstream issue?

Yes, as mentioned in .

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

zimount:
> Last, I miss these comments about old bugs and what you are implicitly
> suggesting with them.  Are you suggesting that old unsolved bugs are
> closed without valid motivation?

You often close bugs with as rationale: ‘no response since X months,
hence closing’, so it seems to me that you would simply close bug
reports if the bug reporter is gone.

> > > Old unsolved bugs are still open
> > 
> > Sometimes they aren't:
> 
> > * https://issues.guix.gnu.org/45828
> 
> Closed because:
> 
> This can happen if guix-daemon was restarted but ‘guix publish’ 
> wasn’t:
> ‘guix publish’ opens only one connection to the store at startup time,
> and then never tries to re-open it.  There was an old bug on this 
> topic:
> 
> https://issues.guix.gnu.org/26705
> 
> Back then I marked it as ‘wontfix’ because:
> 
>   1. Losing a connection to the daemon Does Not Happen™ in normal
>  conditions.  Namely, upon ‘herd restart guix-daemon’, ‘guix
>  publish’ is automatically restarted.  One situation where ‘guix
>  publish’ is not restarted is if one does “killall guix-daemon” or
>  similar.  (Perhaps that’s something to fix in the Shepherd?)
> 
> > * https://issues.guix.gnu.org/26705
> 
> Closed because:
> 
> For now I’m closing this bug as “wontfix” because I’ve never seen any
> occurrence of #2, and because #1 cannot happen on GuixSD (if 
> ‘guix-daemon’
> is restarted, the shepherd will also restart ‘guix-publish’.

It's a bug marked "wontfix" -- sure, I suppose #1 cannot happen on Guix
System, but there are foreign distros too.

> > * https://issues.guix.gnu.org/25719 (exception handling hasn't been cleaned 
> > up before closing)
> 
> Closed because:
> 
> I haven't seen this particular exception in a long time.  I cannot 
> tell whether
> the actual usability has been fixed, though--it could be that only 
> the servers
> are more reliable and this code path is thus not currently being 
> entered.

These kind of things are still bugs -- occassionally we see these kind
of bug reports pop up, so likely the underlying issue is still there
and error handlings is still loosy.

> > * https://issues.guix.gnu.org/44199 (it's a WIP, not completed yet, but 
> > still closed!)
> 
> This history is:
> 
> Maxime Devoswrote on 24 Oct 2020 21:47
> zimoun  wrote on 27 Oct 2020 14:39
> Maxime Devoswrote on 27 Oct 2020 19:50
> Maxime Devoswrote on 1  Nov 2020 01:05
> Ludovic Courtès wrote on 15 Nov 2020 22:13
> 
> > This patch defines a `gnunet-fetch' method, allowing for downloading
> > files from GNUnet by their GNUnet chk-URI.
> 
> While I think this is a laudable goal, I’m reluctant to including 
> GNUnet
> support just yet because, as stated in recent release announcements,
> GNUnet is still in flux and not considered “production ready”.
> 
> So I think we should keep it around and revisit this issue when GNUnet
> is considered “stable”.  WDYT?
> 
> zimoun  wrote on 16 Nov 2020 01:35
> Maxime Devoswrote on 18 Nov 2020 20:14
> 
> > So I think we should keep it around and revisit this issue when
> > GNUnet
> > is considered “stable”.  WDYT?
> 
> Sounds reasonable to me. There are also a lot of missing parts: a
> service definition for Guix System, findings substitutes, finding
> sources by hash (the one Guix uses, not the GNUnet hash) ..., so it
> isn't like my rudimentary patch was usable on large scale anyway.

Oh right that was a bad example, the approach is broken (no http/https
fallbacks, bootstrap problems, etc); current idea is to extend
(guix download) with gnunet://fs/... instead.



> Therefore, if you have more details for one of these reports, feel free
> to comment, provide more info or fix; for sure it will help.

That's the issue I wanted to highlight -- issues are closed before
being fixed when the the reporter disappears (and hence, cannot provide
"more info", or has other things to do than provide a fix by
theirselves), even if the bug is understood.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

zimoun schreef op di 28-06-2022 om 18:21 [+0200]:
> And… as I wrote [1]:
> 
>     I agree; we cannot fix the world. ;-) In the case of patch#55541, the
>     issues of cross-compilation can be reported directly to upstream and
>     another Debbugs number could be open.
> 
> 1: 
> 
> 
> The bug is not ignored, to the contrary.

It is -- where's the bug report upstream or a fix?

> But it is not a blocker for patch inclusion and so patch#55541 can be closed.

It is a blocker -- as previousy mentioned, upstream doesn't even know
about the bug.

> You are free, as the reviewer, to open a report for Guix pointing the
> issue of cross-compilation of ’azpainter’;

As mentioned previously, that is not reviewing, and I don't have time
to fix the world.  I have other things to do than only acting on guix-
patches@.  Also, as mentioned previously, can't a reviewer do a partial
review?  Why the insistence on the reviewer -- isn't making a proper
patch the submitter's job, not the reviewer's?

> Well, you and me spend more time in discussing that than in just
> reporting the issue. ;-)

Yes, but if I just give in and report the issue after other insist I do
it, then I create the expectation that I'll just do everything.  I
refuse, I insist on being allowed to quit doing something and start
doing other things.

(So TBC, I won't be doing that.)

> Ultimately, nothing is perfect and people are doing their best with
> their resource at hand; at least, I do my best with the resource at
> my hands.  I would be more than happy if more people would try to
> sort, classify or fix the old bugs.  Maybe, you will join the effort
> ?

I would have more time for that if we refused patches with issues like
this one and did not insist on letting the reviewer do the submittter's
job.

Greetings,
Maxime.




signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

zimoun schreef op di 28-06-2022 om 18:25 [+0200]:
> My workflow dealing with old bugs is: pick one and read the report (and
> the complete thread, if any), then,
> 
>  1. the report provides enough information for reproducing; I try to
>     reproduce myself and report more info, and then I try to collaborate
>     for fixing or closing.
> 
>  2. the report does not provide enough information to understand what
>     the bug is about or to find a way to reproduce; then I ask more info
>     – sometimes my reply is even the first one, then, [...]

This procedure looks reasonable to me!

Greetings,
Maxime.



signature.asc
Description: This is a digitally signed message part

Re: Dealing with upstream issues

[Maxime Devos]

zimoun schreef op di 28-06-2022 om 19:36 [+0200]:
> Hi,
> 
> On Tue, 28 Jun 2022 at 18:47, Maxime Devos  wrote:
> 
> > It is -- where's the bug report upstream or a fix?
> 
> Upstream [1] does not have a bug tracker, or I am missing it.  See
> bug#56285 for tracking the issue in Guix.
> 
> 

O right I forgot about that.  In that case, it's a lot more complicated
than I thought ...


signature.asc
Description: This is a digitally signed message part

Re: Wiki && Re: [feature request] merge sxml->html from (haunt html) into guile?

[Maxime Devos]

Blake Shaw schreef op wo 29-06-2022 om 01:34 [+0700]:
> Which brings up another thing I've been considering working on thats
> been discussed in the Guix community: the need for click-to-edit
> wiki, written in Guile. [...]

I don't think ‘written in Guile’ has been discussed?

Also, I don't see the point in writing our own wiki software in Guile
when there's already plenty of Wiki software in existence with lots of
tools for moderation, version control, backups, lots of testing, etc.
Why not reuse pre-existing work instead of reinventing the wheel (likely
poorly, at least at first)?

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Wiki && Re: [feature request] merge sxml->html from (haunt html) into guile?

[Maxime Devos]

Maxime Devos schreef op di 28-06-2022 om 22:13 [+0200]:
> Blake Shaw schreef op wo 29-06-2022 om 01:34 [+0700]:
> > Which brings up another thing I've been considering working on
> thats
> > been discussed in the Guix community: the need for click-to-edit
> > wiki, written in Guile. [...]
> 
> I don't think ‘written in Guile’ has been discussed?
> 
> Also, I don't see the point in writing our own wiki software in Guile
> when there's already plenty of Wiki software in existence with lots
> of
> tools for moderation, version control, backups, lots of testing, etc.
> Why not reuse pre-existing work instead of reinventing the wheel
> (likely
> poorly, at least at first)?
> 

(TBC: you can, if you want)


signature.asc
Description: This is a digitally signed message part

Re: Non-free data in Poppler test suite

[Maxime Devos]

Marius Bakke schreef op di 28-06-2022 om 23:19 [+0200]:
> I discovered a potential freedom issue with the Poppler test suite.
> Specifically it includes a file with the CC BY-NC-ND (non-commercial)
> license:

Given that it what (some tests) are based on, this might count as
functional data and hence not covered by the ‘non-functional data’
exception of the FSDG.

OTOH, the FSDG isn't mentioned anywhere in the manual, but ‘software in
Guix is free’ is, so maybe the FSDG doesn't apply.  Though in past
discussions, the conclusion was that the FSDG applies to Guix, so maybe
the exceptions to the freeness policy just are in lack of
documentation.

(No point in rehashing old rationale, but I'd like to point out this
policy isn't actually documented anywhere in Guix.)

Only one way to know sure what the writers of that page meant I think
-- contact the FSF?

> However, we failed to reach a consensus on #guix[0].  What do others
> around here think?  Should we play it safe and disable Poppler tests?
>  Raise the issue with FSF? 

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Autotools-generated 'configure' & 'Makefile.in' considered binaries?

[Maxime Devos]

Small clarification to old discussion:

Hello,

Ludovic Courtès  writes:

> Hi!
>
> Maxime Devos  skribis:
>
>> Ludovic Courtès schreef op vr 01-04-2022 om 10:58 [+0200]:
>>> As a first milestone, maybe we could start running ‘autoreconf’
more
>>> often, for packages higher in the graph.  We could change the
>>> ‘bootstrap’ build phase to do that unless it’s explicitly turned
off.
>>> It may turn out to be a Sisyphean task though…
>>>
>>> Thoughts?
>>
>> Changing all pre-existing packages, maybe.  But doing this for new
>> packages (reducing review effort) and perhaps when a package is
updated
>> (for purity) should be feasible I think?  Then gradually things
would
>> improve and eventually(TM) doing the switch in the bootstrap phase
may
>> become feasible ...
>
> Yes, we could do that as a first step (in fact it’s already happening
as
> some projects no longer distribute tarballs).
>
> What do maintainers think of that policy?

No strong opinion, but I agree that having a complete development
environment capable of building from the bare sources (e.g. a git tree)
is useful in general.  On the other hand, using tarballs is often
more efficient and practical (it's made to be built by downstream
users,
rather than by developers, so it includes everything needed).  Release
tarballs are also often signed by the projects, which is neat.  So
perhaps we can leave some flexibility there and not make it a hard
rule, but a case of best judgment?

We can both use tarballs _and_ unbundle/regenerate, no need to switch
from tarball to git (though that's one way to do things)!  E.g., adjust
the bootstrap phase to always "autoreconf -vif" and/or add snippets or
post-unpack phases to remove generated or bundled Autotools files.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Shall updaters fall back to other updaters?

[Maxime Devos]

Hartmut Goebel schreef op do 30-06-2022 om 10:58 [+0200]:
> Hi,
> 
> while working on refreshing to a specific version (see 
> https://lists.gnu.org/archive/html/guix-devel/2022-06/msg00222.html) I 
> discovered that the updaters fall back to another updater. Is this intended?
> 
> Concrete example (using refresh to a specific version): Package "xlsxio" 
> has no version 0.2.30. When trying to refresh to this version, the 
> github updater comes first and of course fails to get this version. Then 
> the generic-git updater is triggered and tries to get the version.
> 
> IMHO each package should be handled by a single updater.

I think that, in the absence of pre-existing knowledge which updater is
the right one, trying out several until we get the ‘right’ one is
reasonable.

However, to avoid non-determinism, some CPU time, some I/O and
messiness, I think that somehow annotating packages to write down
_which_ updater applies would be reasonable (maybe with some defaults,
e.g. for minetest mods, ContentDB would be considered authoritive)

Anyway, this idea of ‘authoritive updaters’ as it has been raised
before (I think by Liliana, in the context of the Minetest updater and
generic-git), but I couldn't find the mail again.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Rust reprodubility -- .rmeta and shadow-rs

[Maxime Devos]

> https://issues.guix.gnu.org/50015

I don't think that shadow-rs changes Cargo.toml, so I think that one is
a separate issue ...

> https://issues.guix.gnu.org/55928

Maybe related, though .rmeta isn't mentioned there so maybe not.
Seems debugging information related, so maybe the report at (and fix
at)
https://github.com/rust-lang/rust/issues/34902#issuecomment-565557076
is important there.

Ludovic Courtès schreef op do 30-06-2022 om 13:35 [+0200]:
> > Anyway, here the patch I used:
> > 
> > [...]
> > 
> > Maybe that was the cause?
> 
> You mean this issue you identified could have been the cause of
> reproducibility issues found in other Rust packages?

I don't know if it applies to leaf packages or only to dependencies,
but perhaps! 

> Anyway, it looks like the snippet above should be applied to
> ‘rust-shadow-rs’ in current ‘master’, no?

Yes -- also, upstream has a patch now!

That's only for the ordering, not the timestamps, though.
The timestamp issue might be caused due to how antioxidant replaces
some inputs and fiddles with ‘features’ so I haven't reported that yet


Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Non-free data in Poppler test suite

[Maxime Devos]

Ludovic Courtès schreef op vr 01-07-2022 om 14:57 [+0200]:
> Nitpick: it’s not that “the FSDG applies to Guix” but rather the Guix
> project chooses to follow the FSDG (info "(guix) Software Freedom").

OOps, I searched for 'FSDG' but not for 'free software distribution
guidelines' ...

Greetings,
Maxime.

Re: “Building a Secure Software Supply Chain with GNU Guix”

[Maxime Devos]

Arun Isaac schreef op di 19-07-2022 om 12:51 [+0530]:
> 
> Hi Ludo,
> 
> >    https://doi.org/10.22152/programming-journal.org/2023/7/1
> 
> This is an excellent read! Are there plans to release this git
> authentication system as a separate tool so that other non-Guix
> projects may use it easily?

FWIW I have been using "guix git authenticate" + .guix-authorizations
in the Git repo of Scheme-GNUnet for letting users verify whether they
got an 'authentic' copy.

Greetings,
Maxime.

Re: “Building a Secure Software Supply Chain with GNU Guix”

[Maxime Devos]

Ludovic Courtès schreef op ma 18-07-2022 om 10:45 [+0200]:
> The model here is that users trust authorized committers.  When you
> think about it, there’s no way around it, because at the end of the
> day, you’re installing software that an authorized committer added to
> the channel.

FWIW, something I haven't seen mentioned yet is that the trust problem
could be reduced by some kind of multisig system, where multiple
independent persons would need to sign the commit for it to be
accepted, though that might be technically hard to implement and
probably be too people-time-expensive currently.

Greetings,
Maxime.

Re: native-inputs: Go for completeness or minimalism?

[Maxime Devos]

On 20-07-2022 10:33, Hartmut Goebel wrote:

Hi,

shall native-inputs be as complete as possible or as minimal as possible?

Background: I just stepped over a couple of packages where upstream 
requires a lot of code-quality checkers which are not actually run 
when running the tests. (More specific: These are Python packages 
demanding tools like flake8, flake8-docstring, black, bandit.)

Now when going for minimal dependencies and minimal native-inputs,

Pro: Less dependencies, simpler dependency tree, thus less 
computation, faster, less power consumption.


Con: Might need phase to remove dependencies, 'guix shell -D' will not 
provide every development requirement.


Personally I tend to minimal.

WDYT? 
I would go for only the dependencies actually used in the build process 
(including the test process), for less dependencies (we don't include, 
say, 'git' for every package that has a Git repo, only if the package 
actually uses Git in some fashion). For the "guix shell -D" for 
developing on the software, a "guix.scm" or "manifest.scm" can be 
written and included in the source code repo that contains the extra 
dependencies, though I have no idea what percentage of upstreams would 
actually include those.


Alternatively, packages could have an additional set of inputs 
(development-inputs?) for this use case, only added for "guix shell -D" 
and "guix environment", though then the build environment and "guix 
shell -D the-package" would diverge further.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Could Guix System eventually run on top of HyperbolaBSD ? slightly off topic

[Maxime Devos]

On 20-07-2022 18:03, Raghav Gururajan wrote:
[2] IIUC, HyperbolaBSD (OS) consist of a custom-made kernel and a 
custom-made userspace, both of which the components are either derived 
from OpenBSD System or written from scratch. So two things can be 
explored, *separately*.

(A) Guix System with GNU userspace and HyperbolaBSD kernel.
(B) Guix System with HyperbolaBSD userspace and HyperbolaBSD kernel. 


(C) Guix System with HyperbolaBSD userspace (except for GNU C library 
and compiler) and Linux or Hurd kernel


Probably won't work for all userspace things but possibly some of them 
run on Linux/Hurd+glibc


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [WIP Patch] Adding an FHS container to guix shell

[Maxime Devos]

On 20-07-2022 23:22, John Kehayias wrote:


I was thinking mostly of binaries, though I've also encountered code that 
wanted to read the ldcache (e.g. parsing ldconfig -p directly), for some reason.


Do you have an example in the wild?

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Building, packaging and updating Guix with confidence

[Maxime Devos]

On 21-07-2022 18:10, Josselin Poiret wrote:


b...@bokr.com  writes:

Naively:

Why does "the" guix daemon per se need root access at all?

The main thing is that all files in the store end up being written by
the guix daemon user.  So if we want the files to be easily
substitutable, they'd need to have a fixed uid/gid, and the only one we
can guarantee is root.  Other than that, it needs to use a bunch of
Linux namespaces to isolate the builds from the rest of the system,
which depending on the kernel build-time configuration might not be
possible when unprivileged.


Also, resource savings on multi-user systems. And if the guix daemon is 
run as the regular user, then all other daemons (on Guix System) would 
need to be run as that user or as root to be able to access theirselves, 
which is bad from a security perspective.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Is Guix suitable for large monorepos?

[Maxime Devos]

On 22-07-2022 08:58, Liliana Marie Prikler wrote:


15:10 "An engineering organization is not a bottom-up kind of thing"
(X) Doubt.
15:18 "In a well functioning engineering team, priorities and decisions
and effort allocation flow top-down"
(X) Doubt.
15:24 "Some sort of top-down organization is required"
(X) Doubt.


It's a bit of a stretch, but in a sense Guix is a bit top down, if you 
count Guix itself (and the reviewers, committeres, etc. as a while) as 
'top' and individual patch submitters as 'down'.  OTOH with Guix doesn't 
make decisions on what individual people should work on, only on the 
rules that the results should follow and there is no effort allocation, 
so possibly this is not the kind of 'top-down' that the video was 
referring to.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Is Guix suitable for large monorepos?

[Maxime Devos]

On 22-07-2022 08:58, Liliana Marie Prikler wrote:


13:43 "... if you are scrupulous about running tests ..."
Are you really going to run all the tests from hex0 to node whenever
you flip a bit?  Chances are no.


Also, with some exceptions (e.g. emacs packages and texlive), tests are 
run by default in Guix -- you'll have to disable them explicitly for 
them not to be run, and if you try to submit it without tests, reviewers 
will ask why.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Guix-devel Digest, Vol 109, Issue 56

[Maxime Devos]

On 22-07-2022 19:12, kiasoc5 wrote:

We could have packages recommend other packages to make this discovery
easier for users, like Arch's opt-depends.


This sounds like my previous proposal to me:

Alternatively, packages could have an additional set of inputs 
(development-inputs?) for this use case, only added for "guix shell 
-D" and "guix environment", though then the build environment and 
"guix shell -D the-package" would diverge further. 
except it has a more precise semantics than Arch's, or did you have 
something different in mind?


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: python-pytest in references graph

[Maxime Devos]

On 24-07-2022 22:25, Roel Janssen wrote:

I'm trying to understand the output of:
$ guix graph --type=references python-rdflib | dot -Tsvg -o rdflib.svg

Particularly, I'm looking at why python-pytest has an input arrow from 
python-rdflib, while it's
"only" a native-input?  I thought the "references" graph type would only 
include run-time
references, but I don't know what happens in this case.

What am I missing?


It should, but sometimes there are bugs in the package definition or 
build system, in this case causing python-rdflib to refer to the 
native-input python-pytest.  Likely it's the 'add-install-to-path' phase 
adding too much, a known issue, which could be solved by separating 
inputs and native-inputs on the build side when compiling natively (and 
not only when cross-compiling) (currently they are merged together into 
'inputs'), though non-trivial.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: xpip install -U 'xonsh[full]'

[Maxime Devos]

On 25-07-2022 00:29, jgart wrote:

When installing xonsh I get the following after starting:

```
You are currently using the readline backend. For interactive tab-completion, 
on-the-fly syntax highlighting, and more, install prompt_toolkit by running:

   xpip install -U 'xonsh[full]'

```

How should Guix handle optionally installing xonsh[full]?


I guess it could ask to install it with Guix instead. It's a bit 
complicated with the multiple mechanisms (guix install, guix home, "guix 
shell foo bar -- baz"), but that can be avoided by patching it to 
rephrase it a bit:


```
You are currently using the readline backend. For interactive 
tab-completion, on-the-fly syntax highlighting, and more, install the 
'python-prompt-toolkit' package.

```

OTOH, from what I've gathered from previous discussions, some people 
want "guix build --source" to be the unmodified source code, with as 
only exception non-freeness removal, so patching would be out. E.g., 
doing the "-O1" bugfix in the shepherd package as a patch (well, 
snippet, but same thing basically) was rejected by some people in 



But at the same time, we patch glibc, gcc and guile-fibers without any 
complaints while those are more complicates patches and (in case of 
glibc and gcc), even Guix-specific, while the shepherd patch is simple 
and non-Guix specific.


(I don't understand anything from those discussions)

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

Context: it's currently a mess:, and at times contradictory

 * There is policy involving those three, as can be seen from the
   shepherd mess.
 * This policy is partially secret, as can be seen by some people
   treating some things as policy even if it's not in the manual.
 * Some versions of the policy are based on archeology, e.g. see the
   'snippets were introduced for this particular purpose, so don't use
   it for other things’ which is not documented in the manual as sole
   legitimate reason and asking contributors to read all past
   discussions seems too much for me.
 * Sometimes, people refer to the manual (Snippets versus Phases) for
   how things (should) work, but what they say is not actually present
   in that section of the manual.
 * Some variants of the policy are contradictory with each other (IIRC)
 * Some of the policies are contradictory with current practice in
   other Guix packages.
 * '(guix)Snippets versus Phases says Phases' states that it is elusive.
 * The section name implies it's a ‘X versus Y’, which seems
   polarizing? (Maybe?)
 * The section neglects the is/ought-distinction -- it just says what
   is typically used for what, not whether they should be used for them
   and whether they are allowed to be used for other things, so that
   section does not seem policy to me (except for the single 'should
   produce' and 'must not' line), only matters of fact.

I can't work with such a mess. As such, I've a proposal for a 
consistent, clear and non-elusive set of rules and guidelines, based on 
the following principles:


 * It appears we cannot agree on what exactly the policy should be, but
   having a single policy everyone can use even if some would rather
   have the specifics be a tiny bit different, is much better than the
   mess of everyone having their own policy.
 * There are no absolutes, except that the result of "guix build
   --source" must be free software;
 * There can be more than one (acceptable) way to do things, but this
   doesn't make things elusive, this just means there are multiple
   acceptable options and you should probably go for the simplest.

More concretely, I propose the following new contents for (guix)Snippets 
versus Phases (the phrasing could use some work for smooth reading), 
which I believe to be sufficiently clear (except for some phrasing that 
could be tweaked, e.g. the phrases are currently rather long), covers a 
sufficient amount of cases (feel free to respond if you see a missing 
case), free of contradictions (likewise) and mostly in line with current 
practice:


[start]

@c: There is no opposition or such, so no versus, let's not start with 
polarisation.


20.4.5 Snippets, phases and patches

Snippets, phases and patches at times serve overlapping purposes. To 
decide between the three, there are several considerations to keep in mind:


 * Patches must not be used to remove non-free files, because a patch
   by construction contains the non-free file itself so the patch would
   be non-free, which would not be acceptable to Guix. Likewise,
   patches should not be used to remove bundled libraries, to avoid
   large space usage, but this is not an absolute rule unlike as for
   non-free files.
 * Snippets are often convenient for removing unwanted files such as
   bundled libraries, non-free sources and binaries. It is technically
   also possible to use phases for this, albeit slightly less
   convenient at times. However, phases must not be used to remove
   non-free sources, as then the output of "guix build --source" would
   still contain the non-free sources, which is incompatible with Guix'
   stance on free software. Likewise, phases should not be used to
   remove binaries; however, this is not strictly forbidden.
 * Snippets must not embed store items in the source, as this is
   incompatible with cross-compilation and prevents effectively sharing
   the source code produced with "guix build --source" with people
   using non-Guix systems.
 * In principle, you can apply a patch from a phase. However, this
   causes the result of "guix build --source" to not correspond to the
   actual source code anymore (i.e., it doesn't act as corresponding
   source anymore), so consider this a last resort for situations such
   as avoiding causing a world-rebuild for a patch fixing a
   target-specific bug by making the patching conditional upon
   target-foo?. If you apply a patch from a phase, make sure that the
   patch appears in the inputs or native-inputs, such that "guix build
   --source=all" will include the patch.

   @c this relaxes the old rule a little

 * Ideally, the source derived from the origin should be usable for
   building on any system that the upstream package supports (even if
   Guix does not support that system), as a courtesy to the people that
   the source code is shared with. However, this is not an absolute
   rule, most important is that it is usable on Guix and it is allowed
   to neglect

Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 25-07-2022 07:21, Julien Lepiller wrote:

I don't like the wording at all. You're mixing too many things together.
Feel free to try to separate the things, but going previous discussions, 
many tings are important, and they appear all to be inseparable.


I think it would be better to first document the guiding principles 
(eg. the goal that there are no non-free software in Guix, going for 
the simplest thing, etc) and then derive rules for specific cases, 
based on these principles:


How do I remove non-free software? -> snippet because …

How do I remove bundled libraries? -> snippet or phase because …

How do I fix a build issue? -> patch or snippet if this affects 
building from source, can also be a phase if the result of --sources 
can still build


A test issue?

…

This leaves some cases up to interpretation, but that's probably not 
so different from "it's not an absolute rule". It's also much clearer 
and quicker to figure out in which case you are. If not documented as 
a case, you can fall back to the general principles.


TBC, is the issue here the structure of the section, or some individual 
rules? In the former case, I could try rewriting it a bit to follow your 
proposed structure.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: nix installed with guix on a foreign distro

[Maxime Devos]

On 26-07-2022 08:00, jgart wrote:


Hi Guixers,

How can we make it easier to install nix with guix on a foreign distro?

Guix System has a nix-service-type but there is no such thing on Debian.

wdyt


This sounds like a question for Nix or Debian people to me, as AFAICT it 
doesn't concern Guix.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Strategy for Zig packages

[Maxime Devos]

On 26-07-2022 20:48, Liliana Marie Prikler wrote:

4. Convince Zig maintainers to perhaps maybe not join the ranks of Rust
et al. and produce reusable shared libraries?


I'd like to clarify that Rust supports shared libraries (*) just fine, 
it's Cargo that insists on source code.


Looking at 
, it 
might be similar with Zig, as that page is about 'the Zig caching 
system' and mentions Zir code, ABIs and objects.


(*) or, at least, rustc's version of static libraries, I haven't tested 
out shared libraries yet except for the first failed attempt; there 
might be problems with monomorphisation and macros or somehing. Some 
libraries, e.g. recent versions of rust-nom, also are written in such a 
way that appears to require LTO optimisation to be efficient, which 
appears to be non-trivial to be compatible with shared libraries without 
extra disk space.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: developing javascript with guix

[Maxime Devos]

On 27-07-2022 02:25, jgart wrote:


Should we make a guide for developing with js and guix?

For example, how does a js developer use `guix shell` to load a js lib like
node-rimraf in a repl currently?

This one is not in Guix, so I'll replace it by node-mersenne.

There's currently no doc for getting started with that iirc


Let's try not doing anything special:

$ guix shell node-mersenne node
$ node
> mersenne = require('mersenne')
{
  MersenneTwister19937: [Function:think we need documentati 
MersenneTwister19937],

  rand: [Function (anonymous)],
  seed: [Function (anonymous)],
  seed_array: [Function (anonymous)]
}
> mersenne.MersenneTwister19937
[Function: MersenneTwister19937]

No setup appears to be required beyond simply installing node and the 
node library and using node's 'require' function, so I don't think we 
need documentation for this in Guix (I'm assuming that node documents 
'require'), though I suppose we could have it for completeness  (*).  If 
doing that is considered a good thing, it seems to me that it should 
then also be done for Guile, Python, C/C++/etc, Minetest, Vim, ...


Greetings,
Maxime.

(*) E.g., when looking for the 'require' function, there was initially 
some slight confusion with 'require' accepting relative and absolute 
file names so I was fearing it might need to be passed 
$GUIX_ENVIRONMENT/lib/node_modules, but this turned out to be unfounded.




OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: developing javascript with guix

[Maxime Devos]

On 28-07-2022 00:25, jgart wrote:

How are users supposed to know to run node and node-mersenne?:


They know that, because that was the premise of what they were trying to do:

You wrote: "how does a js developer use `guix shell` to load a js line 
like node-rimraf in a repl currently`:


In particular: "load a js like node-rimraf" = run node-rimraf (or in my 
example, node-mersenne).


Also, given that the question contains 'node-SOMETHING', I think it's 
clear that they wanted to use the _node_ REPL (and not, say, the Guile 
REPL).


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: developing javascript with guix

[Maxime Devos]

On 28-07-2022 01:15, Ryan Prior wrote:

  At a minimum, can we make `guix shell` warn on stderr if you create a shell 
with one or more libraries but no interpreter?
I suppose, but this isn't `guix shell` specific really, it hold for all 
users of profiles (including "guix environment", "guix system", "guix 
home", ...).


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: developing javascript with guix

[Maxime Devos]

On 28-07-2022 01:15, Ryan Prior wrote:

Why isn't node a dependency for node-mersenne though?


It is. node-mersenne uses node-build-system, which has node (or 
node-lts, dunno) in its implicit inputs.


Did you mean: 'Why isn't node propagated?'

For the same reasons as why any plugins don't propagate the thing they 
are plugging in, and why any library doesn't propagate its interpreter:


 * There is more than one implementation, how would Guix know which one
   you'd like?
 * Propagation stops you from separately updating the library and the
   interpreter (with "guix package -u this-package", or time machines,
   or defining a package variant).  (Actually, "guix package -u
   this-package" will ‘succeed’ because collision detection is disabled
   IIUC, but you can't decide which one wins)
 * ???

Also, see my answer on 'Is there really a use case for shipping ...’.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: developing javascript with guix

[Maxime Devos]

On 28-07-2022 01:15, Ryan Prior wrote:

Is there really a use case for shipping the source code of a JavaScript library 
without the interpreter?


Yes -- see "guix build --source" (it's not JavaScript-specific).

If you meant the _compiled_ JavaScript library (result of "guix build 
node-...") (even when it's maybe just copying source code around and 
maybe minimisation), then also yes:


JavaScript isn't only used in node, but also on the web, in browsers. 
While not all node-based libraries are usable on the web, I'd assume 
some of them are.  So it would be nice if "guix shell -D 
the-web-server-app-thing" didn't install the node interpreter you didn't 
ask for (functional package management) and don't need (resource savings).


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: developing javascript with guix

[Maxime Devos]

TL;DR: I disagree with many of the claims, I agree that some 
documentation using specific examples is useful, I disagree that this 
isn't what we are already doing in the Guix manual, I agree that we 
don't have per-language landing pages yet and that they could be a 
convenient starting point, I disagree on the degree to which this is 
useful, I disagree that this is required.


On 28-07-2022 01:15, Ryan Prior wrote:

## Explanation in context

The language-specific package managers generally don't take for granted that people know 
anything about the language, because they're designed to be accessible to learners of the 
language. For example, PyPI's explanation for pip starts with the basic "can you 
even run Python?" and goes through a bunch of Python-specific package 
workflows:https://packaging.python.org/en/latest/tutorials/installing-packages/
Neither does the Guix package manager take for granted you know anything 
about the language -- the package management commands are 
language-independent in Guix. (imports are another matter, but those are 
a language manner, which Guix is not involved with). Also, AFAICT, that 
workflow is not Python-specific but PyPi+pip-specific, except for the 
"python3 --version" check. It also seems abstract to me, i.e., it 
doesn't mention any specific packages to install for some project but 
uses abstract commands like "python3 -m pip install "Foo==some-version", 
nothing concrete?


We could write a guide like that


We already have one: see (guix)Getting Started.


with information and example commands specific to Python packaging, and another 
for JavaScript, etc.

These commands are not language-specific in Guix.

These provide explanation in context so beginners and people who are confused 
for whatever reason can see concrete examples of what you're supposed to do.


Context and examples exist in (guix)Getting Started (example: Emacs, 
context: Getting Started,.



Meanwhile, in the Guix docs, everything is abstract. We don't name any specific 
library commons or restrict package search to any specific namespace, we don't 
even have tags or categories for them.


"guix search node mersenne" -> node-mersenne
"guix search python mersenn" -> no results (can happen)

"guix search python aes" -> some python libraries implementing AES
"guix search node aes" -> likewise, no results.
"guix search rust aes" -> many results

No tags or categories are needed if the descriptions are good. Maybe 
this can be mentioned among the examples in (guix)Invoking guix package?



  There is no link to Guix documentation I can give to a Python hacker that 
assures them, in the way PyPI's website does, that Guix has the stuff they need 
and they can find it and make it work. So Guix requires more faith and 
experimentation from users, which means a lot of people will just bounce off it.
Myself, I doubt that this will give much assurance or help them beyond 
what the docs already do, but I suppose we could make a few 
language-specific 'cheat sheets' of sorts, which seems to be the kind of 
thing you are referring to?  E.g.,:


* Install the Python interpreter: $ guix install python

  (see (guix) for more info on "guix install")

* Install some Python libraries: $ guix install python-thisexample 
python-thatexample

* Search for Python libraries: $ guix search python more-keywords...

  (see [...] for more info on [...])

  Example: search for Python cryptography libraries: $ guix search 
python crypto ...


* Start the Python interpreter: $ python3

* Start Python and import some libraries:

  $ python3

  import foobar

  (see https://some-python-site on how to use imports in Python.)

  Important: the package name does not always match the name used in 
Python!.


I don't think this brings much, but not much is still a little, and a 
little is still useful.


If there is interest, such workflows could be worked out in prose.

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Strategy for Zig packages

[Maxime Devos]

On 01-08-2022 22:43, Mája Tomášek wrote:

But I understand it wouldn't be popular, locking an independent language
into the guix ecosystem.


They could consider multiple package managers and distros (e.g.: Guix 
support, Debian support, vcpkg support, ...) according to what users of 
Zig are using, then they aren't locked into a particular distribution or 
package manager.


A bit more work upfront than rolling their own, but it will have to be 
done anyway later.


Greetings,
Maxime



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Strategy for Zig packages

[Maxime Devos]

On 02-08-2022 07:21, mcsi...@disroot.org wrote:

On Mon Aug 1, 2022 at 10:43 PM +0200, Mája Tomášek wrote:

More realistic (imo) is that zig should be encouraged to build
dynamically linked packages, not static ones, and allow the ability
(with their future package manager) for the distribution to distribute
it's libraries C-style.

Technically this is not always possible since Zig relies
on compile-time execution for generic.

I don't think this is a problem, as Guile has macros (*) yet it supports 
compilation (and shared libraries -- .go files are ELF shared libraries) 
-- macros are, in a sense, merely procedures that transform syntax into 
syntax, which can be compiled, and generics could be considered a 
special case of macros.


Likewise, Rust has generics (and macros (*)) and its compiler does 
static compilation, so I don't see why it would be impossible for Zig to 
do static libraries.


(Shared libraries might be harder, but you could just monomorphise 
generics in the 'user' of the shared library, so shared libraries appear 
technically possible to me as well.)


(*) Including macros that can run arbitrary Rust code, see 
, 
so there you have compile-time execution too!


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Strategy for Zig packages

[Maxime Devos]

On 02-08-2022 07:21, mcsi...@disroot.org wrote:


Zig source files could be handled in the same manner as C headers
however, and be used as native inputs, so downstream can still
update a library for all dependees at once.


Going for native-inputs and not non-native sounds incorrect when 
cross-compiling.  We might need to substitute* a "executable" -> 
"/gnu/store/.../bin/executable", and the latter file name is 
architecture-dependent.  (This holds generally, including for C headers).


Greetings,
Maxime



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Strategy for Zig packages

[Maxime Devos]

FWIW I was commenting on the impossibility
of dynamically linking Zig libraries that uses comptime.
It doesn't make a difference, AFAICT my explanation on why it can work 
holds equally for static as for shared libraries.


On 03-08-2022 05:35, mcsi...@disroot.org wrote:

Compile-time execution must be done at, well, compilation time,
thus macros in shared objects are just glorified embedded
source code that are not used at runtime.


My point was that macros and dynamic (or static) linking is compatible 
with generics and macros. Of course, if your Zig code consists purely of 
generics and macros, it's kind of pointless, but presumably there's 
non-generic code too and non-generic library code that chooses a 
particular instance of a generic (monomorphisation?) that could benefit 
from compiling only once instead of being recompiled for every leaf (+- 
application).



That being said, there may be a Zig ABI in the future, with the cost
of missing certain features (like comptime) and optimizations:
https://github.com/ziglang/zig/issues/3786


A fixed, stable, ABI or documented ABI is not required for Guix because 
we have the store and RPATH (or RUNPATH, I always forget which one we 
use) and the store model instead of putting things in /usr/lib and 
searching for them there, so occasional ABI breaks are not a problem for 
Guix.


Distros like, say, Debian, are another matter of course, though they 
could resort to build-from-scratch like done with Rust currently.


You are referring to optimisations, but at least LTO is perfectly 
combinable with static libraries (see, e.g., Rust). Shared libraries 
should be equally possible if it is accepted that it won't be always 
possible anymore to simply replace the shared library with another 
(because of things like inlining).


Greetings,
Maxime


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 05-08-2022 05:38, Philip McGrath wrote:

On Sun, Jul 24, 2022, at 11:17 PM, Maxime Devos wrote:

  * In principle, you can apply a patch from a phase. However, this causes the result of "guix 
build --source" to not correspond to the actual source code anymore (i.e., it doesn't act as 
corresponding source anymore), so consider this a last resort for situations such as avoiding 
causing a world-rebuild for a patch fixing a target-specific bug by making the patching conditional 
upon target-foo?. If you apply a patch from a phase, make sure that the patch appears in the inputs 
or native-inputs, such that "guix build --source=all" will include the patch.

Should we have an option for "guix build --source=all" to also include the Guix 
"scripts used to control compilation and installation"?


What do you mean with "Guix scripts" here? I don't think Guix has a 
notion of 'scripts' (except stuff like 'wrap-script', but that doesn't 
seem relevant here). Do you mean the phases code?


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 05-08-2022 05:23, Philip McGrath wrote:

On Sun, Jul 24, 2022, at 11:17 PM, Maxime Devos wrote:

  * Patches must not be used to remove non-free files, because a patch by 
construction contains the non-free file itself so the patch would be non-free, 
which would not be acceptable to Guix. Likewise, patches should not be used to 
remove bundled libraries, to avoid large space usage, but this is not an 
absolute rule unlike as for non-free files.

It is possible to create patches that do not contain the deleted file, e.g. 
with `git format-patch --irreversible-delete`. That said, I don't know if the 
version of `patch` we use to patch origins is able to apply such patches—but 
maybe it would be a useful feature?

-Philip


Right, this is possible though it would have to be checked whether it is 
supported, so that statement should be weakened a bit -- to allow 
deleting non-free files with a patch, but noting that you'll have to set 
the right options to avoid including the deleted file in the patch.


I would recommend a (delete-file-recursively ".") over a patch here 
though, to avoid having to remember the --irreversible-delete option and 
in case there is not a git repo.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

Currently writing a v2 with a structure like Julien proposed.

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

Here's a v2. I've changed the structure to something close to what 
Julien proposed, it looks a lot better now to me!


The (^) should probably be tested before the final version.

I don't think the list of 'guiding principles' is worded well, probably 
needs more work.


[something I wrote previously]

Feel free to try to separate the things, but going previous 
discussions, many tings are important, and they appear all to be 
inseparable. 

Well seems like I was wrong, it splits nicely in three subsections!


I’d suggest starting with a patch against that section to address one
specific point that you think is the most pressing one.  From there we
can continue the discussion.
As written in another response, I don't really have an opinion on what's 
more pressing than another. I have written three 'points', but we don't 
have to discuss them all at once, maybe first 20.4.5.2? That one sounds 
relatively simple to me.


--- [start]

20.4.5 Snippets, phases and patches.

Snippets, phases and patches at times serve overlapping purposes. To 
decide between the three, there are a few guiding principles:


 * In principle, Guix only has free software; when the upstream source
   contains some non-free software, it has to be removed such that
   ‘guix build --source’ returns the "freed" source code rather than
   the unmodified upstream source (see: 28.4.1 Software Freedom).
 * The source of the package needs to correspond to what is actually
   built (i.e., act as the corresponding source), to fulfill our
   ethical and legal obligations.
 * It is convenient for the source derived from an origin to build on
   any system that the upstream package supports.
 * The source needs to actually work, not only on your Guix system but
   also for other systems; this requires some care for substitutions
   involving store items and other architecture-specific changes.
 * Sometimes, there is more than one way to do it. Let's go for the
   simplest one. Sometimes, which tool is the simplest, is subjective,
   that's fine too.

To make things more concrete and to resolve conflicts between the 
principles, a few cases have been worked out:


20.4.5.1 Removing non-free software.

Non-free software has to be removed in a snippet; the reason is that a 
patch or phase will not work.


For a patch, the problem is that a patch removing a non-free file 
automatically contains the non-free file (^), and we do not want 
anything non-free to appear in Guix even if only in its patches.


For a phase, the problem is that phases do not influence the result of 
‘guix build --source’.


(^) It has been noted that git patches support removing files without 
including the file in the patch in e-mail>. If it is verified that the 'patch' utility supports such 
patches, this method can be used and this policy adjusted appropriately.


20.4.5.2 Removing bundled libraries.

Bundled libraries should not be removed with a patch, because then the 
patch would contain the full bundled library, which can be large. They 
can be removed either in a snippet or a phase, often using the procedure 
'delete-file-recursively'. There are a few benefits for snippets here:


When using snippets, the bundled library does not occur in the source 
returned by ‘guix build --source’, so users and reviewers do not have to 
worry about whether the bundled library contains malware, whether it is 
non-free, if it contains pre-compiled binaries ... There are also less 
licensing concerns: if the bundled libraries are removed, it becomes 
less likely that the licensing conditions apply to people sharing the 
source returned by ‘guix build --source’, especially if the bundled 
library is not actually used on Guix systems. (*)


As such, snippets are recommended here.

(*) This is _not_ a claim that you can simply ignore the licenses of 
libraries when they are unbundled and replaced by Guix packages -- there 
are less concerns, not none.


20.4.5.3 Fixing technical issues (compilation errors, test failures, 
other bugs ...)


Usually, a bug fix comes in the form of a patch copied from upstream or 
another distribution. In that case, simply adding the patch to the 
'patches' field is the most convenient and usually does not cause any 
problems; there is no need to rewrite it as a snippet or a phase.


If no ready-made patch already exists, then choosing between a patch or 
a snippet is a matter of convenience. However, there are two things to 
keep in mind:


First, when the fix is not Guix-specific, it is strongly desired to 
upstream the fix to avoid the additional maintenance cost to Guix. As 
upstreams cannot accept a snippet, writing a patch can be a more 
efficient use of time. Secondly, if the fix of a technical issue embeds 
a store file name, then it has to be a phase. Otherwise, if a store file 
name was embedded in the source, the result of 'guix build --source' 
would be unusable on non-Guix systems and likely also unusable on Guix 
systems of another architectur

Re: GitLab to plans to delete dormant projects

[Maxime Devos]

On 06-08-2022 15:08, Olivier Dion via Development of GNU Guix and the 
GNU System distribution. wrote:

Hi,

Following this article, GitLab is
planning to start deleting project that were idle for > 12 months.

Many packages origin in Guix use an url to a GitLab project.  What are
the consequence of such deletion on Guix reproducibility?  Will it
affects the time-machine?


software heritage should avoid some problems, but from what I've heard 
it doesn't support all edge cases yet (something about recursive 
checkouts?).


I think it would be a good idea to make some Guile script to find all 
GitLab git checkouts in Guix and run the swh linter on them to make sure 
they are archived.


Additionally, it would be nice to support multiple URLs as fallbacks in 
the 'origin' record for git-fetch (like we have for url-fetch) to avoid 
the three points of failures (SWH and the copy at the substitute 
servers) in the fallback mechanism.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: FSDG issues of SCUMMVM-based games

[Maxime Devos]

On 06-08-2022 06:59, Liliana Marie Prikler wrote:


2. The "sources" consist of binaries that are installed as-is.


The credits of ScummVM state that the source code is available for 
Drascula[0]


[0] https://docs.scummvm.org/en/latest/help/credits.html: Emilio de Paz 
Aragón from Alcachofa Soft for sharing the source code of Drascula: The 
Vampire Strikes Back with us and his generosity with freewaring the game.


Maybe that only covered the engine and not the game bytecode though ...

I don't think bytecode falls under the 'non-functional data' exception 
of the FSDG, because code (the sprites, backgrounds and sound are 
another matter).


As such, I agree they should not be distributed (unless the actual 
sources are found, but even then there is (1.)), as per the first 
sentence of ‘(guix)GNU Distribution’.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Translating news on weblate?

[Maxime Devos]

On 07-08-2022 12:56, pelzflorian (Florian Pelz) wrote:

I was thinking we could have our news file translated at weblate, which
would help having more people translate it. Attached is a script that[…]

Translating etc/news.scm via Weblate adds delay but reaches more
translators.  Now delay is not so important when reading old news from
--list-generations or when one hasn’t upgraded in a while, but I would
prefer if those who make news, continue to request translations on some
mailing list (I forgot the name of the list alias; where is it?).  Even
though I was too slow to react to such mail in the past, I think I can
improve.


We could have both, I think?  After a new news entry, directly update 
the .po in the Guix repo, then after the patch is sent to guix-patches@ 
other people can open their favourite .po editor and add translations 
for their language and send that. No need to wait for Weblate, though 
late contributors can still translate with Weblate.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 08-08-2022 23:51, Andreas Enge wrote:

Hello,

Am Fri, Aug 05, 2022 at 03:59:14PM +0200 schrieb Maxime Devos:

Here's a v2. I've changed the structure to something close to what Julien
proposed, it looks a lot better now to me!

thanks, it does! I still find it a bit too verbose compared to Liliana's
suggestion, which I would prefer as a starting point of the discussion.


WDYM with 'still' here? The v2 patch I sent preceded Liliana's patch.

I'm not seeing a 'too verbose'-ity or a difference in verbosity myself

---

Something I liked about Julien's proposed structure is:


[...] derive rules for specific cases, based on these principles:

How do I remove non-free software? -> snippet because …

How do I remove bundled libraries? -> snippet or phase because …

How do I fix a build issue? -> patch or snippet if this affects 
building from source, can also be a phase if the result of --sources 
can still build


A test issue?

…

This leaves some cases up to interpretation, but that's probably not 
so different from "it's not an absolute rule". It's also much clearer 
and quicker to figure out in which case you are. If not documented as 
a case, you can fall back to the general principles.
-- i.e., if I want to do $FOO, I could quickly find out how to do it.  
In the v2 I sent, this was reflected in the subsections, each subsection 
is a 'howto $FOO'. Whereas the patch Liliana sent is kind of the inverse 
-- each @item corresponds to a 'what can the method $FOO be used for'. 
Similarly, in the v1 I sent, I followed a similar structure (an item for 
patches, an item for snippets, an item for phases).


As such, the v2 I sent seems a better basis to me.

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 05-08-2022 18:59, bl...@reproduciblemedia.com wrote:

Hi Maxime,

Adding some basic grammatical corrections below:
I have read several proposed corrections, but most of them don't appear 
to be grammatical corrections but rather stylistic or about 
(non-grammar) clarity or contents.

August 5, 2022 1:59 PM, "Maxime Devos"  wrote:


Technical grammatical correction: the software that Guix "has" is that in the 
monorepo,
but it "distributes" many packages. Thus:
--8<---cut here---start->8---
* In principle, Guix only distributes free software; when the upstream source 
contains some
non-free software, it should be removed such that ‘guix build --source’ returns the 
"freed"
source code rather than the unmodified upstream source (see: 28.4.1 Software 
Freedom).
--8<---cut here---end--->8---

I consider the difference between referring to external source code by 
including a (snippet-sanitised) copy or downloading it from an URL + 
snippet-sanitising to be immaterial, except for space and I/O savings, 
so I consider "has" to include "distributes".


While "distributes" is more specific, I really meant "has" here -- 
gnu/packages/patches/... and gnu/packages/*.scm must be free too, even 
if it is was not a distributed package (more concretely, see the bits 
about patches failing to remove non-freeness).



[...]
* The source of the package needs to correspond to what is actually built 
(i.e., act as the
corresponding source), to fulfill our ethical and legal obligations.

The [i.e.] addendum above is redundant, its better worded as:
--8<---cut here---start->8---
* The source of a package must correspond to what is actually built (i.e., 
there must be
an explicit relation between source code and the result of its build for all 
builds),
to fulfill our ethical and legal obligations.
--8<---cut here---end--->8---


You write that the addendum is redundant, but then change the addendum 
by replacing a word in the addendum by a possible definition. I'm not 
following how that reduces redundancy, and it also appears to be 
contrary to the lack of verbosity that Andreas would like.



* It is convenient for the source derived from an origin to build on any system 
that the upstream
package supports.
* The source needs to actually work, not only on your Guix system but also for 
other systems; this
requires some care for substitutions involving store items and other 
architecture-specific changes.

* Sometimes, there is more than one way to do it. Let's go for the simplest 
one. Sometimes, which
tool is the simplest, is subjective, that's fine too.

I think would be more clearly worded as:
--8<---cut here---start->8---
* When presented with a variety of strategies for defining a package, choose 
whichever is simplest.
Sometimes this is subjective, which is also fine. What matters is that you 
prefer techniques that
are common within the community (i.e. patterns that appear throughout 
gnu/packages/...) and
are thus clearly legible for reviewers.
--8<---cut here---end--->8---


To be clear, this is to replace the third point (Sometimes, there's more 
than one way to do it, etc.), without removing the previous two?


I would not use combinations like 'presented with a variety of 
strategies' however, I don't think that leads to clarity. Also, the 
preferences of 'you' are not all that important here, it's what they 
choose for that's important even if it is not their preference. (Though 
the two coinciding would be ideal of course!) Maybe:


When there is more than one way to do something, choose whichever method is the 
simplest.
Sometimes this is subjective, which is also fine. What matters is that you use 
techniques that
are common within the community (i.e. patterns that appear throughout 
gnu/packages/...) and
are thus clearly legible for reviewers.


To make things more concrete and to resolve conflicts between the principles, a 
few cases have been
worked out:

To a newcomer (the target audience), the above may lead to confusion as to what 
wasn't already
concrete in the above descriptions, or what principles above come into 
conflict. There is a mild,
latent assumption that they are familiar with the Guix workflow, which should 
be avoided. Thus I
suggest:
--8<---cut here---start->8---
For the purpose of clarifying preferred practices and reducing friction in the 
review process
introduced by subjective variation, a few guidelines have been worked out:
--8<---cut here---end--->8---


I don't see how a fancy wording amounting 

Re: [PATCH] doc: Update contribution guidelines on patches, etc.

[Maxime Devos]

On 06-08-2022 08:55, Liliana Marie Prikler wrote:


+If your package has a bug that takes multiple lines to fix,
I don't think this is true for replacing all instances of "foo" by 
"/gnu/store/.../bin/foo" in a file.

  or a fix
+has already been accepted upstream, patches are the preferred way of
+eliminating said bug
+Refer to the @code{origin} record documentation
+(particularly the fields @code{snippet} and @code{modules}), for more
+information (@pxref{origin Reference}).
+


The "Refer to the ... documentation for more information" occurred in 
the old version of (guix)Snippets versus Phases. However, back then, I 
did not find more information on how to decide between snippets, patches 
and phases, and neither do I now.


Maybe:

+Refer to the @code{origin} record documentation
+(@pxref{origin Reference}) (particularly the fields @code{snippet} and 
@code{modules})
+for more information on how to use snippets

, to avoid a reader's assumption that that section contains information 
on deciding between snippets, phases and patches.



+ Furthermore, as with patches, modifying the snippets causes two derivations 
to be built.


This is true, but I don't think reviewers and package authors have to 
worry about that.



Such changes include, but are not limited to fixes of the
+build script(s) or embeddings of store paths (e.g. replacement of
+@file{/bin/sh} with @code{(search-input-file inputs "bin/sh")}).

Include what? I think you need to close the subsentence here:


+Such changes include, but are not limited to, fixes of the build
+script(s) or embeddings of store paths (e.g. [...])



[...]


+Build phases are limited in that they do not modify the source
+derivation.  Thus, they are inadequate for changes that are to be
+reflected in the source code.  On the other hand, they only cause a
+single rebuild and are thus slightly easier to debug than phases and
+snippets.
Derivations are a rather low-level concept, could they be avoided in the 
origin and phases documentation?



+Build phases are limited in that they do not modify the source
+derivation.  Thus, they are inadequate for changes that are to be
+reflected in the source code.  On the other hand, they only cause a
+single rebuild and are thus slightly easier to debug than phases and
+snippets.

See Andreas' comment on phase->snippet.

Also, do I understand correctly that the argument here is that 'single 
rebuild -> less compilation time -> easier to debug'?


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [PATCH] doc: Update contribution guidelines on patches, etc.

[Maxime Devos]

Am Dienstag, dem 09.08.2022 um 18:45 +0200 schrieb Maxime Devos:

On 06-08-2022 08:55, Liliana Marie Prikler wrote:


+If your package has a bug that takes multiple lines to fix,

I don't think this is true for replacing all instances of "foo" by
"/gnu/store/.../bin/foo" in a file.

Should it?
I don't think so. Directly substituting all the instances instead of 
first writing a patch that does "foo" -> "@foo@" or such seems simpler 
to me.  This might be a bit too nit-picky though, maybe it's clear from 
context that this is not the kind of fix meant by that line.

+ Furthermore, as with patches, modifying the snippets causes two
derivations to be built.

This is true, but I don't think reviewers and package authors have to
worry about that.

It does make a difference to the author when debugging their package.
Starting with a phase and then moving it to a snippet can save good
time.
Hm, maybe, I guess I often work on 'small' packages where it doesn't 
matter much.


On 09-08-2022 19:05, Liliana Marie Prikler wrote:

+Such changes include, but are not limited to, fixes of the build
+script(s) or embeddings of store paths (e.g. [...])


[...]

Is that how to English comma?  Sorry, I'm not a native speaker so I get
somewhat weirded out by the when to skip/not to skip rules.

Neither am I. English doesn't seem to do "rules" much. I do think, 
however, that adding a comma after "to" makes things a bit simpler to 
read here, and it doesn't appear to be ungrammatical -- at least, in 
licenses "but is/are not limited to" is often used that way.



Derivations are a rather low-level concept, could they be avoided in
the origin and phases documentation?
I don't quite see how.  You could s/source derivation/the result of
@code{guix build -S}/, but I don't think that's much better.


To be clear, do you mean you:

 * think it's not better, maybe even worse
 * think it's not _much_ better (but still _slightly_ better)
 * are undecided
 * or something else

?

Also, "guix build -S" returns the source code (after snippet / patch, if 
any), not its derivation. For the latter: "guix build -S -d"



+Build phases are limited in that they do not modify the source
+derivation.  Thus, they are inadequate for changes that are to be
+reflected in the source code.  On the other hand, they only cause
a
+single rebuild and are thus slightly easier to debug than phases
and
+snippets.

See Andreas' comment on phase->snippet.

Also, do I understand correctly that the argument here is that
'single rebuild -> less compilation time -> easier to debug'?

Easier to debug for the package author currently fiddling with the
phase/snippet.  Not really a statement in any direction otherwise.
I don't see how "slightly easier to debug than phases" follows from 
"they cause only a single rebuild". My guess was that the intermediate 
step was lower compilation time, but apparently this was not the 
argument. As such, I'm not following.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [PATCH] doc: Update contribution guidelines on patches, etc.

[Maxime Devos]

On 09-08-2022 21:08, Liliana Marie Prikler wrote:

On 06-08-2022 08:55, Liliana Marie Prikler wrote:



+If your package has a bug that takes multiple lines to fix,

I don't think this is true for replacing all instances of "foo"
by
"/gnu/store/.../bin/foo" in a file.

Should it?

I don't think so. Directly substituting all the instances instead of
first writing a patch that does "foo" -> "@foo@" or such seems
simpler to me.  This might be a bit too nit-picky though, maybe it's
clear from context that this is not the kind of fix meant by that
line.

I'm struggling to see the issue here.  For starters, it doesn't take
multiple lines to embed a store path, the change usually happens on a
single line.  Of course, translating this into a substitute*, you
expand this single line into multiple ones, but that's not what is
meant here.


I was thinking of files that contain multiple instances of "foo" 
(usually on multiple lines) to be replaced by "/gnu/store/.../bin/foo", 
not files were a "foo
bar" was broken over multiple lines and it needs to be replaced by 
"/gnu/store/.../bin/foo

bar".

At least for the former, I don't think an intermediate "@foo@" is useful 
(with some exceptions, when a pattern match would catch too much).


The latter could in principle happen, but it doesn't seem to happen in 
practice.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [PATCH] doc: Update contribution guidelines on patches, etc.

[Maxime Devos]

On 09-08-2022 21:08, Liliana Marie Prikler wrote:

To be clear, do you mean you:
  * think it's not better, maybe even worse
  * think it's not_much_  better (but still_slightly_  better)
  * are undecided
  * or something else
?
Also, "guix build -S" returns the source code (after snippet / patch,
if any), not its derivation. For the latter: "guix build -S -d"

FWIW I don't think mentioning patch-and-repack is too helpful here
either.  Also, I'd like to use consistent wording at least within this
section, so here "source" means "upstream source" whereas "source
derivation" is a shorthand for the stuff Guix builds.  Yes, the
derivation is not the same thing as the output, but I again fail to see
how being overly precise is helpful.  That being said, I'm open to
suggestions.


I am not reading an answer to my question.

I don't think I've mentioned patch-and-repack (at least not by name, 
which you seem to be referring to?).


I would not recommend "source = upstream source", as the more general 
meaning is used in (guix)Introduction and 
 and elsewhere, otherwise 
terminology would become inconsistent, which can lead to misinterpretations.


I don't think there's such a thing as 'overly precise'.

My suggestion is the same as your suggestion:


You could s/source derivation/the result of
@code{guix build -S}/, but I don't think that's much better.

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 09-08-2022 20:58, david larsson wrote:

On 2022-08-05 15:59, Maxime Devos wrote:

[..]



20.4.5.3 Fixing technical issues (compilation errors, test failures,
other bugs ...)

Usually, a bug fix comes in the form of a patch copied from upstream
or another distribution. In that case, simply adding the patch to the
'patches' field is the most convenient and usually does not cause any
problems; there is no need to rewrite it as a snippet or a phase.

If no ready-made patch already exists, then choosing between a patch
or a snippet is a matter of convenience. However, there are two things
to keep in mind:

First, when the fix is not Guix-specific, it is strongly desired to
upstream the fix to avoid the additional maintenance cost to Guix. As
upstreams cannot accept a snippet, writing a patch can be a more
efficient use of time. Secondly, if the fix of a technical issue
embeds a store file name, then it has to be a phase. Otherwise, if a
store file name was embedded in the source, the result of 'guix build
--source' would be unusable on non-Guix systems and likely also
unusable on Guix systems of another architecture.


There may be other reasons to add patches: [...]


Agreed (*), but I don't think that subsection claims those are the only 
reasons for patches -- that section is only about fixing technical 
issues, not adding new features, as implied by the name of the section.


I can look at adding a new subsection 'Adding new functionality' for a v3.

Liliana's documentation contains some information not in my v2, I intend 
to look into integrating that information as well.




1. Functionality, that is not yet accepted upstream, because 
maintainer(s) do not have enough time to review all pull requests, or 
are simply slow to review. "if no response within X time from 
upstream, then guix may include your patch" might be a good policy here.


(*) We sometimes do such things already. Example: 
<https://issues.guix.gnu.org/49828> (nowadays upstreamed). I don't think 
this thread is a good place for deciding on the exact rules though -- 
deciding on an appropriate value of X seems difficult, I would like to 
separate that from the current documentation patch.


Greetings,
Maxime.


OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 10-08-2022 08:10, bl...@reproduciblemedia.com wrote:

August 5, 2022 1:59 PM, "Maxime Devos"  wrote:
Technical grammatical correction: the software that Guix "has" is that in the 
monorepo,
but it "distributes" many packages. Thus:
--8<---cut here---start->8---
* In principle, Guix only distributes free software; when the upstream source 
contains some
non-free software, it should be removed such that ‘guix build --source’ returns the 
"freed"
source code rather than the unmodified upstream source (see: 28.4.1 Software 
Freedom).
--8<---cut here---end--->8---

I consider the difference between referring to external source code by 
including a
(snippet-sanitised) copy or downloading it from an URL + snippet-sanitising to 
be immaterial,
except for space and I/O savings, so I consider "has" to include "distributes".

While "distributes" is more specific, I really meant "has" here -- 
gnu/packages/patches/... and
gnu/packages/*.scm must be free too, even if it is was not a distributed 
package (more concretely,
see the bits about patches failing to remove non-freeness).

[...]

This is simply a grammatical error. "Has" is third person singular. While its 
common to speak
in such a way, it's not proper English, and including minor slang should be 
avoided in technical
writing. Otherwise inconsistency of presentation is guaranteed, causing serious 
overhead for
readers.


"Has" is third person singular, and "Guix" is singular and not "I" or 
"you", so "has" seems appropriate here to me.


I think that matching the singular/plural of the verb and the subject is 
common, standard and proper English and not slang.


Going by previous replies, you seem to know English well, so I think 
there's some miscommunication here.


(Even better would be to replace the rather generic ‘to have’ by 
something more specific, but I'll have to think a bit about what would 
be more specific yet not overly specific.)





* The source of the package needs to correspond to what is actually built 
(i.e., act as the
corresponding source), to fulfill our ethical and legal obligations.

The [i.e.] addendum above is redundant, its better worded as:
--8<---cut here---start->8---
* The source of a package must correspond to what is actually built (i.e., 
there must be
an explicit relation between source code and the result of its build for all 
builds),
to fulfill our ethical and legal obligations.
--8<---cut here---end--->8---

You write that the addendum is redundant, but then change the addendum by 
replacing a word in the
addendum by a possible definition. I'm not following how that reduces 
redundancy, and it also
appears to be contrary to the lack of verbosity that Andreas would like.

Redundancy in language is information expressed more than once. Including 
redundant clauses
is bad grammar[1]

You wrote:

The source of the package needs to correspond to what is actually built (i.e., 
act as the
corresponding source)

You simply said the same thing twice. It is by definition, a redundant clause.


then change the addendum by replacing a word in the
addendum by a possible definition.

Elaboration doesnt necessarily add redunancy, it is useful for clarifying 
statements. I was
trying to infer your intention in adding the clause, to offer an example of how 
it could be
more clearly stated.

[1] https://en.wikipedia.org/wiki/Redundancy_(linguistics)


The purpose of 'i.e.' constructs is to state the same thing differently, 
to clarify matters (by elaboration or by redundancy). I don't see how 
redundancy is bad, though it can easily be overdone.


I think that explicitly mentioning the term 'corresponding source' 
instead of only the more implicit 'source of X corresponds to Y', 
because the former 'corresponding source' has a very specific meaning 
(*) in free software, whereas the latter construct is more ambiguous.


Compare with your definition 'there must be an explicit relation ...’, 
which loses a lot of nuance.


(*) E.g., the GPL has a long and detailed definition: ‘The 
"Corresponding Source" for a work in object code form means all the 
source code needed to generate, install, [lots and lots of text]’.


I can look into inserting a footnote linking to the GPL or copyleft.org 
or such, to make clear "corresponding source" is a term on its own and 
not just some description, for people that don't already know about 
"corresponding source".





To make things more concrete and to resolve conflicts between the principles, a 
few cases have been
worked out:

To a newcomer (the target audience), the above may lead to c

Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.

[Maxime Devos]

On 10-08-2022 12:33, bl...@reproduciblemedia.com wrote:


None of the edits I made were in criticism. In an academic context a proof 
reader would mark
almost all the same sentences for revision.


As far as I'm aware, I do not have implied the contrary anywhere previously.

I do think they are criticism though -- more precisely, constructive 
criticism. There might be a difference in terminology here, e.g. 
according to Wikipedia and its source, some languages make a distinction 
between critique and criticism


To be clear, I consider your criticism to be a good thing, I just happen 
to disagree on some points.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: [POSTMORTEM] Subkey is not authorized by .guix-authorizations

[Maxime Devos]

On 11-08-2022 16:26, Andrew Tropin wrote:

* What to do after?
- Accept subkey on guix pull if master key is in .guix-authorizations.


As I've now written on 57091, this would cause security problems with 
old or revoked keys.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: bug#55026: potential prebuilt binaries in the Mono package

[Maxime Devos]

On 12-08-2022 22:03, Maxim Cournoyer wrote:

Hi,

zamfofex  writes:


It seems the package for Mono in Guix uses a tarball that contains a
lot of prebuilt DLLs. This doesn’t seem to have been mentioned when
the package was introduced. Could it have been a mistake? Some related
discussion:  and also


If nobody volunteers to do the work necessary to get our mono package to
build without prebuilt binaries in the next 2 weeks, I suggest we remove
it from our collection.  Only a handful package depend on it, most of
them optionally it seems, and the mono we carry is severely outdated
(2016).

What do you think?

Maxim

I hope someone does the work, but if not, I don't see a problem with 
removing it.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: Who owns guix-mirror?

[Maxime Devos]

On 16-08-2022 14:59, Philip Beadling wrote:

Hi all,

http://github.com/guix-mirror is a mirror for the Guix source code.

I was wondering who owns the mirror as there is no public ownership 
declared on github.com?


Is it owned by Guix or is it run by a trusted third party, etc?


It's unknown who runs it, but it's not Guix, see  
.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: (ice-9 base64)?

[Maxime Devos]

On 16-08-2022 19:21, Aleix Conchillo Flaqué wrote:



On Tue, Aug 16, 2022 at 9:59 AM Maxime Devos  
wrote:



On 16-08-2022 18:10, Aleix Conchillo Flaqué wrote:

Hi,

In many projects I've been copying Göran Weinholt's base64
implementation and I've also seen it in other projects, would it
make sense to include it in Guile's standard library? [...]


If we do this, we should contact the various other projects to
make them use (ice-9 base64).


I think they could switch whenever they want (i.e. whenever this was 
added to Guile) or even not switch at all.


Sure, but they can't switch if they don't know about it. And if they 
don't know about it and hence don't switch, the proposal fails at its 
purpose of unbundling base64. Besides, we need them to switch (see Guix 
no-bundling policy and the reasons behind it) -- if upstream refuses to 
unbundle, then in our locally modified version for Guix.



I think it would be simpler though to consider the base64 in
guile-gcrypt to be 'canonical', it would avoid problems with old
versions of Guile not having the base64 module and newer version
having it, which would prevent using the proposed (ice-9 base64)
in Guile because it would break build-aux/build-self.scm when
pulling or time-machining from old Guix that have an old Guile.


I've been waiting on a guile-gcrypt release for a while now (Ludo, 
Chrisitine... any help here? :-) ).  I ported guile-jwt to use 
guile-gcrypt but I need a release to have latest base64 changes:


https://notabug.org/cwebber/guile-gcrypt/commit/f8934ec94df5868ee8baf1fb0f8ed0f24e7e91eb

Right, it has some fixes that are presumably important.


But you are right that this would cause a backward compatible problem, 
but I guess that would depend on each project. Can we do conditional 
module loading? I've done this in the past with Python... if we are in 
Python 2 load this module, otherwise load this other one. So projects 
could do that.
Yes, using resolve-module with #:ensure #f & module-ref. Or with syntax 
tricks and (version), to decide things at compile-time. Still, if you do 
a conditional module loading, you still need a fallback, and the 
fallback would still be bundling.


Whether we simply replace (guix base64) by (gcrypt base64) depends
on how old (gcrypt base64) is compared to the earliest 'supported'
Guix for pull/time-travel, but even if it is not present in the
old gcrypt, we can work-around that (we have a 'fake-gcrypt-hash'
in build-aux/build-self.scm, so we can easily have a (define
gcrypt-base64 [some copy])).  Or simply update the local
guile-gcrypt in buid-aux/build-self.scm.

guile-gcrypt base64 is pretty new with the patch above (but no release 
after that), I have no idea if Guix has added anything else.


base64 is available in at least 0.3.0, which is packaged in Debian 
bullseye (which is considered "stable"), so not too new, though we might 
need to change build-aux/build-self.scm if 0.1.0 doesn't have base64.  
Guix appears to have the pre-quoted-patch version, without changes of 
its own except for a different module name.



OTOH a similar replacement can be done for (ice-9 base64), but
transitioning to (ice-9 base64) would take much longer, at least
until the various distributions are updated to a Guile that has
(ice-9 base64), whereas (gcrypt base64) could be switched to
immediately.

Maybe this could be handled by each project independently.


They wouldn't have to if the base64 module is put in (guile gcrypt).

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: (ice-9 base64)?

[Maxime Devos]

On 17-08-2022 18:22, Aleix Conchillo Flaqué wrote:

Hi Maxime!

On Tue, Aug 16, 2022 at 12:04 PM Maxime Devos  
wrote:



On 16-08-2022 19:21, Aleix Conchillo Flaqué wrote:



On Tue, Aug 16, 2022 at 9:59 AM Maxime Devos
 wrote:


On 16-08-2022 18:10, Aleix Conchillo Flaqué wrote:

Hi,

In many projects I've been copying Göran Weinholt's base64
implementation and I've also seen it in other projects,
would it make sense to include it in Guile's standard
library? [...]


If we do this, we should contact the various other projects
to make them use (ice-9 base64).


I think they could switch whenever they want (i.e. whenever this
was added to Guile) or even not switch at all.


Sure, but they can't switch if they don't know about it. And if
they don't know about it and hence don't switch, the proposal
fails at its purpose of unbundling base64. Besides, we need them
to switch (see Guix no-bundling policy and the reasons behind it)
-- if upstream refuses to unbundle, then in our locally modified
version for Guix.


Forgive my ignorance, but what do you mean by unbundling? I'm not 
familiar with Guix at all, well, just conceptually and for trying a 
few commands years ago.


Sometimes the source code of a package contains a copy of a dependency. 
This is called 'bundling'. 'Unbundling' is the act of undoing the 
'bundling', this is often done by cleaning up the source code (with what 
we call a 'snippet' in Guix: (snippet #~(delete-file-recursively 
"googletest"))) and setting some configuration flags 
("-DUSE_SYSTEM_GOOGLETEST=yes" or such).


For example, in Guix we occasionally encounter a bundled "googletest" (a 
test framework).


In this case, we are kind of (un)bundling the base64 module, though it's 
not _exactly_ (un)bundling because, AFAIK, there is canonical upstream 
location for the base64 module to replace things with. Still seems 
pretty close to me.


Upsides of unbundling, as mentioned in '(guix)Submitting Patches':


 Sometimes, packages include copies of the source code of their
 dependencies as a convenience for users.  However, as a
 distribution, we want to make sure that such packages end up using
 the copy we already have in the distribution, if there is one.
 This improves resource usage (the dependency is built and stored
 only once), and allows the distribution to make transverse changes
 such as applying security updates for a given software package in a
 single place and have them affect the whole system—something that
 bundled copies prevent.
Another benefit: reviewing for absence of malware is less work when 
there's only a single copy to review, though I suppose that in this case 
the module is so small the reviewing benefit is minimal.



Whether we simply replace (guix base64) by (gcrypt base64)
depends on how old (gcrypt base64) is compared to the
earliest 'supported' Guix for pull/time-travel, but even if
it is not present in the old gcrypt, we can work-around that
(we have a 'fake-gcrypt-hash' in build-aux/build-self.scm, so
we can easily have a (define gcrypt-base64 [some copy])). Or
simply update the local guile-gcrypt in buid-aux/build-self.scm.

guile-gcrypt base64 is pretty new with the patch above (but no
release after that), I have no idea if Guix has added anything else.


base64 is available in at least 0.3.0, which is packaged in Debian
bullseye (which is considered "stable"), so not too new, though we
might need to change build-aux/build-self.scm if 0.1.0 doesn't
have base64.  Guix appears to have the pre-quoted-patch version,
without changes of its own except for a different module name.


One more time, forgive me, but what is build-aux/build-self.scm?
It's an implementation detail of Guix, it's a file (from the new 
version, not the old) that is loaded by "guix pull" in the old Guix to 
compile the new version of Guix.



OTOH a similar replacement can be done for (ice-9 base64),
but transitioning to (ice-9 base64) would take much longer,
at least until the various distributions are updated to a
Guile that has (ice-9 base64), whereas (gcrypt base64) could
be switched to immediately.

Maybe this could be handled by each project independently.


They wouldn't have to if the base64 module is put in (guile gcrypt).


And the last forgiveness... (guile gcrypt)?


Oops, that should have been guile-gcrypt -- it's a Guile package -- 
"guix show guile-gcrypt" / <https://notabug.org/cwebber/guile-gcrypt>.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: (ice-9 base64)?

[Maxime Devos]

Then, if I understood correctly, IMO I would say Guile should not 
really care about Guix's bundling/unbundling. That is, adding (ice-9 
base64) (or however we want to call it... maybe (encoding base64) 
following Golang and Guile's (web ) module) should be totally 
independent of Guix. So, if we add (ice-9 base64) to Guile then Guix 
should figure out what to do with it, but it's Guix's concern not Guile's.


It's not some Guix-specific quirk. It's the same for at least Debian. It 
benefits not only Guix itself but all users of the software:



[...] allows [...] to make transverse changes
such as applying security updates for a given software package in a
single place and have them affect the whole system—something that
bundled copies prevent.
... that was written with Guix in mind, but it applies to every 
distribution and everyone.


Besides, your goal appears to be to unbundle the base64 into a single 
location (as a module of Guile), if we do that I think we should go all 
the way -- just adding it to Guile increases bundling instead of 
decreasing bundling, only if the various upstreams are modified to 
unbundle and use the new location then the unbundling is completed.


On 18-08-2022 02:09, Aleix Conchillo Flaqué wrote:
About Guix's unbundling (maybe that's something that should go on 
Guix's mailing list),
I don't see why, there's nothing to write about except "oops some 
packages are bundling base64, let's unbundle those", and for unbundling 
those, it seems more practical to write about that here on guile-devel. 
Also I noticed I sent some messages to guix-devel instead of 
guile-devel, correcting now.
I don't think currently there's any unbundling for base64 modules or 
at least not in a package I maintain guile-jwt (guile-jwt bundles 
base64). And probably there's no unbundling because there's no 
canonical implementation? Even if there was a canonical 
implementation, how would that look like in Guix's guile-jwt package? 
What would the snippet actually do?
Currently, it's not done yet, presumably for that reason and maybe also 
due to nobody having noticed it yet?


How it would look like, for upstreams that refuse to unbundle or are 
unresponsive:


#~(begin
    (delete-file "local/copy/of/base64.scm")
    [also remove it from the Makefile.am]
    (substitute* (find-files "." "\\.scm$")
  (("(\\local base64 module\\)") "(gcrypt base64)")))

For responsive upstreams that do not mind these kind of improvements, 
there is a preference for submitting a patch upstream -- that way, 
everyone benefits, not only Guix.


Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature

Re: (ice-9 base64)?

[Maxime Devos]

On 19-08-2022 02:20, Aleix Conchillo Flaqué wrote:

So, what do you think would be the way to proceed in order to include 
a base64 implementation in Guile itself?


For example:

1. Add (ice-9 base64) (or (encoding base64)) to Guile and let new 
projects and existing projects to update with conditional module 
loading to support old versions of Guile.
2. Do unbundling in Guix packages both for projects that have not 
updated upstream and for projects in (1). The unbundling would be done 
by pointing to Guix's (or guile-gcrypt) base64 implementation, or is 
there a way they could point to Guile's implementation?


If the canonical location of base64 becomes Guile itself instead of 
guile-gcrypt, then it needs to be pointed at Guile's base64. Likewise, 
Guix' base64 implementation would need to be replaced by Guile's, with a 
fallback.


I don't see why we would point to Guix' implementation, it's missing 
some bug fixes.




Does that make sense or am I still missing something (I'm about to 
catch a cold so my brain is not working quite well this week)? 
Originally, I was thinking only in (1).


Except for the remark about (1), I think so.  I think the following list 
is a bit more clear though:


1. Add (ice-9 base64) to Guile (or another name (encoding base64)).
2. Inform a few upstreams that used to include a copy of base64 that it
   is now part of Guile itself -- those upstreams can then remove their
   local copy and use Guile's base64, and do conditional module loading
   if they cannot increase their minimal Guile version yet.
3. In Guix, we will have to update Guile to a new version that has the
   base64 module and remove the local fallback copies.  And if upstream
   refuses patches to use Guile's base64 (maybe with a fallback), then
   it will need to be patched locally in Guix.

On (2): I don't think it's necessary to contact _all_ the upstreams, 
though to give a good example it would be nice to contact some of them.


(3) is a Guix concern, not really a Guile concern.

Greetings,
Maxime.



OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key


OpenPGP_signature
Description: OpenPGP digital signature