Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
On 05-08-2022 05:38, Philip McGrath wrote: On Sun, Jul 24, 2022, at 11:17 PM, Maxime Devos wrote: * In principle, you can apply a patch from a phase. However, this causes the result of "guix build --source" to not correspond to the actual source code anymore (i.e., it doesn't act as corresponding source anymore), so consider this a last resort for situations such as avoiding causing a world-rebuild for a patch fixing a target-specific bug by making the patching conditional upon target-foo?. If you apply a patch from a phase, make sure that the patch appears in the inputs or native-inputs, such that "guix build --source=all" will include the patch. Should we have an option for "guix build --source=all" to also include the Guix "scripts used to control compilation and installation"? What do you mean with "Guix scripts" here? I don't think Guix has a notion of 'scripts' (except stuff like 'wrap-script', but that doesn't seem relevant here). Do you mean the phases code? Greetings, Maxime. OpenPGP_0x49E3EE22191725EE.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
On 05-08-2022 05:23, Philip McGrath wrote: On Sun, Jul 24, 2022, at 11:17 PM, Maxime Devos wrote: * Patches must not be used to remove non-free files, because a patch by construction contains the non-free file itself so the patch would be non-free, which would not be acceptable to Guix. Likewise, patches should not be used to remove bundled libraries, to avoid large space usage, but this is not an absolute rule unlike as for non-free files. It is possible to create patches that do not contain the deleted file, e.g. with `git format-patch --irreversible-delete`. That said, I don't know if the version of `patch` we use to patch origins is able to apply such patches—but maybe it would be a useful feature? -Philip Right, this is possible though it would have to be checked whether it is supported, so that statement should be weakened a bit -- to allow deleting non-free files with a patch, but noting that you'll have to set the right options to avoid including the deleted file in the patch. I would recommend a (delete-file-recursively ".") over a patch here though, to avoid having to remember the --irreversible-delete option and in case there is not a git repo. Greetings, Maxime. OpenPGP_0x49E3EE22191725EE.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Re: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
Currently writing a v2 with a structure like Julien proposed. Greetings, Maxime. OpenPGP_0x49E3EE22191725EE.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
A Few Packaging Questions
Hello All,
1. Is there a way to call non-exported procedures from a module in
Guile?
I am looking to solve the non-determinism in one of the above-submitted
packages (dbqn), and per https://github.com/dzaima/BQN/issues/14 it
seems the issue is timestamps in jarfiles, which ant-build-system
handles in its strip-jar-timestamps procedure. I'd like to call that
procedure from that module rather than copy it into the bqn.scm module,
but I can't build the package with ant, so I'm looking for a way to call
it without switching systems.
2. Is there a list of variables accessible using gexps somewhere, or
some source code I might be able to glean the same information from?
I have just discovered #$output as a somewhat useful tool, and I'm
currently looking to learn more ahead of when I might need them.
3. What is the canonical way in the new input system to reference a
specific output of a package?
I first learned `("label" ,package "output"), and I've seen `(,package
"output") in the devel version of the manual, but when I recently did a
guix lint on some package definitions that used `(,openjdk "jdk") I got
a warning:
label 'openjdk' does not match package name 'openjdk:jdk'
I want to make sure I am doing things the "most correct" way, so I
figured I'd ask here.
Thank You for taking the time to read my questions, and I hope You have
a great day!
--
Christopher Rodriguez
signature.asc
Description: PGP signature
Re: A real-life test of long-term reproducibility
Hi Konrad, Just one quick comment: August 4, 2022 8:43 AM, "Konrad Hinsen" wrote: > Hi everyone, > > One of our claims is that Guix can rebuild code identically as long as > we have a machine with a Linux kernel and a POSIX filesystem. This actually isn't the claim. Reproducibility is only guaranteed on Guix systems. It's important that you reproduce the entire system, which means you will have needed to have saved the commit of your version of the Guix package manager in order to return to that system. See the section of the Guix manual "Replicating Guix" for more info. https://guix.gnu.org/en/manual/devel/en/html_node/Replicating-Guix.html Happy hacking! > This week > I had an occasion to put this to a real-life test. So far it's a > failure. I can guess reasons for my failed attempts, but I don't think > they were unreasonable to try. So I'd like to document something that > works, to avoid others falling into the same traps. I just don't know > yet what the Right Way To Do It is! > > The package I want to rebuild and use is "nmoldyn" from Guix commit > f250a868d8c687df08559682fa68fb4ea2a1ea69. That's the commit referenced > in my notes, obtained via "guix describe" in early 2018. I am pretty > sure it worked fine back then. > > First attempt: > > $ guix time-machine --commit=f250a868d8c687df08559682fa68fb4ea2a1ea69 -- > build nmoldyn > Updating channel 'guix' from Git repository at > 'https://git.savannah.gnu.org/git/guix.git'... > Backtrace: > In guix/store.scm: > 672:3 19 (_) > In ice-9/boot-9.scm: > 1752:10 18 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _) > In guix/store.scm: > 659:37 17 (thunk) > In guix/status.scm: > 815:4 16 (call-with-status-report _ _) > In guix/store.scm: > 1298:8 15 (call-with-build-handler # guix/ui.scm:1…> …) > 2168:25 14 (run-with-store # _ # _ # _ > # _) > In guix/inferior.scm: > 903:8 13 (_ _) > In guix/channels.scm: > 944:2 12 (_ _) > 891:2 11 (_ _) > In ./guix/monads.scm: > 487:9 10 (_ _) > In guix/store.scm: > 1996:8 9 (_ _) > In guix/channels.scm: > 642:36 8 (_ #) > 703:11 7 (_) > In ice-9/eval.scm: > 619:8 6 (_ #(#(#(#) "/gnu/store/…" …) …)) > 626:19 5 (_ #(#(#(#) "/gnu/store/…" …) > …)) > 155:9 4 (_ #(#(#(#) "/gnu/store/…" …) …)) > 223:20 3 (proc #(#(#(#) "/gnu/sto…" …) > …)) > In unknown file: > 2 (%resolve-variable (7 . %guix-register-program) #) > In ice-9/boot-9.scm: > 1685:16 1 (raise-exception _ #:continuable? _) > 1685:16 0 (raise-exception _ #:continuable? _) > > ice-9/boot-9.scm:1685:16: In procedure raise-exception: > error: %guix-register-program: unbound variable > > I don't understand what is going wrong here, but it may be related to > the fact that the commit I am trying to go back to is older than "guix > time-machine". If that's the explanation, it would help if Guix showed > some clear error message instead of crashing. > > Next I tried checking out the source code for commit > f250a868d8c687df08559682fa68fb4ea2a1ea69, and building it from > source. This is a bit tricky because 2018 Guix cannot be built in > today's Guix build environment. For example, today we have Guile 3, but > back then we had Guile 2.2. So I need to do "guix environment guix" in > an older Guix, before the Guile 3 transition, but later than the > introduction of time-machine. I picked one somewhat at random: > > $ guix time-machine --commit=e2293cbbe0cd20ddeb932e6f5616565ab468c087 > -- environment –pure guix > > Then I did "bootstrap", "configure –localstatedir=/var", "make > check". The latter shows 15 failures, some of which look important: > > FAIL: tests/builders.scm > FAIL: tests/derivations.scm > FAIL: tests/packages.scm > FAIL: tests/guix-environment.sh > FAIL: tests/guix-daemon.sh > > And indeed I cannot build much with my compiled guix: > > $ ./pre-inst-env guix build nmoldyn > > hangs after a while, running a binary called "test-lock" for hours. > > Given the time lapse, I suppose there have been incompatible changes in > the build daemon, making the old Guix incompatible with the rather > recent build daemon running on my machine. But is there a way around > this, other than installing an old Guix in a fully isolated VM? > > And if installing the old Guix in a VM is the only solution, what would > be the best way to do that? I can't think of much else than starting > from another distribution (e.g. Debian) and following the installation > instructions. That's already a lot of work, but it's made worse by the > installation instructions being hidden inside the manual of that old > commit, which I cannot easily consult. > > I'd be grateful for any suggestions! > > Cheers, > Konrad
v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
Here's a v2. I've changed the structure to something close to what Julien proposed, it looks a lot better now to me! The (^) should probably be tested before the final version. I don't think the list of 'guiding principles' is worded well, probably needs more work. [something I wrote previously] Feel free to try to separate the things, but going previous discussions, many tings are important, and they appear all to be inseparable. Well seems like I was wrong, it splits nicely in three subsections! I’d suggest starting with a patch against that section to address one specific point that you think is the most pressing one. From there we can continue the discussion. As written in another response, I don't really have an opinion on what's more pressing than another. I have written three 'points', but we don't have to discuss them all at once, maybe first 20.4.5.2? That one sounds relatively simple to me. --- [start] 20.4.5 Snippets, phases and patches. Snippets, phases and patches at times serve overlapping purposes. To decide between the three, there are a few guiding principles: * In principle, Guix only has free software; when the upstream source contains some non-free software, it has to be removed such that ‘guix build --source’ returns the "freed" source code rather than the unmodified upstream source (see: 28.4.1 Software Freedom). * The source of the package needs to correspond to what is actually built (i.e., act as the corresponding source), to fulfill our ethical and legal obligations. * It is convenient for the source derived from an origin to build on any system that the upstream package supports. * The source needs to actually work, not only on your Guix system but also for other systems; this requires some care for substitutions involving store items and other architecture-specific changes. * Sometimes, there is more than one way to do it. Let's go for the simplest one. Sometimes, which tool is the simplest, is subjective, that's fine too. To make things more concrete and to resolve conflicts between the principles, a few cases have been worked out: 20.4.5.1 Removing non-free software. Non-free software has to be removed in a snippet; the reason is that a patch or phase will not work. For a patch, the problem is that a patch removing a non-free file automatically contains the non-free file (^), and we do not want anything non-free to appear in Guix even if only in its patches. For a phase, the problem is that phases do not influence the result of ‘guix build --source’. (^) It has been noted that git patches support removing files without including the file in the patch in e-mail>. If it is verified that the 'patch' utility supports such patches, this method can be used and this policy adjusted appropriately. 20.4.5.2 Removing bundled libraries. Bundled libraries should not be removed with a patch, because then the patch would contain the full bundled library, which can be large. They can be removed either in a snippet or a phase, often using the procedure 'delete-file-recursively'. There are a few benefits for snippets here: When using snippets, the bundled library does not occur in the source returned by ‘guix build --source’, so users and reviewers do not have to worry about whether the bundled library contains malware, whether it is non-free, if it contains pre-compiled binaries ... There are also less licensing concerns: if the bundled libraries are removed, it becomes less likely that the licensing conditions apply to people sharing the source returned by ‘guix build --source’, especially if the bundled library is not actually used on Guix systems. (*) As such, snippets are recommended here. (*) This is _not_ a claim that you can simply ignore the licenses of libraries when they are unbundled and replaced by Guix packages -- there are less concerns, not none. 20.4.5.3 Fixing technical issues (compilation errors, test failures, other bugs ...) Usually, a bug fix comes in the form of a patch copied from upstream or another distribution. In that case, simply adding the patch to the 'patches' field is the most convenient and usually does not cause any problems; there is no need to rewrite it as a snippet or a phase. If no ready-made patch already exists, then choosing between a patch or a snippet is a matter of convenience. However, there are two things to keep in mind: First, when the fix is not Guix-specific, it is strongly desired to upstream the fix to avoid the additional maintenance cost to Guix. As upstreams cannot accept a snippet, writing a patch can be a more efficient use of time. Secondly, if the fix of a technical issue embeds a store file name, then it has to be a phase. Otherwise, if a store file name was embedded in the source, the result of 'guix build --source' would be unusable on non-Guix systems and likely also unusable on Guix systems of another architectur
Welcome to our new committer!
The Guix Maintainer Collective (tm) would like to welcome Andrew Tropin, aka abcdw, as our newest committer! I'm sure many of you recognize them from their work on Guix Home and their regular videos hacking on Guix, among other things. So join us in welcoming them! -- Efraim Flashner אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted signature.asc Description: PGP signature
Re: Welcome to our new committer!
Welcome! Efraim Flashner ezt írta (időpont: 2022. aug. 5., P, 17:59): > The Guix Maintainer Collective (tm) would like to welcome Andrew Tropin, > aka abcdw, as our newest committer! I'm sure many of you recognize them > from their work on Guix Home and their regular videos hacking on Guix, > among other things. > > So join us in welcoming them! > > -- > Efraim Flashner אפרים פלשנר > GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 > Confidentiality cannot be guaranteed on emails sent or received unencrypted >
Re: A Few Packaging Questions
Hi! Christopher Rodriguez writes: > [[PGP Signed Part:Undecided]] > Hello All, > > 1. Is there a way to call non-exported procedures from a module in > Guile? There is one but you should not use it. Instead you should send a patch that adds an export to the previously private declaration. And the way is to use the @@ function, but code that relies on it is AFAIK not acceptable in Guix. > 2. Is there a list of variables accessible using gexps somewhere, or > some source code I might be able to glean the same information from? > > I have just discovered #$output as a somewhat useful tool, and I'm > currently looking to learn more ahead of when I might need them. The info pages should have more, well, info. Easiest way to find them IMHO is through Emacs. There is also an HTML mirror: https://guix.gnu.org/en/manual/devel/en/html_node/G_002dExpressions.html#G_002dExpressions > 3. What is the canonical way in the new input system to reference a > specific output of a package? According to the URL above, it's #$output[:output]. I hope that helped. Happy packaging. UwU
Re: Welcome to our new committer!
On 2022-08-05 18:59, Efraim Flashner wrote: > The Guix Maintainer Collective (tm) would like to welcome Andrew Tropin, > aka abcdw, as our newest committer! I'm sure many of you recognize them > from their work on Guix Home and their regular videos hacking on Guix, > among other things. > > So join us in welcoming them! Hi everyone! You can obtain my GPG key with: gpg --locate-keys and...@trop.in Or from savannah: https://savannah.gnu.org/users/abcdw Don't hesitate to CC me to interesting threads or ping me once in a while! Have a good weekend, see you in a bit! :) -- Best regards, Andrew Tropin signature.asc Description: PGP signature
Re: v2: A proposal of a consistent set of clear rules and guidelines involving snippets, phases and patches.
Hi Maxime, Adding some basic grammatical corrections below: August 5, 2022 1:59 PM, "Maxime Devos" wrote: > Here's a v2. I've changed the structure to something close to what Julien > proposed, it looks a lot > better now to me! > > The (^) should probably be tested before the final version. > > I don't think the list of 'guiding principles' is worded well, probably needs > more work. > > [something I wrote previously] > > Feel free to try to separate the things, but going previous discussions, many > tings are important, > and they appear all to be inseparable. > Well seems like I was wrong, it splits nicely in three subsections! >> I’d suggest starting with a patch against that section to address one >> specific point that you think is the most pressing one. From there we >> can continue the discussion. > > As written in another response, I don't really have an opinion on what's more > pressing than > another. I have written three 'points', but we don't have to discuss them all > at once, maybe first > 20.4.5.2? That one sounds relatively simple to me.--- [start] > > 20.4.5 Snippets, phases and patches. > > Snippets, phases and patches at times serve overlapping purposes. To decide > between the three, > there are a few guiding principles: > > * In principle, Guix only has free software; when the upstream source > contains some non-free > software, it has to be removed such that ‘guix build --source’ returns the > "freed" source code > rather than the unmodified upstream source (see: 28.4.1 Software Freedom). Technical grammatical correction: the software that Guix "has" is that in the monorepo, but it "distributes" many packages. Thus: --8<---cut here---start->8--- * In principle, Guix only distributes free software; when the upstream source contains some non-free software, it should be removed such that ‘guix build --source’ returns the "freed" source code rather than the unmodified upstream source (see: 28.4.1 Software Freedom). --8<---cut here---end--->8--- > * The source of the package needs to correspond to what is actually built > (i.e., act as the > corresponding source), to fulfill our ethical and legal obligations. The [i.e.] addendum above is redundant, its better worded as: --8<---cut here---start->8--- * The source of a package must correspond to what is actually built (i.e., there must be an explicit relation between source code and the result of its build for all builds), to fulfill our ethical and legal obligations. --8<---cut here---end--->8--- > * It is convenient for the source derived from an origin to build on any > system that the upstream > package supports. > * The source needs to actually work, not only on your Guix system but also > for other systems; this > requires some care for substitutions involving store items and other > architecture-specific changes. > > * Sometimes, there is more than one way to do it. Let's go for the simplest > one. Sometimes, which > tool is the simplest, is subjective, that's fine too. I think would be more clearly worded as: --8<---cut here---start->8--- * When presented with a variety of strategies for defining a package, choose whichever is simplest. Sometimes this is subjective, which is also fine. What matters is that you prefer techniques that are common within the community (i.e. patterns that appear throughout gnu/packages/...) and are thus clearly legible for reviewers. --8<---cut here---end--->8--- > To make things more concrete and to resolve conflicts between the principles, > a few cases have been > worked out: To a newcomer (the target audience), the above may lead to confusion as to what wasn't already concrete in the above descriptions, or what principles above come into conflict. There is a mild, latent assumption that they are familiar with the Guix workflow, which should be avoided. Thus I suggest: --8<---cut here---start->8--- For the purpose of clarifying preferred practices and reducing friction in the review process introduced by subjective variation, a few guidelines have been worked out: --8<---cut here---end--->8--- > > 20.4.5.1 Removing non-free software. Non-free software has to be removed in a > snippet; the reason is > that a patch or phase will not work. Well, it might work on their machine, but not for community standards. To reduce confusion: --8<---cut here---start->8--- 20.4.5.1 Removing non-free software. Non-free software should be removed using snippets; when removing non-free software, a patch or phase will not be accepted. --8<---cut here---end--->8--- > > For a patch, the problem is that a patch rem
Re: Welcome to our new committer!
Congrats! August 5, 2022 4:27 PM, "Gábor Boskovits" mailto:gboskov...@gmail.com?to=%22G%C3%A1bor%20Boskovits%22%20)> wrote: Welcome! Efraim Flashner mailto:efr...@flashner.co.il)> ezt írta (időpont: 2022. aug. 5., P, 17:59): The Guix Maintainer Collective (tm) would like to welcome Andrew Tropin, aka abcdw, as our newest committer! I'm sure many of you recognize them from their work on Guix Home and their regular videos hacking on Guix, among other things. So join us in welcoming them! -- Efraim Flashner mailto:efr...@flashner.co.il)> אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted
Re: Welcome to our new committer!
Hi Andrew, A big fat welcome from me as well. Savannah UI being what it is, I'm almost completely certain that I managed to make you a member of the ‘GNU Guix’ organisation there. You should be able to push to the Git repository. For my own future reference: did you get any automated mails about that from Savannah? Kind regards, T G-R signature.asc Description: PGP signature
FSDG issues of SCUMMVM-based games
Hi Guix, The packages - drascula, - lure, - queen, and - sky all share issues that make me question whether they should be in Guix. 1. Their license explicitly prohibits selling of the games themselves and may thus be qualified as non-free. 2. The "sources" consist of binaries that are installed as-is. Given the above, I think we ought not distribute them. Note that this is not a statement on SCUMMVM itself, but only the packages built with it. WDYT?
Re: FSDG issues of SCUMMVM-based games
Hi Liliana, Liliana Marie Prikler 写道： The packages - drascula, […] 1. Their license explicitly prohibits selling of the games themselves and may thus be qualified as non-free. Yep, it's pretty explicit, and I agree that it's an unreasonable restriction that makes the software non-free. 2. The "sources" consist of binaries that are installed as-is. Wow, you weren't kidding. Of the 1145(!) or 63 MiB(!) of files, literally not one is source code. At best, the archive contains 3 ‘text’ files: one with only numbers, one with only asterisks, and one with only blank lines. - lure, - queen, and - sky I didn't check these, but I believe you if you say they're just as bad. I see no way to keep these in Guix. Thanks! T G-R signature.asc Description: PGP signature
