Question about packaging TexLive

[Simon Tournier]

Hi,

CC: TeX team

For context, TeX (broad meaning) is deep in the graph, thus its
preservation is important when considering time-travel.  And because TeX
packages relies on Subversion, it raises some questions – for the
interesting reader, see [1].

Here, I would like to open a discussion about the way we are packaging
TeX, quoting Nicolas from [2]:

There are two approaches going on here. `simple-texlive-package' tries
to provide a tessellation of the full texlive, i.e., if you install
(assuming they are defined in Guix) every standalone texlive package,
you will get the exact full texlive distribution, whereas
`texlive-ref' + `texlive-build-system' tries to generate a minimal (a
bit too minimal considering missing documentation) working system.
I think the old way is error prone.

So, do we (not only the two of us, obviously) agree on the
`simple-texlive-package' approach? If so, what should we do about
"source"?

2: https://issues.guix.gnu.org/issue/62570#10

Other said, any objection to go from this pattern:

--8<---cut here---start->8---
(define-public texlive-foo
  (package
(name "texlive-foo")
(version (number->string %texlive-revision))
(source (origin
  (method svn-multi-fetch)
  (uri (svn-multi-reference
(url (string-append "svn://www.tug.org/texlive/tags/"
%texlive-tag "/Master/texmf-dist"))
(locations '("/metafont/"
 "/fonts/source/public/modes/"))
(revision %texlive-revision)))
  (file-name (string-append name "-" version "-checkout"))
  (sha256
   (base32
""
(build-system gnu-build-system)
(arguments
 `(#:tests? #f ; no test target
   #:phases
[...]
--8<---cut here---end--->8---

to this pattern:

--8<---cut here---start->8---
(define-public texlive-foo
  (package
(inherit (simple-texlive-package
  "texlive-foo"
  (list "/tex/generic/foo")
  (base32
   "")
  #:trivial? #t))
--8<---cut here---end--->8---

It is about ~35 packages, if I count correctly.

WDYT?

Cheers,
simon

1: https://issues.guix.gnu.org/43442#20

Re: Question about packaging TexLive

[Nicolas Goaziou]

Hello,

Simon Tournier  writes:

> Other said, any objection to go from this pattern:
>
> --8<---cut here---start->8---
> (define-public texlive-foo
>   (package
> (name "texlive-foo")
> (version (number->string %texlive-revision))
> (source (origin
>   (method svn-multi-fetch)
>   (uri (svn-multi-reference
> (url (string-append "svn://www.tug.org/texlive/tags/"
> %texlive-tag "/Master/texmf-dist"))
> (locations '("/metafont/"
>  "/fonts/source/public/modes/"))
> (revision %texlive-revision)))
>   (file-name (string-append name "-" version "-checkout"))
>   (sha256
>(base32
> ""
> (build-system gnu-build-system)
> (arguments
>  `(#:tests? #f ; no test target
>#:phases
> [...]
> --8<---cut here---end--->8---
>
>
> to this pattern:
>
> --8<---cut here---start->8---
> (define-public texlive-foo
>   (package
> (inherit (simple-texlive-package
>   "texlive-foo"
>   (list "/tex/generic/foo")
>   (base32
>"")
>   #:trivial? #t))
> --8<---cut here---end--->8---
>
> It is about ~35 packages, if I count correctly.
>
> WDYT?

I think this is necessary but not sufficient.

1. "#:trivial? #t" means we're giving up generating ".sty" files from
   source;

2. Some affected packages are also missing propagated inputs (e.g.,
   texlive-latex-graphics for texlive-latex-fncychap);

3. Some affected package also need to be renamed
   (texlive-latex-fncychap, texlive-latex-framed).

AFAIU, this changes will probably ease time travelling, but the massive
rebuild it will entail will have to be repeated for the other steps.

So maybe we could do every step above in one go?

Regards,
-- 
Nicolas Goaziou

Re: Question about packaging TexLive

[Ricardo Wurmus]

Nicolas Goaziou  writes:

> Hello,
>
> Simon Tournier  writes:
>
>> Other said, any objection to go from this pattern:
>>
>> --8<---cut here---start->8---
>> (define-public texlive-foo
>>   (package
>> (name "texlive-foo")
>> (version (number->string %texlive-revision))
>> (source (origin
>>   (method svn-multi-fetch)
>>   (uri (svn-multi-reference
>> (url (string-append "svn://www.tug.org/texlive/tags/"
>> %texlive-tag "/Master/texmf-dist"))
>> (locations '("/metafont/"
>>  "/fonts/source/public/modes/"))
>> (revision %texlive-revision)))
>>   (file-name (string-append name "-" version "-checkout"))
>>   (sha256
>>(base32
>> ""
>> (build-system gnu-build-system)
>> (arguments
>>  `(#:tests? #f ; no test target
>>#:phases
>> [...]
>> --8<---cut here---end--->8---
>>
>>
>> to this pattern:
>>
>> --8<---cut here---start->8---
>> (define-public texlive-foo
>>   (package
>> (inherit (simple-texlive-package
>>   "texlive-foo"
>>   (list "/tex/generic/foo")
>>   (base32
>>"")
>>   #:trivial? #t))
>> --8<---cut here---end--->8---
>>
>> It is about ~35 packages, if I count correctly.
>>
>> WDYT?
>
> I think this is necessary but not sufficient.
>
> 1. "#:trivial? #t" means we're giving up generating ".sty" files from
>source;

Trivial should only ever be used for packages where there aren’t any
sources to build from.

> 2. Some affected packages are also missing propagated inputs (e.g.,
>texlive-latex-graphics for texlive-latex-fncychap);

Correct.  The old importer didn’t know about dependencies.  The newer
importer uses tlpdb to get a list of dependencies.

> 3. Some affected package also need to be renamed
>(texlive-latex-fncychap, texlive-latex-framed).
>
> AFAIU, this changes will probably ease time travelling, but the massive
> rebuild it will entail will have to be repeated for the other steps.
>
> So maybe we could do every step above in one go?

Using svn-multi-reference is the old way; we should be using
simple-texlive-package for all packages wherever possible.

Yes, we still have a lot of old package names.  They all should be
renamed to the new style.

All packages must be checked for completeness.  All new packages I’ve
added have been checked.  All old packages must be checked before
renaming them.  The way to check them is to build them and then check
the outputs with something like this:

--8<---cut here---start->8---
,use (guix import texlive)
,pp (files-differ? "/gnu/store/…-texlive-amsfonts-fixed-59745/share/" 
"amsfonts")
--8<---cut here---end--->8---

This compares outputs with tlpdb.

Note that simple-texlive-package is far from perfect.  We have many
tex packages that require excessive workarounds to ensure that files end
up in the correct location and that superfluous files are removed.
simple-texlive-package should eventually take care of all of this, but I
haven’t been able to make time to work on it.

All this work should be done on a separate feature branch.

-- 
Ricardo

Latest news on core-updates

[Andreas Enge]

Hello all,

just a quick update to share good news and to heap praise on people who are
not rewarded by seeing their name in a git commit.

Am Mon, Apr 17, 2023 at 11:03:25AM +0200 schrieb Andreas Enge:
> On aarch64 and powerpc, we are still stuck by CI problems.

Thanks to tireless work by Ricardo, Ludovic, Julien, Christopher and myself
enough machines for these architectures are running in the build farm to
hopefully clear the backlog. We went from 1 to 5 aarch machines (or from 4
to 14 build slots), and from 1 to 2 powerpc64le machines (or from 2 to 8
build slots).

The dash boards are filling up, and on x86_64 and i686 the situation looks
comparable to master. On powerpc64le, I have been restarting cancelled jobs
with a little script, and the red dots are also becoming less. On aarch64,
I am waiting for the backlog from master to clear up before trying something
similar. I think the situation on these architectures is better than it
looks on the dashboard.

I hope we will be able to merge core-updates after next weekend; also for
personal reasons, since I will not be able to spend much time on it any
more after that.

So please help us in a last sprint to get core-updates into shape!

Andreas

Re: Latest news on core-updates

[Christopher Baines]

Andreas Enge  writes:

> I hope we will be able to merge core-updates after next weekend; also for
> personal reasons, since I will not be able to spend much time on it any
> more after that.

After next weekend (around the end of April) sounds good to me.

The bordeaux build farm has pretty much caught up after the staging
merge which should mean there's some capacity to attempt to build
core-updates changes prior to the merge around next weekend.

I haven't restarted submitting builds for core-updates given I think
there's still some big changes going to land, but in any case, it would
be good to get this going once there are no planned big changes (aka the
branch is frozen). Getting the builds to happen currently requires a
code change in the qa-frontpage, but I can do this, I just need someone
to let me know when I should.

Thanks,

Chris


signature.asc
Description: PGP signature

Re: GOOPS-less Shepherd

[Christine Lemmer-Webber]

Ludovic Courtès  writes:

> Hello!
>
> I’d like to release the Shepherd 0.10.0 in a few weeks at most, with the
> hope that it’ll be the last stable series before 1.0, which would be
> released in a few months.
>
> As part of this, I’d like to clean up the API, which includes removing
> the dependency on GOOPS.  The Shepherd had been using GOOPS from the
> start but for no particular reason: there’s no inheritance and only a
> couple of cases of method overloading.

Neat!

"Obviously" Shepherd should use Spritely Goblins, which also does not
use GOOPS ;D

> I started that work, which mostly involves renaming things like
> accessors following typical Scheme conventions:
>
>   https://git.savannah.gnu.org/cgit/shepherd.git/log/?h=wip-goopsless

404s for me.

> I imagine the following deprecation scenario:
>
>   • 0.10.x would still support GOOPS, as in (make  …), but
> that would be deprecated in favor of a more Schemey (service …)
> form.  Under the hood it’s still GOOPS.
>
>   • 1.0.x would no longer use GOOPS at all.  We could provide a ‘make’
> macro so that (make  …) would still kinda work.
>
> What’s at stake, mostly, is the ability to reconfigure a long-running
> shepherd instance.  Once 1.0.x is in Guix, ‘guix system reconfigure’
> will most likely fail to upgrade services on a pre-0.10.x shepherd:
> users will have to reboot.
>
> Thoughts?
>
> Ludo’.

Re: Question about packaging TexLive

[Nicolas Goaziou]

Ricardo Wurmus  writes:

> Nicolas Goaziou  writes:
>
>> 1. "#:trivial? #t" means we're giving up generating ".sty" files from
>>source;
>
> Trivial should only ever be used for packages where there aren’t any
> sources to build from.

We are on the same length.

However, some packages do use "#:trivial #t" even though they should
not, e.g., `texlive-latex-marginfix'. My point is that we should not
make "#:trivial #t" the default, assuming that was a possibility
stemming from Simon's question.

>> 2. Some affected packages are also missing propagated inputs (e.g.,
>>texlive-latex-graphics for texlive-latex-fncychap);
>
> Correct.  The old importer didn’t know about dependencies.  The newer
> importer uses tlpdb to get a list of dependencies.

AFAIU, this is not sufficient. All dependencies do not appear in
"texlive.tlpbd". For example, `texlive-halloweenmath' has no "depend"
entry in the tlpbd. Yet, it requires both texlive-amsmath and
texlive-pict2e (i.e., "halloweenmath.sty" contains
"\RequirePackage{amsmath,pict2e}").

So, unless I'm mistaken, most dependencies have to be resolved manually.

> All packages must be checked for completeness.  All new packages I’ve
> added have been checked.  All old packages must be checked before
> renaming them.  The way to check them is to build them and then check
> the outputs with something like this:
>
> --8<---cut here---start->8---
> ,use (guix import texlive)
> ,pp (files-differ? "/gnu/store/…-texlive-amsfonts-fixed-59745/share/" 
> "amsfonts")
> --8<---cut here---end--->8---
>
> This compares outputs with tlpdb.

I use this valuable tool, but this is somewhat tedious. Could this
become a test in the linter?

> Note that simple-texlive-package is far from perfect.  We have many
> tex packages that require excessive workarounds to ensure that files end
> up in the correct location and that superfluous files are removed.
> simple-texlive-package should eventually take care of all of this, but I
> haven’t been able to make time to work on it.

We may need to update more Texlive packages to get a better grasp of the
changes required in `simple-texlive-packages'. When doing such updates,
I mostly copy and paste definition from previously updated packages,
which is a good thing. However, some translations seem complicated.

> All this work should be done on a separate feature branch.

Agreed.

Regards,

Re: Question about packaging TexLive

[Andreas Enge]

Am Wed, Apr 19, 2023 at 01:47:16PM +0200 schrieb Nicolas Goaziou:
> AFAIU, this is not sufficient. All dependencies do not appear in
> "texlive.tlpbd". For example, `texlive-halloweenmath' has no "depend"
> entry in the tlpbd. Yet, it requires both texlive-amsmath and
> texlive-pict2e (i.e., "halloweenmath.sty" contains
> "\RequirePackage{amsmath,pict2e}").
> So, unless I'm mistaken, most dependencies have to be resolved manually.

Could the result be upstreamed as bug reports?

Andreas

Re: Question about packaging TexLive

[Ricardo Wurmus]

Nicolas Goaziou  writes:

>>> 2. Some affected packages are also missing propagated inputs (e.g.,
>>>texlive-latex-graphics for texlive-latex-fncychap);
>>
>> Correct.  The old importer didn’t know about dependencies.  The newer
>> importer uses tlpdb to get a list of dependencies.
>
> AFAIU, this is not sufficient. All dependencies do not appear in
> "texlive.tlpbd". For example, `texlive-halloweenmath' has no "depend"
> entry in the tlpbd. Yet, it requires both texlive-amsmath and
> texlive-pict2e (i.e., "halloweenmath.sty" contains
> "\RequirePackage{amsmath,pict2e}").
>
> So, unless I'm mistaken, most dependencies have to be resolved manually.

Yes.  My point was that tlpdb contains some dependency information,
which is completely absent in those packages produced with the earlier
importer.

We should change the importer to scan the directory for RequirePackage
and the like.

>> All packages must be checked for completeness.  All new packages I’ve
>> added have been checked.  All old packages must be checked before
>> renaming them.  The way to check them is to build them and then check
>> the outputs with something like this:
>>
>> --8<---cut here---start->8---
>> ,use (guix import texlive)
>> ,pp (files-differ? "/gnu/store/…-texlive-amsfonts-fixed-59745/share/" 
>> "amsfonts")
>> --8<---cut here---end--->8---
>>
>> This compares outputs with tlpdb.
>
> I use this valuable tool, but this is somewhat tedious. Could this
> become a test in the linter?

I don’t see why it could not.

>> Note that simple-texlive-package is far from perfect.  We have many
>> tex packages that require excessive workarounds to ensure that files end
>> up in the correct location and that superfluous files are removed.
>> simple-texlive-package should eventually take care of all of this, but I
>> haven’t been able to make time to work on it.
>
> We may need to update more Texlive packages to get a better grasp of the
> changes required in `simple-texlive-packages'.

A good example for the required changes is texlive-totpages or
texlive-soulutf8.  All these overrides to #:tex-directory and #:phases
are pretty common.  We usually need to add a chdir phase to change
directories to the sources and override the copy-files phase, because
otherwise it copies garbage.

This is the least I’d expect simple-texlive-packages to take care of.

-- 
Ricardo

Re: Latest news on core-updates

[Andreas Enge]

Am Wed, Apr 19, 2023 at 11:58:03AM +0100 schrieb Christopher Baines:
> I haven't restarted submitting builds for core-updates given I think
> there's still some big changes going to land, but in any case, it would
> be good to get this going once there are no planned big changes (aka the
> branch is frozen).

I am not sure what "frozen" means exactly; I would say we must not make
commits to core-updates unless they repair a broken package, or maybe help
repair broken packages further down towards the leaves. (And this has been
true for a while now.) And maybe we should go further and refrain from
repairing things in core-updates that are already broken on master, at
least if they are relatively close to the root of the dependency graph.
These can be done later in feature branches; I think we have the infra-
structure to shoulder bigger changes outside of core-updates.

On the other hand, this report is a bit worrying:
   [core-updates] locales not installed properly
   https://issues.guix.gnu.org/62934
It could potentially mean changes to glibc, which would throw us back
a week.

Andreas

Mesa vulkan layer path fix for core-updates

[Kaelyn]

Hi,

Some time back I mailed in https://issues.guix.gnu.org/59453 to fix an issue 
with the manifests for the vulkan layers that ship with mesa. Basically the 
error is that the manifests don't include the store path to the referenced 
libraries, only the file name. An example the error can be seen by running 
"vulkaninfo &| less", where this message is printed a few times:

ERROR: [Loader Message] Code 0 : libVkLayer_MESA_device_select.so: cannot open 
shared object file: No such file or directory


While I know it is late in the process of preparing core-updates for merging 
into master, I wanted to ask if it would be possible to have the patch 
addressed prior to the merge? I just encountered the error message again after 
having a need to check vulkaninfo and it reminded me of the patch. (The 
vulkan-validationlayers package that was recently merged to master from staging 
includes a similar phase for fixing the layer object path in its layer 
manifest.)

Thanks,
Kaelyn

Re: Mesa vulkan layer path fix for core-updates

[Andreas Enge]

Hello,

thanks for bringing this back to our attention!

Am Wed, Apr 19, 2023 at 02:41:57PM + schrieb Kaelyn:
> While I know it is late in the process of preparing core-updates for merging 
> into master, I wanted to ask if it would be possible to have the patch 
> addressed prior to the merge?

Given how many packages depend on mesa and their importance, I think it
would be safer to postpone the fix until after the merge; be it in a
dedicated mesa feature branch that could quickly be merged afterwards,
or better yet regroup other changes in this area. (Searching for open mesa
packages on issues.guix.gnu.org returns quite a few matches, this could be
a good occasion to sort them out.)

Andreas

Re: Mesa vulkan layer path fix for core-updates

[Kaelyn]

--- Original Message ---
On Wednesday, April 19th, 2023 at 3:26 PM, Andreas Enge  wrote:


> 
> 
> Hello,
> 
> thanks for bringing this back to our attention!

You're welcome! :)

> 
> Am Wed, Apr 19, 2023 at 02:41:57PM + schrieb Kaelyn:
> 
> > While I know it is late in the process of preparing core-updates for 
> > merging into master, I wanted to ask if it would be possible to have the 
> > patch addressed prior to the merge?
> 
> 
> Given how many packages depend on mesa and their importance, I think it
> would be safer to postpone the fix until after the merge; be it in a
> dedicated mesa feature branch that could quickly be merged afterwards,
> or better yet regroup other changes in this area. (Searching for open mesa
> packages on issues.guix.gnu.org returns quite a few matches, this could be
> a good occasion to sort them out.)

I'm okay with it waiting until after the core-updates merge and going into a 
feature branch, especially if that branch includes updating mesa to the latest 
release (currently 23.0.0, or 22.3.7 if going with the latest patch of the 
previous feature release instead of the x.y.0 release of the newest series).

Inspired by your comment, I also just did a quick scan of 
https://issues.guix.gnu.org/search?query=mesa+is%3Aopen and took a peek at some 
tickets that looked like they may be related to the mesa package. From that 
perspective, six other tickets caught my eye in the results:

* https://issues.guix.gnu.org/62176 can be closed when core-updates is merged, 
since core-updates contains mesa 22.2.4

* https://issues.guix.gnu.org/58887 looks like an alternate way of solving the 
layer path issues that https://issues.guix.gnu.org/59453 also addresses. 
Additionally, it adds two new nonstandard VK_*_PATH variables to vulkan-loader, 
with at least VK_ILAYER_PATH seeming very similar in functionality to 
VK_LAYER_PATH and VK_ADD_LAYER_PATH described at 
https://github.com/KhronosGroup/Vulkan-Loader/blob/main/docs/LoaderInterfaceArchitecture.md

* https://issues.guix.gnu.org/58251 would be fixed by 
https://issues.guix.gnu.org/59453

* https://issues.guix.gnu.org/62313 might need a modification to mesa e.g. to 
add VDPAU_DRIVER_PATH as a native-search-path (one possible solution; in my 
home profile I made VDPAU work by setting 
"VDPAU_DRIVER_PATH=/run/current-system/profile/lib/vdpau").

* https://issues.guix.gnu.org/48868 appears to be the same VDPAU_DRIVER_PATH 
issue as https://issues.guix.gnu.org/62313.

* Though not exactly mesa-related, https://issues.guix.gnu.org/61364 can 
possibly be closed now, and almost certainly once the core-updates merge is 
completed. (The ticket is a number of workarounds the user applied in early Feb 
to be able to build their system profile using core-updates, to pick up Mesa 22 
for newer hardware support. I'm not sure if any of the patches are still 
relevant.)

Cheers,
Kaelyn

Re: Mesa vulkan layer path fix for core-updates

[John Kehayias]

Hi Kaelyn, Andreas, and Guix!

On Wed, Apr 19, 2023 at 04:07 PM, Kaelyn wrote:

> --- Original Message ---
> On Wednesday, April 19th, 2023 at 3:26 PM, Andreas Enge  
> wrote:
>
>>
>> Hello,
>>
>> thanks for bringing this back to our attention!
>
> You're welcome! :)

Yes, thanks Kaelyn, especially since I believe I said I wanted to help
these patches through for core-updates, before I had to step away for
a bit (all good now!).

>>
>> Am Wed, Apr 19, 2023 at 02:41:57PM + schrieb Kaelyn:
>>
>> Given how many packages depend on mesa and their importance, I think it
>> would be safer to postpone the fix until after the merge; be it in a
>> dedicated mesa feature branch that could quickly be merged afterwards,
>> or better yet regroup other changes in this area. (Searching for open mesa
>> packages on issues.guix.gnu.org returns quite a few matches, this could be
>> a good occasion to sort them out.)
>
> I'm okay with it waiting until after the core-updates merge and going
> into a feature branch, especially if that branch includes updating
> mesa to the latest release (currently 23.0.0, or 22.3.7 if going with
> the latest patch of the previous feature release instead of the x.y.0
> release of the newest series).
>

I was going to suggest the same, a feature branch just after
core-updates is merged, including getting Mesa to the latest version
(I suspect a stable v23 will be here soon). I've used a newer Mesa
locally for some packages and I didn't hit any issues in the packaging
for v23.

I would be happy to team up with you to prepare this branch once it is
time. I see you did a good search below collecting the various related
issues/patches. I can take a look as well.

> Inspired by your comment, I also just did a quick scan of
>  and took a
> peek at some tickets that looked like they may be related to the mesa
> package. From that perspective, six other tickets caught my eye in the
> results:
>
> *  can be closed when core-updates is
> merged, since core-updates contains mesa 22.2.4
>

Great, we'll be able to close it Real Soon Now.

> *  looks like an alternate way of
> solving the layer path issues that 
> also addresses. Additionally, it adds two new nonstandard VK_*_PATH
> variables to vulkan-loader, with at least VK_ILAYER_PATH seeming very
> similar in functionality to VK_LAYER_PATH and VK_ADD_LAYER_PATH
> described at
> 
>

Thanks, will have a look at the differences.

> *  would be fixed by 
> 
>

Great, always good to close some issues :)

> *  might need a modification to mesa
> e.g. to add VDPAU_DRIVER_PATH as a native-search-path (one possible
> solution; in my home profile I made VDPAU work by setting
> "VDPAU_DRIVER_PATH=/run/current-system/profile/lib/vdpau").
>

Right, this one is my report. I haven't had a chance to return to it
to figure out what is best here. Things do work with manual
intervention. It is a little tricky since my guess is we want VDPAU to
work without needing to install e.g. Mesa explicitly. But I don't
remember the details right now. Happy to discuss!

> *  appears to be the same
> VDPAU_DRIVER_PATH issue as .
>

Yes, I think so too, though VLC does have Mesa as an input so the fix
might be easier directly for VLC at least.

> * Though not exactly mesa-related, 
> can possibly be closed now, and almost certainly once the core-updates
> merge is completed. (The ticket is a number of workarounds the user
> applied in early Feb to be able to build their system profile using
> core-updates, to pick up Mesa 22 for newer hardware support. I'm not
> sure if any of the patches are still relevant.)
>

Also my quick reading of it; will double check none of those are still
out of date and we can close with core-updates.

> Cheers,
> Kaelyn

Thanks again Kaelyn! I think this will make for a small (but
non-trivial) set of patches for a feature branch and trying out that
workflow. We can then have the CI build the branch and people can try
their system with the newer Mesa plus fixes for testing. I don't think
there is any hardware support being dropped (unlike with v22) and it
will help for newer hardware, so I don't anticipate issues. But let me
not say that too loudly.

Looking forward to it!

John

Re: Help wanted: search paths for Erlang and Elixir

[Ivan Sokolov]

Ivan Sokolov  writes:

> In order for the Erlang runtime to find the installed application it
> must appear in the load path.  If there is ERL_ROOTDIR in the
> environment, erl(1) will search for application directories in
> $ERL_ROOTDIR/lib and will add them to the load path.  ([2] mentions
> $OTPROOT but it's probably an error in the documentation.)

It turns out that OTPROOT is not a real variable, but a placeholder for
the Erlang installation directory, and ERL_ROOTDIR is intented for
internal use only.  Thus ERL_LIBS is the only option left.

> [2]: https://www.erlang.org/doc/man/code.html#code-path

Re: [core-updates] issues with python's zip

[Andreas Enge]

Hello,

Am Fri, Apr 14, 2023 at 06:47:04PM -0400 schrieb Brian Cully via Development of 
GNU Guix and the GNU System distribution.:
> I've run into issues with two packages on core-updates that fail to build
> because the zip library being used can't handle dates before 1980. I assume
> there are more.
> Do we have a way to handle this gracefully? In the mean time I've patched
> both packages (‘sssd’ and ‘criu’) with a new build phase:

criu fails like this for me:
copying crit.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
pycriu.images.__pycache__.__init__.cpython-310: module references __file__
creating dist
creating 'dist/crit-3.17.1-py3.10.egg' and adding 
'build/bdist.linux-x86_64/egg' to it
Traceback (most recent call last):
  File "/tmp/guix-build-criu-3.17.1.drv-0/source/scripts/crit-setup.py", line 
16, in 
setup(name="crit",
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/core.py",
 line 177, in setup
return run_commands(dist)
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/core.py",
 line 193, in run_commands
dist.run_commands()
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/dist.py",
 line 968, in run_commands
self.run_command(cmd)
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/dist.py",
 line 1217, in run_command
super().run_command(command)
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/dist.py",
 line 987, in run_command
cmd_obj.run()
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/install.py",
 line 74, in run
self.do_egg_install()
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/install.py",
 line 123, in do_egg_install
self.run_command('bdist_egg')
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/cmd.py",
 line 317, in run_command
self.distribution.run_command(command)
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/dist.py",
 line 1217, in run_command
super().run_command(command)
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/_distutils/dist.py",
 line 987, in run_command
cmd_obj.run()
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py",
 line 226, in run
make_zipfile(self.egg_output, archive_root, verbose=self.verbose,
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py",
 line 452, in make_zipfile
visit(z, dirname, files)
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py",
 line 445, in visit
z.write(path, p)
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/zipfile.py",
 line 1739, in write
zinfo = ZipInfo.from_file(filename, arcname,
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/zipfile.py",
 line 518, in from_file
zinfo = cls(arcname, date_time)
  File 
"/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/zipfile.py",
 line 362, in __init__
raise ValueError('ZIP does not support timestamps before 1980')
ValueError: ZIP does not support timestamps before 1980


I came across this:
   https://docs.python.org/3/library/zipfile.html#zipfile-objects
So apparently when creating a zipfile object (this is the constructor
we are looking at, right?), one can pass a parameter that tells it to
not bother about time stamps of old files.

Then it is a matter of finding out where it happens. I think it is
line 450 of (in my case)
/gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py
which reads
z = zipfile.ZipFile(zip_filename, mode, compression=compression)
Maybe we should patch our Python so that it becomes
z = zipfile.ZipFile(zip_filename, mode, compression=compression, 
strict_timestamps=False)
?
(not before the core-updates merge, however)

But is a bit surprising that it does not happen all the time;
or are only a few packages calling bdist_egg.py?

Andreas

PSA for LUKS users

[Development of GNU Guix and the GNU System distribution.]

Hi,

Given the broad popularity of LUKS full-disk encryption among our
fellow Guix users, I thought the community might appreciate reading
about potentially weak key-derivation functions in older LUKS
installations. [1]

The article even offers fixes, although I cannot say whether your
system will boot after you follow the steps since I do not use LUKS
personally. Stay safe!

Kind regards
Felix Lechner

[1] https://mjg59.dreamwidth.org/66429.html

Re: [core-updates] issues with python's zip

[Development of GNU Guix and the GNU System distribution.]


Andreas Enge  writes:

> I came across this:
>   https://docs.python.org/3/library/zipfile.html#zipfile-objects
> So apparently when creating a zipfile object (this is the constructor
> we are looking at, right?), one can pass a parameter that tells it to
> not bother about time stamps of old files.
> 
> Then it is a matter of finding out where it happens. I think it is
> line 450 of (in my case)
> /gnu/store/dy3xh053ahkhrp2jamggq8cpsyvp8mg0-python-3.10.7/lib/python3.10/site-packages/setuptools/command/bdist_egg.py
> which reads
>z = zipfile.ZipFile(zip_filename, mode, compression=compression)
> Maybe we should patch our Python so that it becomes
>z = zipfile.ZipFile(zip_filename, mode, compression=compression, 
> strict_timestamps=False)
> ?
> (not before the core-updates merge, however)

Thanks for digging into this. Maybe it makes sense to patch bdist_egg.py in 
setuptools, then, rather than chase down every failed build?

> But is a bit surprising that it does not happen all the time;
> or are only a few packages calling bdist_egg.py?

I don't know. According to 
https://www.python101.pythonlibrary.org/chapter38_eggs.html it's an older 
format, which has been superseded by ‘wheel’. I know I see wheel a fair amount, 
but I'm not much of a Python hacker.

-bjc

Re: PSA for LUKS users

[Vagrant Cascadian]

On 2023-04-19, Felix Lechner via wrote:
> Given the broad popularity of LUKS full-disk encryption among our
> fellow Guix users, I thought the community might appreciate reading
> about potentially weak key-derivation functions in older LUKS
> installations. [1]
>
> The article even offers fixes, although I cannot say whether your
> system will boot after you follow the steps since I do not use LUKS
> personally. Stay safe!
...
> [1] https://mjg59.dreamwidth.org/66429.html

In short, those instructions will almost certainly break Guix System!

While recent grub2 finally has limited support for luks2, it only
supports the weaker KDF (key derivation function) (PBKDF2?), as I
understand it, though would be happy to be proven wrong!

Because Guix System does not yet support a separate /boot partition,
this means if you want "full-disk encryption" you are limited to weak
KDF for the whole filesystem, instead of just a weak /boot partition
(e.g. either luks1, luks2 with weaker pbkdf2, or entirely
unencrypted). There is a bug about being able to use a split /boot
partition:

  https://issues.guix.gnu.org/48172

Alternately, you could probably get a weaker encrypted rootfs (using
luks1 or luks2+PBKDF) and still have a state-of-the-art luks2+argon2id
partition for /home. Maybe if you were adventurous /var/guix, which
might allow detecting a compromise with "guix gc" which contains the
checksums of files in /gnu/store?

With both the split /boot approach or the weaker rootfs with stronger
/home partition, there is some risk of a (admittedly very sophisticated
and still probably quite expensive) evil maid attack.

  https://en.wikipedia.org/wiki/Evil_maid_attack


Well... fun times, folks!


live well,
  vagrant


signature.asc
Description: PGP signature