Re: Assisting reviewing & committing with tags?

[Maxim Cournoyer]

Hi,

Leo Famulari  writes:

> On Sun, Jan 09, 2022 at 11:54:25AM +0100, Maxime Devos wrote:
>> Hi guix reviewers and committers,
>> 
>> WDYT of tagging reviewed patches that look good with a usertag,
>> e.g. 'reviewed-looks-good':
>> 
>> https://debbugs.gnu.org/cgi/pkgreport.cgi?tag=reviewed-looks-good&users=guix
>> 
>> then if a committer doesn't have much time to review and hence doesn't
>> subscribe to guix-patches@, but they do trust the reviewer, they can visit
>> that URL to look for reviewed patches that can be applied.
>> 
>> There could also be a tag 'reviewed-looks-good2' if the patch appears ok
>> to two reviewers, or a 'reviewed-needs-work', etc.
>
> This is a great idea. I guess we will need to adjust the software that
> runs issues.guix.gnu.org to make use of it, but in the meantime you
> should keep using this tag. Thanks!

I like it as well.

Maxim

Re: WARNING: Git merges are tricky. Rebasing is better?

[Maxim Cournoyer]

Hello Kyle and Leo,

Kyle Meyer  writes:

> Leo Famulari writes:
>
>> On Mon, Jan 17, 2022 at 05:26:08PM -0500, Kyle Meyer wrote:
>>> Fwiw I don't think Git automatically resolved that conflict:
>>> 
>>>   $ git checkout 276f40fdc349d2ad62582b23ea55e061b689cfc0^
>>>   $ git merge 276f40fdc349d2ad62582b23ea55e061b689cfc0^2
>> [...]   
>>>   ++<<< HEAD
>>>+"0r7m34xzz3shdfxf2abxb069izak3yv3ijlg29qy4pfmyawkilfs"))
>>>+  (patches
>>>+   (search-patches 
>>> "epiphany-update-libportal-usage.patch"
>>>   ++||| d91de53caa
>>>   ++
>>> "0r7m34xzz3shdfxf2abxb069izak3yv3ijlg29qy4pfmyawkilfs"
>>>   ++
>>>   ++===
>>>   + 
>>> "0k7b22zq3z1kllzqxgwsvwb1lp0j6rjb3k1hvhna3i573wc4mpji"
>>>   + 
>>>   ++>>> 276f40fdc349d2ad62582b23ea55e061b689cfc0^2
>>
>> So, you think it was a mistake made when resolving conflicts by hand? I
>> can certainly understand that.
>
> Yes.
>
>> Either way, it's bad that the Git history doesn't show these types of
>> changes.
>
> Right, inspecting merge results can be confusing.  By default that
> change is pruned from diffs as "uninteresting" because the merge result
> matched the content in one of the parents (discussed in the "COMBINED
> DIFF FORMAT" section of git-diff's manpage and the --diff-merges
> description of git-show's manpage).
>
>   $ git show 276f40fd | grep 0r7m34xzz
>   $ git show --diff-merges=combined 276f40fd | grep 0r7m34xzz
>+"0r7m34xzz3shdfxf2abxb069izak3yv3ijlg29qy4pfmyawkilfs"))

Thanks for this interesting discussion, and apologies for mis-resolving
the conflict :-)

Maxim

Expat 2.4.4 with security fixes released

[Sebastian Pipping]

Hello everyone!


Expat 2.4.4 with security fixes has been released.  There is a summary 
blog post [1] and the change log [2] with more details.


If you have patches for Expat that are still required with version
2.4.4, please send them my way.  Thank you!

Best



Sebastian


[1] https://blog.hartwork.org/posts/expat-2-4-4-released/
[2] https://github.com/libexpat/libexpat/blob/R_2_4_4/expat/Changes

Re: Outreachy

[Maxim Cournoyer]

Hi Gabor,

Gábor Boskovits  writes:

[...]

> Q2: Have the situation regarding the funding of these interships been
> settled?

I think so, yes.  We should be able to transfer what we need from our
funds held at the FSF to the Guix Europe-held bank account and pay the
Outreachy stipend from there.

Maxim

Re: linux-libre kernel 4.4 series will end next month (Feb 2022)

[Maxim Cournoyer]

Hi Leo,

Leo Famulari  writes:

> The Linux / linux-libre kernel "longterm" support series version 4.4 is
> scheduled to reach the end of its life next month, in February 2022:
>
> https://www.kernel.org/category/releases.html
>
> There is a project by the Civil Infrastructure Platform (CIP) called
> "SLTS" kernels, which offer "super longterm support". They begin by
> pledging support for the 4.4 series until 2027:
>
> https://wiki.linuxfoundation.org/civilinfrastructureplatform/start
>
> So, we could use that if somebody wanted to make the necessary
> adjustments to our packaging and agree to help keep it working.
>
> I think this is the first time we face end of support of a kernel with
> longterm suport in Guix, at least since we started to keep all the
> longterm kernels packaged [0].
>
> What should we do?

It's been 4 weeks, and nobody offered an opinion.  I suggest we stick to
upstream Linux LTS schedule and drop 4.4, otherwise we'll have even more
variants to maintain.

Thanks,

Maxim

Re: File search

[Maxim Cournoyer]

Hi,

Ricardo Wurmus  writes:

> Ludovic Courtès  writes:
>
>> Ricardo Wurmus  skribis:
>>
>>> raingloom  writes:
>>>
 One use case that I hope can be addressed is TeXlive packages. Trying
 to figure out which package corresponded to which missing file was a
 nightmare the last I had to use LaTeX.
>>>
>>> The texlive package database is the authoritative source of information.
>>> The file texlive.tlpdb is included in the texlive-bin package, and we’re
>>> using it in the importer.  I also added (@ (guix import texlive)
>>> files-differ?), which compares a texlive package’s output directory with
>>> the files that the texlive.tlpdb lists for that package.
>>>
>>> You can also use it to check what package should provide a certain file.
>>> I do this all the time to figure out if our existing packages are
>>> incomplete or if we’re just missing a package.
>>
>> Oh, I had never tried that.  Is there a command that browses
>> texlive.tlpdb, or do you just open it or grep it?
>
> I just have it open in Emacs and search inside.  But we could easily add
> a procedure to (guix import texlive) to check the texlive.tlpdb.  All
> the hard work has already been done; we’re using the same mechanism for
> the importer and “files-differ?”.

It used to be broken, but with the c-u-f merge the 'tlmgr' tool now
works as expected to search for things in the local texlive.tlpdb
database:

--8<---cut here---start->8---
$ guix shell --pure texlive-bin grep which coreutils sed gnupg -- tlmgr info 
cite.sty
tlmgr: cannot find package cite.sty, searching for other matches:

Packages containing `cite.sty' in their title/description:

Packages containing files matching `cite.sty':
abntex2:
texmf-dist/tex/latex/abntex2/abntex2cite.sty
apacite:
texmf-dist/tex/latex/apacite/apacite.sty
chscite:
texmf-dist/tex/latex/chscite/chscite.sty
cite:
texmf-dist/tex/latex/cite/cite.sty
texmf-dist/tex/latex/cite/drftcite.sty
texmf-dist/tex/latex/cite/overcite.sty
combine:
texmf-dist/tex/latex/combine/combcite.sty
computational-complexity:
texmf-dist/tex/latex/computational-complexity/cc2cite.sty
texmf-dist/tex/latex/computational-complexity/cccite.sty
emojicite:
texmf-dist/tex/lualatex/emojicite/emojicite.sty
gcite:
texmf-dist/tex/latex/gcite/gcite.sty
icite:
texmf-dist/tex/latex/icite/icite.sty
kluwer:
texmf-dist/tex/latex/kluwer/klucite.sty
lwarp:
texmf-dist/tex/latex/lwarp/lwarp-cite.sty
texmf-dist/tex/latex/lwarp/lwarp-drftcite.sty
mcite:
texmf-dist/tex/latex/mcite/mcite.sty
notoccite:
texmf-dist/tex/latex/notoccite/notoccite.sty
velthuis:
texmf-dist/tex/latex/velthuis/dvngcite.sty
xcite:
texmf-dist/tex/latex/xcite/xcite.sty
--8<---cut here---end--->8---

It's not great that references to 'grep which coreutils sed gnupg'
aren't patched though (I thought I cared they were, perhaps it
regressed).

HTH,

Maxim

weird OpenBLAS time-machine

[zimoun]

Hi,

Investigating [0], I note that, for these 2 commits,

$ git log --oneline 4b1538e6ef^..ab0cf06244
ab0cf06244 gnu: rust: Remove #:rust ,rust-1.52 arguments.
4b1538e6ef gnu: kexec-tools: Fix build on i686-linux.

where the latter is simply the parent of the former,

$ git cat-file -p ab0cf06244 | grep parent
parent 4b1538e6ef76bf46993f0a368a0abbe2f6eb8ffb

then the version of OpenBLAS is different.

--8<---cut here---start->8---
$ guix time-machine --commit=ab0cf06244 -- show openblas | recsel -P version
0.3.18

$ guix time-machine --commit=4b1538e6ef -- show openblas | recsel -P version
0.3.9
--8<---cut here---end--->8---

The update to 0.3.18 from 0.3.13 is done by bd771edd6c, and as far I
see, there is no merge around; from bd771edd6c to ab0cf06244, all are
direct parents.

Sadly, it is not possible to time-machine to bd771edd6c (Update to
0.3.18) because an unrelated error ((value "Unbound variable: ~S")
(value (rust-1.51))).  For some commits between this range, the
time-machine is also raising an error because (value "Unbound variable:
~S") (value (rust-1.52)).  Anyway, another story. :-)


What is really weird is:

--8<---cut here---start->8---
$ cd $CHECKOUT
$ git show 4b1538e6ef:gnu/packages/maths.scm \
  | grep -n -A 5 'define-public openblas'
4402:(define-public openblas
4403-  (package
4404-(name "openblas")
4405-(version "0.3.18")
4406-(source
4407- (origin
--
4486:(define-public openblas-ilp64
4487-  (package/inherit openblas
4488-(name "openblas-ilp64")
4489-(supported-systems '("x86_64-linux" "aarch64-linux" "mips64el-linux"))
4490-(arguments
4491- (substitute-keyword-arguments (package-arguments openblas)
--8<---cut here---end--->8---

For 4b1538e6ef the version should be 0.3.18 but then:

--8<---cut here---start->8---
$ guix time-machine --commit=4b1538e6ef -- show openblas
name: openblas
version: 0.3.9
outputs: out
systems: x86_64-linux i686-linux
dependencies: cunit@2.1-3 gfortran@7.5.0 perl@5.30.2
location: gnu/packages/maths.scm:3655:2
homepage: https://www.openblas.net/
license: Modified BSD
synopsis: Optimized BLAS library based on GotoBLAS
description: OpenBLAS is a BLAS library forked from the GotoBLAS2-1.13 BSD 
version.
--8<---cut here---end--->8---


Other said, the package defined in
/gnu/store/…-guix-4b1538e6e-modules/share/guile/site/3.0/gnu/packages
does not match the version is the checkout.


Concretely, compare the file gnu/packages/maths.scm from:

--8<---cut here---start->8---
$ guix time-machine --commit=4b1538e6ef -- edit openblas
$ git -C 
~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/ \
  show 4b1538e6ef:gnu/packages/maths.scm
--8<---cut here---end--->8---

Here the diff for only OpenBLAS.

--8<---cut here---start->8---
$ diff /tmp/maths-modules.scm /tmp/maths-checkout.scm | grep -A 45 -B 3 '0.3.18'
3657c4405
< (version "0.3.9")
---
> (version "0.3.18")
3660,3663c4408,4412
<(method url-fetch)
<(uri (string-append "mirror://sourceforge/openblas/v" version 
"/OpenBLAS%20"
(method git-fetch)
>(uri (git-reference
>  (url "https://github.com/xianyi/OpenBLAS";)
>  (commit (string-append "v" version
>(file-name (git-file-name name version))
3666c4415
<  "14iz9xnrb9xiwgj84j94mc74gg0zn2vsy9fmsijxxma1n7dck4w3"
---
>  "17zdd8asylz2w71hczrz5y344p6d5ds1jn4901maw7zcp3dbk63g"
3682a4432
>  "NO_STATIC=1"  ;avoid a 67 MiB static archive
3693c4443
<  ;; Unfortunately, this is not supported on non-x86 architectures,
---
>  ;; Unfortunately, this is not supported on all architectures,
3698,3699c4448,4454
<(string-prefix? "i686" system))
<'("DYNAMIC_ARCH=1"))
---
>(string-prefix? "i686" system)
>(string-prefix? "powerpc64le" system)
>(string-prefix? "aarch64" system))
>;; Dynamic older enables a few extra CPU architectures that
>;; were released before 2010.
>'("DYNAMIC_ARCH=1" "DYNAMIC_OLDER=1" "TARGET=GENERIC"))
>   ;; On some of these architectures the CPU can't be detected.
3705,3707c4460,4462
<   ;; On aarch64 force the generic 'armv8-a' target
<   ((string-prefix? "aarch64" system)
<'("TARGET=ARMV8"))
---
>   ;; Failed to det

Re: Investigating a reproducibility failure

[zimoun]

Hi Konrad,

I get the same error as you.  And for more versions than the only one
your tested.  For instance, for these commits,

* substitutes and rebuilds

923dcc3597 Fri Jan 14 12:59:33 2022 +0100 gnu: iverilog: Update to 11.0.
79ca578182 Thu Nov 11 21:52:08 2021 -0500 gnu: fpc: Fix build.
ab0cf06244 Thu Nov 11 13:35:51 2021 -0500 gnu: rust: Remove #:rust ,rust-1.52 
arguments.
bbd2864272 Sun Dec 27 15:50:08 2020 +0100 gnu: openblas: Update to 0.3.13.


* substitutes but failed rebuilds (--no-grafts --check)

4b1538e6ef Thu Nov 11 12:18:37 2021 -0500 gnu: kexec-tools: Fix build on 
i686-linux.
ade7638d84 Fri Sep 18 14:05:51 2020 +0200 Revert "gnu: openblas: Update to 
0.3.10."
c59e9f0a03 Fri Sep 18 08:57:48 2020 +0200 gnu: openblas: Update to 0.3.10
5969598149 Sat Mar 7 12:48:18 2020 +0100 gnu: openblas: Use HTTPS home page.
2ea095300a Tue Oct 8 21:23:06 2019 +0200 gnu: OpenBLAS: Update to 0.3.7.


* no substitute and failed builds

a4384dc970 Tue Oct 8 21:23:06 2019 +0200 gnu: OpenBLAS: Incorporate grafted 
changes.
ba05be2249 Fri Sep 13 10:50:11 2019 +0200 gnu: openblas: Set 'NUM_THREADS'.
5855756c81 Thu Feb 21 22:04:48 2019 -0600 gnu: openblas: Honor 
parallel-job-count.
602a5ef9f3 Sun Feb 10 21:04:23 2019 +0100 gnu: OpenBLAS: Update to 0.3.5.



Last, note that the time-machine is failing earlier for these commits:

d26584fcda Thu Nov 11 12:18:37 2021 -0500 gnu: binutils-gold: Inherit from 
binutils-next.
ac6f677249 Thu Nov 11 12:18:37 2021 -0500 gnu: Add binutils-next.
661b25a2ed Thu Nov 11 12:18:36 2021 -0500 gnu: openblas: Do not build static 
library.
9e497f44ba Thu Nov 11 12:18:36 2021 -0500 gnu: openblas: Add support for older 
x86 processors.
bd771edd6c Thu Nov 11 12:18:31 2021 -0500 gnu: openblas: Update to 0.3.18
e364758d44 Fri Sep 18 22:26:33 2020 +0200 gnu: openblas: Update to 0.3.10
df5a2e4f83 Thu Mar 5 23:36:05 2020 +0100 gnu: OpenBLAS: Update to 0.3.9
087c94019d Sat Feb 15 22:02:56 2020 +0100 gnu: OpenBLAS: Update to 0.3.8.
e77412362f Sat May 4 16:25:53 2019 +0200 gnu: OpenBLAS: Update to 0.3.6.

which reduces the range for testing.


This bug#51536 is discussing reproducibility of openblas and the
compilation flags.


1: 


Cheers,
simon

missing patch for texlive-bin (e77412362f)

[zimoun]

Hi,

Investigating [0], and working on rebuild a part of "Preservation of
Guix", I hit a missing patch which breaks time-machine.  For instance,
let consider the commit e77412362f from May 2019 (post v1.0.0, I guess).

--8<---cut here---start->8---
$ guix time-machine --commit=e77412362f -- help
Updating channel 'guix' from Git repository at 
'https://git.savannah.gnu.org/git/guix.git'...
Computing Guix derivation for 'x86_64-linux'... |
The following derivations will be built:
   /gnu/store/8mszw785bs8rws21jmmcsgnc6wnz5ik7-profile.drv
   /gnu/store/d7q819gh181gfv3hk4v38g2c1gq5vfgp-guix-e77412362.drv
[...]
building 
/gnu/store/50ib3szgx1h0dyk9rifpsbris7yafgzd-texlive-bin-pdftex-poppler0.75.patch.drv...
\sha256 hash mismatch for 
/gnu/store/7c5jx9zbnh8nlapbxqv1wl8056lhhl2l-texlive-bin-pdftex-poppler0.75.patch:
  expected hash: 1cqpcp7h1qyxyp3wjbpcmx2wgvj9ywpz60hvy280mp9w633yzyg3
  actual hash:   0ribvsg4bka1cyj1wz4cd6vrxkvhqvlmjr75d7fdz5pw9r3rlgk8
hash mismatch for store item 
'/gnu/store/7c5jx9zbnh8nlapbxqv1wl8056lhhl2l-texlive-bin-pdftex-poppler0.75.patch'
build of 
/gnu/store/50ib3szgx1h0dyk9rifpsbris7yafgzd-texlive-bin-pdftex-poppler0.75.patch.drv
 failed
View build log at 
'/var/log/guix/drvs/50/ib3szgx1h0dyk9rifpsbris7yafgzd-texlive-bin-pdftex-poppler0.75.patch.drv.bz2'.
cannot build derivation 
`/gnu/store/10fdksqj03xhykr4xf17jfdy34983aq9-texlive-20180414-source.tar.xz.drv':
 1 dependencies couldn't be built
cannot build derivation 
`/gnu/store/53mpxdvzrfvqdz103sypq9d94px7lx4y-texlive-bin-20180414.drv': 1 
dependencies couldn't be built
cannot build derivation 
`/gnu/store/n75vlzsyrwjarh03y8p18xz6s647cdig-po4a-0.55.drv': 1 dependencies 
couldn't be built
cannot build derivation 
`/gnu/store/k0illak880sh1yr721l96gxjm19kyg8z-guix-translated-texinfo.drv': 1 
dependencies couldn't be built
cannot build derivation 
`/gnu/store/21rj3r95rrs65g053df6gfbij3impdm2-guix-manual.drv': 1 dependencies 
couldn't be built
cannot build derivation 
`/gnu/store/d7q819gh181gfv3hk4v38g2c1gq5vfgp-guix-e77412362.drv': 1 
dependencies couldn't be built
cannot build derivation 
`/gnu/store/8mszw785bs8rws21jmmcsgnc6wnz5ik7-profile.drv': 1 dependencies 
couldn't be built
guix time-machine: error: build of 
`/gnu/store/8mszw785bs8rws21jmmcsgnc6wnz5ik7-profile.drv' failed
--8<---cut here---end--->8---


where the error message is just that the Arch community switched their
hosting URLs:

--8<---cut here---start->8---
Starting download of 
/gnu/store/7c5jx9zbnh8nlapbxqv1wl8056lhhl2l-texlive-bin-pdftex-poppler0.75.patch
>From 
>https://git.archlinux.org/svntogit/packages.git/plain/trunk/pdftex-poppler0.75.patch?h=packages/texlive-bin&id=418dd6f008c3d41a461353fdb60f2d73d87c58ed...
following redirection to `https://gitlab.archlinux.org'...
following redirection to `https://gitlab.archlinux.org/explore/groups'...
@ download-started 
/gnu/store/7c5jx9zbnh8nlapbxqv1wl8056lhhl2l-texlive-bin-pdftex-poppler0.75.patch
 
https://git.archlinux.org/svntogit/packages.git/plain/trunk/pdftex-poppler0.75.patch?h=packages/texlive-bin&id=418dd6f008c3d41a461353fdb60f2d73d87c58ed
 -
@ download-progress 
/gnu/store/7c5jx9zbnh8nlapbxqv1wl8056lhhl2l-texlive-bin-pdftex-poppler0.75.patch
 
https://git.archlinux.org/svntogit/packages.git/plain/trunk/pdftex-poppler0.75.patch?h=packages/texlive-bin&id=418dd6f008c3d41a461353fdb60f2d73d87c58ed
 - 57620
@ download-progress 
/gnu/store/7c5jx9zbnh8nlapbxqv1wl8056lhhl2l-texlive-bin-pdftex-poppler0.75.patch
 
https://git.archlinux.org/svntogit/packages.git/plain/trunk/pdftex-poppler0.75.patch?h=packages/texlive-bin&id=418dd6f008c3d41a461353fdb60f2d73d87c58ed
 - 57620
@ download-succeeded 
/gnu/store/7c5jx9zbnh8nlapbxqv1wl8056lhhl2l-texlive-bin-pdftex-poppler0.75.patch
 
https://git.archlinux.org/svntogit/packages.git/plain/trunk/pdftex-poppler0.75.patch?h=packages/texlive-bin&id=418dd6f008c3d41a461353fdb60f2d73d87c58ed
 57620
--8<---cut here---end--->8---


This asks two questions:

 1. How to get now this patch?  Someone?  Or is it lost forever?

 2. How to preserve such patches?  Other said, are they part of the
 Disarchive machinery?  For now, some patches are probably missed by
 sources.json and thus not ingested by SWH.  Or, do we systematically
 include them in 'gnu/packages/patches' (which appears to me the easiest
 and the robustest) and as proposed for instance by patch#49820 [1]?


On a side note, let point that these patches are downloaded via
'url-fetch' and if instead 'git-fetch' had been used, "guix lint" is
probably not submitting recursive requests to SWH, I guess.

Note also that the question is also related to packages as linux-libre.


0: 
1: 

Cheers,
simon

mesboot broken for old commits

[zimoun]

Hi,

Investigating [0], and working on rebuild a tiny part of "Preservation
of Guix", I note that mesboot is broken for several old commits:

 - 087c94019d from Feb 2020 because of gcc-core-mesboot
 - df5a2e4f83 from Mar 2020 because of coreutils-mesboot
 - e364758d44 from Sep 2020 because of many mesboot components


Two questions:

 1. Is it fixable?  Other said, can we fix the time-machine for these
 commits?

 2. How can we prevent that?


0: 

Cheers,
simon

failure when rebuilding the past: long term?

[zimoun]

Hi,

Investigating [0], and working on rebuilding a tiny part of "Preservation
of Guix", I note some failures of the time-machine.

Let just display the help message using the time-machine for all the 120
commits updating the package guix.  Other said, let just check that it
is possible to time-machine some commits.

--8<---cut here---start->8---
for ci in $(git log --format="%H %s" v1.0.0.. | grep 'guix: Update' | cut -f1 
-d' '); do printf "$ci "; guix time-machine --commit=$ci -- help 1> /dev/null 
2> /dev/null; if [ $? -eq 1 ]; then echo KO; else echo OK; fi ;done
--8<---cut here---end--->8---

These commits are failing:

7bae88b5b9dcacad4dcd11b353b486dc2f8a78e2 Sep 2021
f08587682a631d3fe30159af838c6766dd65205b Oct 2020
7db32c94b0b7d7fe0896389772f7cda802536693 Oct 2020
29d3569c9c712d70466d9175474c8fd1a3262234 Aug 2020
d3eee3c0643a20ba06941ba45d9d27146a8b634d Jul 2020
b778989e9a299102355b7145d1963baed5db7268 Mar 2020
cd2c3dc2d6ed1372ba457d7856b3fdbf097c7095 Nov 2019

7/120, not that bad! :-)


However, for instance, I miss how it is possible to get:

--8<---cut here---start->8---
@ build-started 
/gnu/store/9p560p4gd4f7jpbwnc8sarqkxfyxpxb9-guix-1.1.0-28.d27dbeb-checkout.drv 
- x86_64-linux 
/var/log/guix/drvs/9p//560p4gd4f7jpbwnc8sarqkxfyxpxb9-guix-1.1.0-28.d27dbeb-checkout.drv.bz2
 3761976
@ build-log 3761976 41
guile: warning: failed to install locale
@ build-log 3761976 152
environment variable `PATH' set to 
`/gnu/store/378zjf2kgajcfd7mfr98jn5xyc5wa3qv-gzip-1.10/bin:/gnu/store/sf3rbvb6iqcphgm1afbplcs72hsywg25-tar-1.32/bin'
@ build-log 3761976 117
Initialized empty Git repository in 
/gnu/store/6ccci55npzlzb0pnpm6sf5f8swnnrpfg-guix-1.1.0-28.d27dbeb-checkout/.git/
/@ build-log 3761976 65
fatal: dumb http transport does not support shallow capabilities
@ build-log 3761976 55
Failed to do a shallow fetch; retrying a full fetch...
\@ build-log 3761976 324
>From https://git.savannah.gnu.org/r/guix
[...]
HEAD is now at d27dbeb9d8 gnu: guix: Install OpenRC init files to $(prefix)/etc.
@ hash-mismatch 
/gnu/store/6ccci55npzlzb0pnpm6sf5f8swnnrpfg-guix-1.1.0-28.d27dbeb-checkout 
r:sha256 05mvljdr4clnv8i89db2hpjm33xg7jcg1vs00dbb4jcivlpkmqrl 
0j60m9s47n23flfp2yn4ww4vsk8qvp500m2x1x0ib5bjywj1hiwl
hash mismatch for store item 
'/gnu/store/6ccci55npzlzb0pnpm6sf5f8swnnrpfg-guix-1.1.0-28.d27dbeb-checkout'
@ build-failed 
/gnu/store/9p560p4gd4f7jpbwnc8sarqkxfyxpxb9-guix-1.1.0-28.d27dbeb-checkout.drv 
- 1 hash mismatch for store item 
'/gnu/store/6ccci55npzlzb0pnpm6sf5f8swnnrpfg-guix-1.1.0-28.d27dbeb-checkout'
--8<---cut here---end--->8---

or another:

--8<---cut here---start->8---
fatal: reference is not a tree: 537080fad8dfa63df2f1d0b0d046a28077d56a56
@ build-log 3768401 160
git-fetch: 
'/gnu/store/i5b1vv7qc6l2gi4xwa9mqzjy3shvgk30-git-minimal-2.28.0/bin/git 
checkout 537080fad8dfa63df2f1d0b0d046a28077d56a56' failed with exit code 128
--8<---cut here---end--->8---

Other said, why is time-machine fully cloning from network and not
reusing '~/.cache/guix/checkouts' since it has already done earlier with
«Updating channel 'guix' from Git repository at
'https://git.savannah.gnu.org/git/guix.git'...»?


The questions are then:

 1. For these failed commits, is it fixable?  If yes, how?

 2. If not, what could be done to cut earlier?  For instance, collect a
 list of commits known to be unreachable.

 3. Having all the sources is one thing, but being able to rebuild is
 another.  Failure of OpenBLAS [0] is one example, of some mesboot [1]
 or of texlive [2] are others.  It appears to me that something is
 inadequate with the current workflow pushing all to master without any
 automated* checks. Other said, failures as 8f9fd9b70c (value "Unbound
 variable: ~S") (value (r-biobase) seems wrong by design.  Well,
 ac6f677249 is another recent example.  Somehow, because the package
 collection is becoming larger and larger (which is good!)  then it is
 becoming harder and harder to maintain the consistency both forward and
 backward.  For the last Guix revision, my rough estimate is that ~5% of
 packages are broken and my guess is that this number is “independant”
 of the package collection size.  However, I already have some
 collection of unreachable commits by the time-machine and then for some
 reachable commits, I do not have numbers for what is effectively
 rebuildable.  As discussed in this thread [3], maybe Guix is moving too
 fast; or better worded, maybe the current workflow is inadequate with
 some goals of long term and build all from sources.  I do not know…  My
 point here: do we provide a list of commits (release, others) where we
 apply more care for long term?

*automated check: “guix lint” is not automated since it depends on the
 submitter and/or the committer; and for having spent some time to che

Re: missing patch for texlive-bin (e77412362f)

[Maxime Devos]

zimoun schreef op wo 02-02-2022 om 19:42 [+0100]:
> [...]
> 
> This asks two questions:
> 
>  1. How to get now this patch?  Someone?  Or is it lost forever?

>  2. How to preserve such patches?  Other said, are they part of the
>  Disarchive machinery?  For now, some patches are probably missed by
>  sources.json and thus not ingested by SWH.  Or, do we systematically
>  include them in 'gnu/packages/patches' (which appears to me the easiest
>  and the robustest) and as proposed for instance by patch#49820 [1]?
> 

My suggestion: in these cases, we can do

  (origin
(patches
  (list (file-append
  (origin (uri (git-reference
 (url "https://git.archlinux.org/...";)
 (commit "deadbeef...")))
  (method git-fetch)
  (sha256 "...")))
  "/.../pdftex-poppler0.76.patch".

It's a bit long-winded, but SWH should be happy with this.
This doesn't retroactively unbreak old commits though ...

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: missing patch for texlive-bin (e77412362f)

[Maxime Devos]

zimoun schreef op wo 02-02-2022 om 19:42 [+0100]:
>  1. How to get now this patch?  Someone?  Or is it lost forever?

I found it!  It's in the
https://github.com/archlinux/svntogit-packages/ repository (warning:
0.5 GiB).  Checkout commit 155510dd18d2f290085f40d2a95a3701db4a224d,
then do

$ guix download texlive-bin/repos/extra-x86_64/pdftex-poppler0.75.patch
/gnu/store/fbvknc65339dnf89aljz8wkbnnms7ydp-pdftex-poppler0.75.patch
1cqpcp7h1qyxyp3wjbpcmx2wgvj9ywpz60hvy280mp9w633yzyg3

^ this is the expected hash!

Greetings,
Maxime.
commit 67290f70d8143b18f24ffd6a1827def9bdc21108
Author: Akira Kakuto 
Date:   Thu Dec 6 23:31:33 2018 +

support system poppler 0.72.0

git-svn-id: svn://tug.org/texlive/trunk/Build/source@49336 c570f23f-e606-0410-a88d-b1316a301751


diff --git a/texk/web2c/pdftexdir/pdftoepdf-poppler0.75.0.cc b/texk/web2c/pdftexdir/pdftoepdf-poppler0.75.0.cc
new file mode 100644
index 0..630d3366d
--- /dev/null
+++ b/texk/web2c/pdftexdir/pdftoepdf-poppler0.75.0.cc
@@ -0,0 +1,1113 @@
+/*
+Copyright 1996-2017 Han The Thanh, 
+
+This file is part of pdfTeX.
+
+pdfTeX is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+pdfTeX is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License along
+with this program.  If not, see .
+*/
+
+/*
+This is based on the patch texlive-poppler-0.59.patch <2017-09-19> at
+https://git.archlinux.org/svntogit/packages.git/plain/texlive-bin/trunk
+by Arch Linux. A little modifications are made to avoid a crash for
+some kind of pdf images, such as figure_missing.pdf in gnuplot.
+The poppler should be 0.72.0 or newer versions.
+POPPLER_VERSION should be defined.
+*/
+
+/* Do this early in order to avoid a conflict between
+   MINGW32  defining 'boolean' as 'unsigned char' and
+defining Pascal's boolean as 'int'.
+*/
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#ifdef POPPLER_VERSION
+#include 
+#include 
+#include 
+#include 
+#include 
+#define GString GooString
+#else
+#error POPPLER_VERSION should be defined.
+#endif
+#include 
+
+#include "Object.h"
+#include "Stream.h"
+#include "Array.h"
+#include "Dict.h"
+#include "XRef.h"
+#include "Catalog.h"
+#include "Link.h"
+#include "Page.h"
+#include "GfxFont.h"
+#include "PDFDoc.h"
+#include "GlobalParams.h"
+#include "Error.h"
+
+// This file is mostly C and not very much C++; it's just used to interface
+// the functions of xpdf, which are written in C++.
+
+extern "C" {
+#include 
+#include 
+
+// These functions from pdftex.web gets declared in pdftexcoerce.h in the
+// usual web2c way, but we cannot include that file here because C++
+// does not allow it.
+extern int getpdfsuppresswarningpagegroup(void);
+extern integer getpdfsuppressptexinfo(void);
+extern integer zround(double);
+}
+
+// The prefix "PTEX" for the PDF keys is special to pdfTeX;
+// this has been registered with Adobe by Hans Hagen.
+
+#define pdfkeyprefix "PTEX"
+
+#define MASK_SUPPRESS_PTEX_FULLBANNER 0x01
+#define MASK_SUPPRESS_PTEX_FILENAME   0x02
+#define MASK_SUPPRESS_PTEX_PAGENUMBER 0x04
+#define MASK_SUPPRESS_PTEX_INFODICT   0x08
+
+// When copying the Resources of the selected page, all objects are copied
+// recusively top-down. Indirect objects however are not fetched during
+// copying, but get a new object number from pdfTeX and then will be
+// appended into a linked list. Duplicates are checked and removed from the
+// list of indirect objects during appending.
+
+enum InObjType {
+objFont,
+objFontDesc,
+objOther
+};
+
+struct InObj {
+Ref ref;// ref in original PDF
+InObjType type; // object type
+InObj *next;// next entry in list of indirect objects
+int num;// new object number in output PDF
+fd_entry *fd;   // pointer to /FontDescriptor object structure
+int enc_objnum; // Encoding for objFont
+int written;// has it been written to output PDF?
+};
+
+struct UsedEncoding {
+int enc_objnum;
+GfxFont *font;
+UsedEncoding *next;
+};
+
+static InObj *inObjList;
+static UsedEncoding *encodingList;
+static bool isInit = false;
+
+// 
+// Maintain list of open embedded PDF files
+// 
+
+struct PdfDocument {
+char *file_name;
+PDFDoc *doc;
+XRef *xref;
+InObj *inObjList;
+int occurences; // number of referenc

Re: missing patch for texlive-bin (e77412362f)

[Maxime Devos]

zimoun schreef op wo 02-02-2022 om 19:42 [+0100]:
>  2. How to preserve such patches?  Other said, are they part of the
>  Disarchive machinery?  For now, some patches are probably missed by
>  sources.json and thus not ingested by SWH.

Can we tell SWH to ingest svntogit-packages, in such a way that every
individual file from every commit can be individually fetched from
SWH?  If so, then all (historical?) patches in Archlinux would be
saved.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: missing patch for texlive-bin (e77412362f)

[zimoun]

Hi Maxime,

Thanks for the pointer.


On Wed, 2 Feb 2022 at 23:57, Maxime Devos  wrote:
>
> zimoun schreef op wo 02-02-2022 om 19:42 [+0100]:
> >  2. How to preserve such patches?  Other said, are they part of the
> >  Disarchive machinery?  For now, some patches are probably missed by
> >  sources.json and thus not ingested by SWH.
>
> Can we tell SWH to ingest svntogit-packages, in such a way that every
> individual file from every commit can be individually fetched from
> SWH?  If so, then all (historical?) patches in Archlinux would be
> saved.

The request for saving the repo is done; if it is not already in.  See [1].

But the question is if Disarchive dissambles and preserves external
patches.  Timothy?

For instance, the file sources.json ingested by SWH is not doing that
yet, if I remember correctly.


1: 

Re: Investigating a reproducibility failure

[Ricardo Wurmus]

Konrad Hinsen  writes:

> Konrad Hinsen  writes:
>
>> To see the failure, do
>>
>>guix time-machine \
>> --commit=7357b3d7a52eb5db1674012c50d308d792741c48 \
>> -- build openblas
>>
>> The build log is attached, the first error is
>
> Oops... Two mistakes ! First, I forgot the attachment, so here it comes,
> Second, I didn't quote the right commit. The failure happens with
>
> guix time-machine \
>  --commit=87e7faa2ae641d8302efc8b90f1e45f43f67f6da \
>  -- build openblas

It builds fine on this laptop.

--8<---cut here---start->8---
$ lscpu
Architecture:x86_64
  CPU op-mode(s):32-bit, 64-bit
  Address sizes: 39 bits physical, 48 bits virtual
  Byte Order:Little Endian
CPU(s):  4
  On-line CPU(s) list:   0-3
Vendor ID:   GenuineIntel
  Model name:Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
CPU family:  6
Model:   78
Thread(s) per core:  2
Core(s) per socket:  2
Socket(s):   1
Stepping:3
CPU max MHz: 3100.
CPU min MHz: 400.
BogoMIPS:5199.98
Flags:   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge 
mca cmov pat pse36 clflush dts acpi mmx fxsr ss
 e sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm 
constant_tsc art arch_perfmon pebs bts rep_good nop
 l xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq 
dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg 
 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe 
popcnt tsc_deadline_timer aes xsave avx f16c rdra
 nd lahf_lm abm 3dnowprefetch cpuid_fault epb 
invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi
  flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 
avx2 smep bmi2 erms invpcid mpx rdseed adx sm
 ap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves 
dtherm ida arat pln pts hwp hwp_notify hwp_act
 _window hwp_epp md_clear flush_l1d
Virtualization features: 
  Virtualization:VT-x
Caches (sum of all): 
  L1d:   64 KiB (2 instances)
  L1i:   64 KiB (2 instances)
  L2:512 KiB (2 instances)
  L3:4 MiB (1 instance)
NUMA:
  NUMA node(s):  1
  NUMA node0 CPU(s): 0-3
--8<---cut here---end--->8---

CPU detection is a bottomless can of worms.

-- 
Ricardo

Re: Investigating a reproducibility failure

[zimoun]

Hi Konrad,

What is the output of 'lscpu'?

For instance, on machine A running on Intel(R) Xeon(R) Gold 5218 CPU @
2.30GHz, OpenBLAS for commit 87e7faa2ae641d8302efc8b90f1e45f43f67f6da
builds.
On machine B running Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz,
OpenBLAS for the same commit fails.


Cheers,
simon

Re: Investigating a reproducibility failure

[Ricardo Wurmus]

Ricardo Wurmus  writes:

> Konrad Hinsen  writes:
>
>> Konrad Hinsen  writes:
>>
>>> To see the failure, do
>>>
>>>guix time-machine \
>>> --commit=7357b3d7a52eb5db1674012c50d308d792741c48 \
>>> -- build openblas
>>>
>>> The build log is attached, the first error is
>>
>> Oops... Two mistakes ! First, I forgot the attachment, so here it comes,
>> Second, I didn't quote the right commit. The failure happens with
>>
>> guix time-machine \
>>  --commit=87e7faa2ae641d8302efc8b90f1e45f43f67f6da \
>>  -- build openblas
>
> It builds fine on this laptop.
>
> $ lscpu
> Architecture:x86_64
>   CPU op-mode(s):32-bit, 64-bit
>   Address sizes: 39 bits physical, 48 bits virtual
>   Byte Order:Little Endian
> CPU(s):  4
>   On-line CPU(s) list:   0-3
> Vendor ID:   GenuineIntel
>   Model name:Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
> CPU family:  6
> Model:   78
> Thread(s) per core:  2
> Core(s) per socket:  2
> Socket(s):   1
> Stepping:3
> CPU max MHz: 3100.
> CPU min MHz: 400.
> BogoMIPS:5199.98
> Flags:   fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge 
> mca cmov pat pse36 clflush dts acpi mmx fxsr ss
>  e sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm 
> constant_tsc art arch_perfmon pebs bts rep_good nop
>  l xtopology nonstop_tsc cpuid aperfmperf pni 
> pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg 
>  fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe 
> popcnt tsc_deadline_timer aes xsave avx f16c rdra
>  nd lahf_lm abm 3dnowprefetch cpuid_fault epb 
> invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi
>   flexpriority ept vpid ept_ad fsgsbase tsc_adjust 
> bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx sm
>  ap clflushopt intel_pt xsaveopt xsavec xgetbv1 
> xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act
>  _window hwp_epp md_clear flush_l1d
> Virtualization features: 
>   Virtualization:VT-x
> Caches (sum of all): 
>   L1d:   64 KiB (2 instances)
>   L1i:   64 KiB (2 instances)
>   L2:512 KiB (2 instances)
>   L3:4 MiB (1 instance)
> NUMA:
>   NUMA node(s):  1
>   NUMA node0 CPU(s): 0-3

I also built this on a different machine, foreign distro.  Here’s the
output of lscpu:

--8<---cut here---start->8---
[rwurmus@beast:~] (571) $ lscpu 
Architecture:  x86_64
CPU op-mode(s):32-bit, 64-bit
Byte Order:Little Endian
CPU(s):120
On-line CPU(s) list:   0-119
Thread(s) per core:2
Core(s) per socket:15
Socket(s): 4
NUMA node(s):  4
Vendor ID: GenuineIntel
CPU family:6
Model: 62
Model name:Intel(R) Xeon(R) CPU E7-4870 v2 @ 2.30GHz
Stepping:  7
CPU MHz:   2127.050
CPU max MHz:   2900.
CPU min MHz:   1200.
BogoMIPS:  4588.44
Virtualization:VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache:  256K
L3 cache:  30720K
NUMA node0 CPU(s): 
0,4,8,12,16,20,24,28,32,36,40,44,48,52,56,60,64,68,72,76,80,84,88,92,96,100,104,108,112,116
NUMA node1 CPU(s): 
1,5,9,13,17,21,25,29,33,37,41,45,49,53,57,61,65,69,73,77,81,85,89,93,97,101,105,109,113,117
NUMA node2 CPU(s): 
2,6,10,14,18,22,26,30,34,38,42,46,50,54,58,62,66,70,74,78,82,86,90,94,98,102,106,110,114,118
NUMA node3 CPU(s): 
3,7,11,15,19,23,27,31,35,39,43,47,51,55,59,63,67,71,75,79,83,87,91,95,99,103,107,111,115,119
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx 
pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology 
nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est 
tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt 
tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ssbd ibrs ibpb stibp 
tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida 
arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
--8<---cut here---end--->8---

The output differs, but the build did not fail.

-- 
Ricardo

Re: weird OpenBLAS time-machine

[zimoun]

Hi,

On Wed, 02 Feb 2022 at 18:00, zimoun  wrote:

> --8<---cut here---start->8---

[...]

> $ guix time-machine --commit=4b1538e6ef -- show openblas | recsel -P version
> 0.3.9
> --8<---cut here---end--->8---

The issue is because concurrency.  If two time-machines are run
concurrently, they both update ~/.cache/guix/checkouts/ and the end
result is hard to predict.

Well, I probably ran inside one terminal “guix time-machine
--commit= -- help” where  a commit with openblas@0.3.9; and in
the same time inside another terminal, “guix time-machine
--commit=4b1538e6ef -- help”.  Depending on “the same time”, the state
of the checkout
~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq
used by the time-machine at 4b1538e6ef was the state for the commit
.

Then, all is messed!

Somehow, IIUC the code behind, what is missing is a lock when updating
the cached checkout to prevent concurrent unrelated updates.


For sure, on 3 independent other machines, and after GC the initial
machine, I get the same error for this commit 4b1538e6ef:

--8<---cut here---start->8---
(repl-version 0 1 1)
Generating package cache for 
'/gnu/store/9zq283291ak72xrvlhax89gyl578kbbg-profile'...
(exception unbound-variable (value #f) (value "Unbound variable: ~S") (value 
(rust-1.52)) (value #f))
--8<---cut here---end--->8---


Sorry for the noise.

Cheers,
simon