Hi Simon,
zimoun <zimon.touto...@gmail.com> skribis:
> On Sun, 28 Jun 2020 at 22:29, Ludovic Courtès <l...@gnu.org> wrote:
>
>> Most likely the problem with non-reproducible .go files is that the fix
>> for <https://bugs.gnu.org/20272> was incomplete. In particular, I think
>> that gensyms are not reproducible when building things in parallel,
>> because the gensym depends on what’s loaded vs. interpreted.
>
> Thank you for the pointer.
>
> How can I test the "hypothesis" of "building things in parallel"?
>
> guix describe -f channels > /tmp/chan.scm
> guix pull -C /tmp/chan.scm --cores=1 -p /tmp/repull1
>
> guix build --check --no-grafts --cores=1 \
> $(guix gc --derivers \
> $(readlink -f /tmp/repull1/lib/guix/package.cache))
> The following profile hook will be built:
> /gnu/store/qbrgxbnx0hi13xm36a6a0zijzc1rcz22-guix-package-cache.drv
I realize I was a bit off-topic: I was commenting on the more general
issue of .go non-reproducibility.
The problem with the package cache seems to be different. Sorry for the
confusion!
After --check, we can compare both caches like this:
--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (define a (load-compiled
"/gnu/store/0yd3kaar87zyxhbrjqjypp5rar3zj4gb-guix-package-cache/lib/guix/package.cache"))
scheme@(guile-user)> (define b (load-compiled
"/gnu/store/0yd3kaar87zyxhbrjqjypp5rar3zj4gb-guix-package-cache-check/lib/guix/package.cache"))
scheme@(guile-user)> (length a)
$2 = 13949
scheme@(guile-user)> (length b)
$3 = 13949
scheme@(guile-user)> ,use(srfi srfi-1)
scheme@(guile-user)> (lset= equal? a b)
$4 = #f
scheme@(guile-user)> (car (lset-difference equal? a b))
$5 = #("python2" "2.7.17" (gnu packages python) python-2 ("out" "tk") #t #f
"gnu/packages/python.scm" 107 2)
--8<---------------cut here---------------end--------------->8---
So, surprisingly, it’s not just an ordering issue: the caches do contain
different pieces of information.
This patch solves the ordering issue:
diff --git a/gnu/packages.scm b/gnu/packages.scm
index d22c992bb1..5154d73deb 100644
--- a/gnu/packages.scm
+++ b/gnu/packages.scm
@@ -407,13 +407,25 @@ reducing the memory footprint."
(_
result+seen)))
+ (define entry-key
+ (match-lambda
+ (#(name version module symbol outputs supported? deprecated?
+ file line column)
+ (string-append name version (or file "")
+ (if line (number->string line) "")))))
+
+ (define (entry<? a b)
+ (string<? (entry-key a) (entry-key b)))
+
(define exp
- (first
- (fold-module-public-variables* expand-cache
- (cons '() vlist-null)
- (all-modules (%package-module-path)
- #:warn
- warn-about-load-error))))
+ (sort
+ (first
+ (fold-module-public-variables* expand-cache
+ (cons '() vlist-null)
+ (all-modules (%package-module-path)
+ #:warn
+ warn-about-load-error)))
+ entry<?))
(mkdir-p (dirname cache-file))
(call-with-output-file cache-file
But it’s not quite right because the order in which variables are
traversed is still non-deterministic, so between two runs of
‘generate-package-cache’, caches differ like this:
--- a 2020-06-29 14:27:32.291028456 +0200
+++ b 2020-06-29 14:27:37.930993059 +0200
@@ -8581,7 +8581,7 @@
#("clang-runtime"
"9.0.1"
(gnu packages llvm)
- clang-runtime
+ clang-runtime-9
("out")
#t
#f
@@ -26511,7 +26511,7 @@
#("gcc-objc++"
"7.5.0"
(gnu packages gcc)
- gcc-objc++-7
+ gcc-objc++
("out" "lib" "debug")
#t
#f
@@ -26641,7 +26641,7 @@
#("gcc-toolchain"
"7.5.0"
(gnu packages commencement)
- gcc-toolchain
+ gcc-toolchain-7
("out" "debug" "static")
#t
#f
@@ -33311,7 +33311,7 @@
#("ghc"
"8.6.5"
(gnu packages haskell)
- ghc-8.6
+ ghc-8
("out" "doc")
#t
#f
@@ -41876,7 +41876,7 @@
#("icedtea"
"3.7.0"
(gnu packages java)
- icedtea-8
+ icedtea
("out" "jdk" "doc")
#t
#f
@@ -54376,7 +54376,7 @@
#("linux-libre-headers"
"5.4.20"
(gnu packages linux)
- linux-libre-headers-5.4.20
+ linux-libre-headers
("out")
#t
#f
@@ -54636,7 +54636,7 @@
#("llvm"
"9.0.1"
(gnu packages llvm)
- llvm-9
+ llvm
("out" "opt-viewer")
#t
#f
@@ -61826,7 +61826,7 @@
#("ocaml"
"4.09.0"
(gnu packages ocaml)
- ocaml
+ ocaml-4.09
("out")
#t
#f
@@ -62256,7 +62256,7 @@
#("opencl-headers"
"2.2.0-0.e986688"
(gnu packages opencl)
- opencl-headers
+ opencl-headers-2.2
("out")
#t
#f
@@ -92636,7 +92636,7 @@
#("python2"
"2.7.17"
(gnu packages python)
- python-2
+ python-2.7
("out" "tk")
#t
#f
@@ -92646,7 +92646,7 @@
#("python"
"3.8.2"
(gnu packages python)
- python-3
+ python
("out" "tk")
#t
#f
@@ -123676,7 +123676,7 @@
#("rust"
"1.39.0"
(gnu packages rust)
- rust-1.39
+ rust
("out" "doc" "cargo")
#t
#f
The problem has to do with aliases: we don’t always see the same
variable first. So we have to sort before calling ‘expand-cache’.
To be continued…
Ludo’.
