Re: Performance on NFS
Hello! Ludovic Courtès writes: > Hello! > > I just pushed another improvement: > d27cc3bfaafe6b5b0831e88afb1c46311d382a0b significantly reduces the > number of ‘stat’ calls when discovering .scm files. I’d be curious to > see what impact it has on NFS. > > Commit cbee955901b3e252ebdeb8066a2196055149198a also reduces the number > of ‘stat’ calls. > > After that I have: > > --8<---cut here---start->8--- > $ ./pre-inst-env strace -c guix environment --ad-hoc coreutils -- true > % time seconds usecs/call callserrors syscall > -- --- --- - - > 27.030.006916 1 13585 read > 24.640.006306 1 9701 8029 stat > 21.250.005438 1 10698 write > 11.050.0028282828 1 wait4 > 3.600.000922 1 1598 122 open > 2.880.000738 4 20328 futex > 1.790.000458 1 670 mmap > 1.750.000448 1 642 mprotect > 1.640.000419 0 1476 close > 1.360.000347 0 1411 3 lseek > 0.740.000190 2 107 sendfile > 0.700.000180 451 getdents > 0.620.000159 0 886 fstat > 0.540.000137 1 196 lstat > 0.220.57 199 brk > 0.090.24 4 6 clone > 0.020.05 1 5 munmap > 0.020.05 019 rt_sigprocmask > 0.020.04 027 clock_gettime > 0.010.03 3 1 getrandom > 0.010.02 2 1 connect > 0.010.02 1 3 prlimit64 > 0.000.01 1 2 2 access > 0.000.01 1 1 socket > 0.000.00 0 5 rt_sigaction > 0.000.00 0 7 1 ioctl > 0.000.00 0 1 madvise > 0.000.00 0 1 execve > 0.000.00 0 1 uname > 0.000.00 0 3 fcntl > 0.000.00 0 1 getcwd > 0.000.00 0 1 statfs > 0.000.00 0 1 arch_prctl > 0.000.00 0 2 sched_getaffinity > 0.000.00 0 1 set_tid_address > 0.000.00 0 1 set_robust_list > 0.000.00 0 3 pipe2 > -- --- --- - - > 100.000.025590 41417 8185 total > $ git describe > v0.13.0-804-gf4453df9a > --8<---cut here---end--->8--- > > The number of ‘stat’ calls is a function of the number of entries in > ‘GUILE_LOAD_PATH’ and ‘GUILE_LOAD_COMPILED_PATH’. I can’t think of any > easy way to reduce it further. > > The many ‘read’ and ‘write’ are due to the unbuffered port used for > RPCs, and definitely not great when talking to a remote store. I’ve > tried to add some buffering but that turned out to be trickier than I > had hoped for. > > To be continued! > > Ludo’. I applied the patch, and here are the results: [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure -- true The following derivations will be built: /gnu/store/0hz8g844432b5h9zbqr9cpsjy0brg15h-profile.drv /gnu/store/wkksb7bbx3jr0p6p5cj4kkphbwday0yd-info-dir.drv /gnu/store/cd2mwx9qprdy23p7j3pik2zs14nifn36-manual-database.drv Creating manual page database for 1 packages... done in 1.816 s real1m14.686s user0m5.761s sys 0m0.498s [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure -- true real0m34.100s user0m5.599s sys 0m0.414s [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure -- true real0m33.821s user0m5.140s sys 0m0.432s [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure --no-substitutes --no-grafts -- true The following derivations will be built: /gnu/store/rvh0imjdimwm90nzr0fmr5gmp97lyiix-profile.drv /gnu/store/5hm3v4afjf9gix92ixqzv9bwc11a608s-fonts-dir.drv real0m37.200s user0m3.408s sys 0m0.284s [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure --no-substitutes --no-grafts -- true real0m19.415s user0m3.466s sys 0m0.306s [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure --no-substitutes --no-grafts -- true real0m18.850s user0m3.536s sys 0m0.346s [roel@hpcguix guix]$ tim
ld-wrapper and funny bytes in LIBRARY_PATH
Hi,
so I'm trying to enable Rust tests and one of their tests does the following
(paraphrased) in order to exercise the non-UTF8 linker case (which they want to
succeed):
--
bad_dir := zzz$$'\xff'
all:
mkdir $(bad_dir)
cp ... $(bad_dir)/liblibrary.a
LIBRARY_PATH=$(bad_dir) rustc exec.rs
--
This fails for us.
strace yields:
[pid 15059]
open("/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8/zzz?/liblibrary.a",
O_RDONLY) = -1 ENOENT (No such file or directory)
But:
/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8
[env]$ ls zzz\377/liblibrary.a
'zzz'$'\377''/liblibrary.a'
$ ls
/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8/zzz?/liblibrary.a
'/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8/zzz'$'\377''/liblibrary.a'
How do we best proceed?
opendir* missing
Hi, since commit fa73c1937364872560c509f02b3d7648a5bed006 and/or d27cc3bfaafe6b5b0831e88afb1c46311d382a0b I cannot use “guix build” or “guix package” any more. They both fail to find “opendir*” and exit without a backtrace. Running “make clean-go && make” did not help. -- Ricardo GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC https://elephly.net
Re: opendir* missing
Ricardo Wurmus writes: > Hi, > > since commit fa73c1937364872560c509f02b3d7648a5bed006 and/or > d27cc3bfaafe6b5b0831e88afb1c46311d382a0b I cannot use “guix build” or > “guix package” any more. > > They both fail to find “opendir*” and exit without a backtrace. Running > “make clean-go && make” did not help. --8<---cut here---start->8--- $ guix refresh -l libbluray Backtrace: 10 (primitive-load "/gnu/store/js4ml3w20ysh4znp9wl0da0ljji…") In guix/ui.scm: 1324:12 9 (run-guix-command _ . _) In guix/scripts/refresh.scm: 358:7 8 (guix-refresh . _) In unknown file: 7 (force #) In guix/scripts/refresh.scm: 179:39 6 (_) 168:8 5 (importer-modules) In srfi/srfi-1.scm: 466:18 4 (fold # …) In guix/discovery.scm: 108:19 3 (_ _ (# # …)) 95:14 2 (scheme-modules _ _) 70:14 1 (scheme-files "/gnu/store/l2paa4ka8lglar0b778qzl7a6h2v5…") In guix/build/syscalls.scm: 929:19 0 (scandir* "/gnu/store/l2paa4ka8lglar0b778qzl7a6h2v5dzn…" …) guix/build/syscalls.scm:929:19: In procedure scandir*: guix/build/syscalls.scm:929:19: In procedure opendir*: opendir*: No such file or directory --8<---cut here---end--->8--- opendir* is defined as: signature.asc Description: PGP signature
Re: opendir* missing
Marius Bakke writes: > opendir* is defined as: Oops, fat fingers. guix/build/syscalls.scm:866: --8<---cut here---start->8--- (define opendir* (let ((proc (syscall->procedure '* "opendir" '(* (lambda* (name #:optional (string->pointer string->pointer/utf-8)) (let-values (((ptr err) (proc (string->pointer name (if (null-pointer? ptr) (throw 'system-error "opendir*" "opendir*: ~A" (list (strerror err)) (list err)) ptr) --8<---cut here---end--->8--- signature.asc Description: PGP signature
python-pbr-minimal
[env]$ ./pre-inst-env guix lint python-pbr-minimal guix lint: error: python-pbr-minimal: unknown package This is after applying Muriithi's patch that updates python-pbr-minimal to 3.0.1 I didn't try before applying the patch Note that python-pbr (without the -minimal) works like a charm and it inherits from python-pbr-minimal
Re: zipbomb handling should not be done in url-fetch/zipbomb
Arun Isaac skribis: > * Proposal > > zip bomb (zip archives without a top level directory) handling should > not be done in `url-fetch/zipbomb'. It should be implemented as a > boolean argument to the `unpack' phase. I guess the Boolean argument would determine whether to do (chdir (first-subdirectory ".")), right? Unfortunately that’s not enough for the cases where an origin has patches or a snippet, because that code also assumes there’s only one subdirectory (see ‘patch-and-repack’ in (guix packages)). Perhaps the right fix would be to fix ‘patch-and-repack’ somehow. > * Rationale > > I'm changing the download method of a few packages to url-fetch/zipbomb, > and the source gets downloaded again. This should not happen. It is the > same source archive after all. Why download another copy? In my > particular case, these source archives happen to be quite big (around > 500 MB) as well. The only reason it triggers a redownload is because the name is different (specifically, ‘url-fetch/zipbomb’ prepends “zipbomb-” to the name.) To avoid the redownload you could do “guix download file://…” with the right file name. > The download method in source/origin/method should only be involved in > downloading. It should not handle how the downloaded source archive is > unpacked. That is the job of the `unpack' phase. url-fetch/zipbomb > unnecessarily duplicates the "unzip" command invocation. I admit it was a bit of a hack :-), but fixing the ‘unpack’ phase won’t be enough. Thanks, Ludo’.
Re: User-Friendlyness of Guix and non-scaryness, printing messages
Hello, Danny Milosavljevic skribis: > On Tue, 30 May 2017 10:24:34 +0200 > Ricardo Wurmus wrote: > >> Upon failure it can print the location of the build log. > > I'm trying to get this to work but > > guix build --log-file foo > > doesn't seem to print the location of the build log in the failure case... It should work (for local builds at least). Perhaps you also need --no-grafts? Or maybe there’s a bug. > Is it possible to get to it somehow? Sure, just look for in in /var/log/guix/drvs. > Apparently the log files go into log/guix/drvs - but I can't find the place > where the build log is created in the first place... The logs are named after the derivation; see ‘log-file’ in (guix store). HTH! Ludo’.
Re: opendir* missing
Marius Bakke writes: > Marius Bakke writes: > >> opendir* is defined as: > > Oops, fat fingers. guix/build/syscalls.scm:866: > > --8<---cut here---start->8--- > (define opendir* > (let ((proc (syscall->procedure '* "opendir" '(* > (lambda* (name #:optional (string->pointer string->pointer/utf-8)) > (let-values (((ptr err) > (proc (string->pointer name > (if (null-pointer? ptr) > (throw 'system-error "opendir*" >"opendir*: ~A" >(list (strerror err)) >(list err)) > ptr) > --8<---cut here---end--->8--- Well, I suppose the error actually means that whatever directory is supposed to be opened does not exist. It would be great if the error included the directory it tried to open. -- Ricardo GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC https://elephly.net
Expat 2.2.1 with security fixes has been released
Hi! I'm contacting you because Expat 2.2.1 -- containing security fixes -- has just been released. For details please check the change log, online at https://github.com/libexpat/libexpat/blob/master/expat/Changes . Best Sebastian
Syncthing and Go Build System
I saw some posts back in December about adding syncthing to guix, but they stopped in January with no noticeable result. I'd like to see syncthing in guix, although I have an idea of how difficult go is to deal with, so it's understandable that there aren't any go projects in guix. Has there been any progress with this? (If this message goes through twice, I apologize.) -- Ian Dunn
Re: opendir* missing
Heya, Marius Bakke skribis: > Ricardo Wurmus writes: > >> Hi, >> >> since commit fa73c1937364872560c509f02b3d7648a5bed006 and/or >> d27cc3bfaafe6b5b0831e88afb1c46311d382a0b I cannot use “guix build” or >> “guix package” any more. >> >> They both fail to find “opendir*” and exit without a backtrace. Running >> “make clean-go && make” did not help. > > $ guix refresh -l libbluray > Backtrace: > 10 (primitive-load "/gnu/store/js4ml3w20ysh4znp9wl0da0ljji…") > In guix/ui.scm: > 1324:12 9 (run-guix-command _ . _) > In guix/scripts/refresh.scm: > 358:7 8 (guix-refresh . _) > In unknown file: >7 (force #) > In guix/scripts/refresh.scm: >179:39 6 (_) > 168:8 5 (importer-modules) > In srfi/srfi-1.scm: >466:18 4 (fold # …) > In guix/discovery.scm: >108:19 3 (_ _ (# # …)) > 95:14 2 (scheme-modules _ _) > 70:14 1 (scheme-files "/gnu/store/l2paa4ka8lglar0b778qzl7a6h2v5…") > In guix/build/syscalls.scm: >929:19 0 (scandir* "/gnu/store/l2paa4ka8lglar0b778qzl7a6h2v5dzn…" …) > > guix/build/syscalls.scm:929:19: In procedure scandir*: > guix/build/syscalls.scm:929:19: In procedure opendir*: opendir*: No such file > or directory Oops, my bad. Fixed in commit d46c4423f46278bd2f96770ceb0667431414349e. (For me ‘guix package’ and ‘guix build’ worked fine, but that depends on ‘GUIX_PACKAGE_PATH’; ‘guix refresh’ was failing as show above.) Thanks for reporting it! Ludo’.
Re: ld-wrapper and funny bytes in LIBRARY_PATH
Hi Danny,
Danny Milosavljevic skribis:
> so I'm trying to enable Rust tests and one of their tests does the following
> (paraphrased) in order to exercise the non-UTF8 linker case (which they want
> to succeed):
>
> --
> bad_dir := zzz$$'\xff'
>
> all:
> mkdir $(bad_dir)
> cp ... $(bad_dir)/liblibrary.a
> LIBRARY_PATH=$(bad_dir) rustc exec.rs
> --
>
> This fails for us.
So they expect ‘ld’ to properly deal with UTF-8 file names, right?
> strace yields:
>
> [pid 15059]
> open("/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8/zzz?/liblibrary.a",
> O_RDONLY) = -1 ENOENT (No such file or directory)
>
> But:
>
> /tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8
> [env]$ ls zzz\377/liblibrary.a
> 'zzz'$'\377''/liblibrary.a'
>
> $ ls
> /tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8/zzz?/liblibrary.a
> '/tmp/guix-build-rustc-1.16.0.drv-0/rustc-1.16.0-src/src/test/run-make/linker-output-non-utf8/zzz'$'\377''/liblibrary.a'
>
> How do we best proceed?
This is another instance of the file name decoding problem with Guile.
As you know, Guile 2.0/2.2 decodes file names according to the current
locale, and ‘ld-wrapper’ runs Guile.
However, on Guile 2.0 (current ‘master’) ‘ld-wrapper’ runs in the C
locale because it does not explicitly call ‘setlocale’, hence the
question mark instead of \377 in the file name shown by strace.
Fortunately, Guile 2.2 programs (current ‘core-updates’) always start in
the current locale. So I think the problem won’t exist in
‘core-updates’, as long as the build runs in a UTF-8 locale (which is
the case.)
HTH!
Ludo’.
Re: Syncthing and Go Build System
On Sat, Jun 17, 2017 at 03:00:19PM -0400, Ian Dunn wrote: > I saw some posts back in December about adding syncthing to guix, but > they stopped in January with no noticeable result. I'd like to see > syncthing in guix, although I have an idea of how difficult go is to > deal with, so it's understandable that there aren't any go projects in > guix. > > Has there been any progress with this? Recently I dove in to that prototype of the go-build-system, which does build a working Syncthing package. The prototype illustrates what is required for the go-build-system, although it's not ready to be merged into Guix. If nobody else does it, I will eventually finish the go-build-system. However, if somebody would like to try it themselves, I can give advice and explain what parts of the prototype go-build-system should change and how. Or, you are welcome to take your own dive :) By the way, I have an out-of-tree Syncthing package that adapts the gnu-build-system and uses the dozens of bundled dependencies: https://github.com/lfam/guix/commits/contrib-syncthing signature.asc Description: PGP signature
Re: python-pbr-minimal
On Sat, Jun 17, 2017 at 09:32:41PM +0200, Catonano wrote: > [env]$ ./pre-inst-env guix lint python-pbr-minimal > guix lint: error: python-pbr-minimal: unknown package > > This is after applying Muriithi's patch that updates python-pbr-minimal to > 3.0.1 > > I didn't try before applying the patch > > Note that python-pbr (without the -minimal) works like a charm and it > inherits from python-pbr-minimal python-pbr-minimal is a "private" package variable, not exported, and thus not typically available to the command-line tools. It's the difference between (define-public python-pbr-minimal) and (define python-pbr-minimal). You can still pass it as a Scheme expression to some commands: guix build --expression='(@@ (gnu packages python ) python-pbr-minimal)' However, `guix lint` doesn't honor --expression. signature.asc Description: PGP signature
Re: Upgrading packages with substitutes only (bug #26608)
Hello Timothy, Timothy Sample skribis: > I’ve been using GuixSD for a couple of months now, and it is super > cool. Thanks for all your hard work! There is one little thing, > though. :) If it’s just this one thing, that’s okay. ;-) > Ludo suggested to check each package using the “substitute-paths” RPC > and filter them accordingly. I did exactly this, but there was a > problem. Packages were being built before my code even ran! It turns out > that calling “package-output” (or “package-derivation”) to get a > package’s output path may result in building the package. It’s the > grafting code that does this − it needs to get a package’s references, > and if the daemon gives it any trouble, it tells the daemon to just > build the package to figure it out. This is a real problem if you are > trying to avoid building. This all happens pretty early when running > “guix package -u”, since it compares output paths when figuring out what > needs upgrading. Indeed, I hadn’t thought of this “little detail”! > Therefore, to make it work, I introduced a flag, “#:fail-on-build?”, > that I threaded through a few functions that lets me try to get a > package’s output path without resorting to building the package. The > change looks like this: I would probably take a different route: we could check for the substitutes of the ungrafted packages (hydra.gnu.org does not provide substitutes for the grafted packages anyway) and decide based on that. How does that sound? I haven’t checked the code, let me know if there’s something else I’m missing. BTW, should --only-substitutes filter out packages without a substitute, or should it simply stop and report the list of missing substitutes (after which the user could use --do-not-upgrade)? > If adding this extra parameter isn’t too ugly, then I will put together > two patches: one with this parameter, and another that uses it to > implement “--only-substitutes”. (And maybe a third that fixes a very > minor bug in “guix package”, but that’s another story.) I’m new to > Scheme programming, so this may be an atrocity of some sort that I just > don’t know about; go easy! :) Well that’s a very nice dive into this code base in a foreign language. :-) Thank you! Ludo’.
Re: Performance on NFS
Hi! Roel Janssen skribis: > I applied the patch, and here are the results: > > [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure -- true > The following derivations will be built: >/gnu/store/0hz8g844432b5h9zbqr9cpsjy0brg15h-profile.drv >/gnu/store/wkksb7bbx3jr0p6p5cj4kkphbwday0yd-info-dir.drv >/gnu/store/cd2mwx9qprdy23p7j3pik2zs14nifn36-manual-database.drv > Creating manual page database for 1 packages... done in 1.816 s > > real 1m14.686s > user 0m5.761s > sys 0m0.498s > [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure -- true > > real 0m34.100s > user 0m5.599s > sys 0m0.414s > [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure -- true > > real 0m33.821s > user 0m5.140s > sys 0m0.432s You’re telling me it’s just as bad as before, right? > [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure > --no-substitutes --no-grafts -- true > The following derivations will be built: >/gnu/store/rvh0imjdimwm90nzr0fmr5gmp97lyiix-profile.drv >/gnu/store/5hm3v4afjf9gix92ixqzv9bwc11a608s-fonts-dir.drv > > real 0m37.200s > user 0m3.408s > sys 0m0.284s > [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure > --no-substitutes --no-grafts -- true > > real 0m19.415s > user 0m3.466s > sys 0m0.306s > [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure > --no-substitutes --no-grafts -- true > > real 0m18.850s > user 0m3.536s > sys 0m0.346s > [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure > --no-grafts -- true > > real 0m16.003s > user 0m3.246s > sys 0m0.301s > [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure > --no-grafts -- true > > real 0m18.205s > user 0m3.470s > sys 0m0.314s > [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure > --no-substitutes -- true > > real 0m33.731s > user 0m5.111s > sys 0m0.428s > [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure > --no-substitutes -- true > > real 0m30.993s > user 0m5.049s > sys 0m0.458s > > Why is grafting so slow, even if it doesn't have to graft anything? Grafting leads to a bunch of additional RPCs: --8<---cut here---start->8--- $ GUIX_PROFILING=rpc ./pre-inst-env guix build coreutils /gnu/store/mskh7zisxa313anqv68c5lr4hajldjc5-coreutils-8.27-debug /gnu/store/xbvwxf4k5njnb3hn93xwqlppjkiz4hdv-coreutils-8.27 Remote procedure call summary: 379 RPCs build-things ... 1 built-in-builders ... 1 valid-path?... 5 query-substitutable-path-infos ... 8 query-references ...22 query-valid-derivers ...48 add-text-to-store ... 294 $ GUIX_PROFILING=rpc ./pre-inst-env guix build coreutils --no-grafts /gnu/store/mskh7zisxa313anqv68c5lr4hajldjc5-coreutils-8.27-debug /gnu/store/xbvwxf4k5njnb3hn93xwqlppjkiz4hdv-coreutils-8.27 Remote procedure call summary: 294 RPCs built-in-builders ... 1 query-substitutable-path-infos ... 1 build-things ... 1 valid-path?... 5 add-text-to-store ... 286 --8<---cut here---end--->8--- So the problem is probably not NFS in this case but rather RPC performance. However, I can’t help with this until you drop ‘guixr’ and use GUIX_DAEMON_SOCKET=guix:// instead. Hint hint. ;-) Thanks for your feedback, Ludo’.
Re: [PATCH] Prototype register-path
Howdy, Caleb Ristvedt skribis: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Apologies for the delay - in a shocking twist, it turns out professors > insisting on only one exit from functions had a point, and trying to > fully understand what optimisePath_, with its 14 exits and a goto, does, > was a challenge. I think I handle most of the main error cases (EEXIST, > ENOSPC, EMLINK) sort of okay now (the "shrug and ignore" sort of > exceptions are now caught, the rest are still propagated). I've also > created guix/store/deduplication.scm, and moved the relevant stuff > there. The tests pass to the same degree as before (2 failing, but > neither of them are register-path). Heheh. :-) > location: /home/reepca/Programming/guix/tests/store.scm:798 > source: > + (test-assert > + "verify-store" > + (let* ((text (random-text)) > + (file1 (add-text-to-store %store "foo" text)) > + (file2 (add-text-to-store > + %store > + "bar" > + (random-text) > + (list file1 > + (and (pk 'verify1 (verify-store %store)) > + (begin > +(delete-file file1) > +(not (pk 'verify2 (verify-store %store > + (begin > +(call-with-output-file > + file1 > + (lambda (port) (display text port))) > +(pk 'verify3 (verify-store %store)) > reading the Nix store... > path > `/home/reepca/Programming/code-downloads/guix/test-tmp/store/000hh9y0f7i6x9jkhhc3wq1iln4lvwvy-guile-xcb-1.3-guile-builder' > is not in the Nix store > actual-value: #f > actual-error: > + (srfi-34 > + # FOREIGN KEY constraint failed" status: 1] 2d0dea0>) > result: FAIL The dreaded foreign key constraint! Can you rerun the tests after “rm -rf test-tmp”? It might be that you somehow borked the database under test-tmp while testing stuff. > When I create a new module that directly incorporates code written by > others (as in the case of moving reset-timestamps to (guix store > deduplication)), should I add anything to the copyright notice at the > top? Would that involve looking through the commit history to find > exactly who wrote that? Yes, try to preserve the relevant copyright lines (by looking at C-x v l or C-x v g), and also remember to add a line for you when adding new code. > I read HACKING - does the part about waiting two weeks for feedback on > guix-patches before committing still apply to branches? In your case (GSoC) you can go ahead. We’ll probably review more closely when we merge but that can be very quick if we’re already on the same line of thought. > I've now updated my savannah account with a key, used to sign this. > -BEGIN PGP SIGNATURE- Though something went from with your email client. :-) What are your thoughts on the next steps? Thanks, Ludo’.
Re: Performance on NFS
Ludovic Courtès writes: > Hi! > > Roel Janssen skribis: > >> I applied the patch, and here are the results: >> >> [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure -- true >> The following derivations will be built: >>/gnu/store/0hz8g844432b5h9zbqr9cpsjy0brg15h-profile.drv >>/gnu/store/wkksb7bbx3jr0p6p5cj4kkphbwday0yd-info-dir.drv >>/gnu/store/cd2mwx9qprdy23p7j3pik2zs14nifn36-manual-database.drv >> Creating manual page database for 1 packages... done in 1.816 s >> >> real 1m14.686s >> user 0m5.761s >> sys 0m0.498s >> [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure -- true >> >> real 0m34.100s >> user 0m5.599s >> sys 0m0.414s >> [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure -- true >> >> real 0m33.821s >> user 0m5.140s >> sys 0m0.432s > > You’re telling me it’s just as bad as before, right? Sorry for the somewhat hasted response. Well, before it was more variable what the speed was. Now it seems to be pretty stable around ~30 to ~35 seconds with grafting, and ~15 to ~20 seconds without grafting. I really appreciate the effort for optimizing. And I feel it is improving. > >> [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure >> --no-substitutes --no-grafts -- true >> The following derivations will be built: >>/gnu/store/rvh0imjdimwm90nzr0fmr5gmp97lyiix-profile.drv >>/gnu/store/5hm3v4afjf9gix92ixqzv9bwc11a608s-fonts-dir.drv >> >> real 0m37.200s >> user 0m3.408s >> sys 0m0.284s >> [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure >> --no-substitutes --no-grafts -- true >> >> real 0m19.415s >> user 0m3.466s >> sys 0m0.306s >> [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure >> --no-substitutes --no-grafts -- true >> >> real 0m18.850s >> user 0m3.536s >> sys 0m0.346s >> [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure >> --no-grafts -- true >> >> real 0m16.003s >> user 0m3.246s >> sys 0m0.301s >> [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure >> --no-grafts -- true >> >> real 0m18.205s >> user 0m3.470s >> sys 0m0.314s >> [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure >> --no-substitutes -- true >> >> real 0m33.731s >> user 0m5.111s >> sys 0m0.428s >> [roel@hpcguix guix]$ time guixr environment --ad-hoc coreutils --pure >> --no-substitutes -- true >> >> real 0m30.993s >> user 0m5.049s >> sys 0m0.458s >> >> Why is grafting so slow, even if it doesn't have to graft anything? > > Grafting leads to a bunch of additional RPCs: > > --8<---cut here---start->8--- > $ GUIX_PROFILING=rpc ./pre-inst-env guix build coreutils > /gnu/store/mskh7zisxa313anqv68c5lr4hajldjc5-coreutils-8.27-debug > /gnu/store/xbvwxf4k5njnb3hn93xwqlppjkiz4hdv-coreutils-8.27 > Remote procedure call summary: 379 RPCs > build-things ... 1 > built-in-builders ... 1 > valid-path?... 5 > query-substitutable-path-infos ... 8 > query-references ...22 > query-valid-derivers ...48 > add-text-to-store ... 294 > $ GUIX_PROFILING=rpc ./pre-inst-env guix build coreutils --no-grafts > /gnu/store/mskh7zisxa313anqv68c5lr4hajldjc5-coreutils-8.27-debug > /gnu/store/xbvwxf4k5njnb3hn93xwqlppjkiz4hdv-coreutils-8.27 > Remote procedure call summary: 294 RPCs > built-in-builders ... 1 > query-substitutable-path-infos ... 1 > build-things ... 1 > valid-path?... 5 > add-text-to-store ... 286 > --8<---cut here---end--->8--- > > So the problem is probably not NFS in this case but rather RPC > performance. > > However, I can’t help with this until you drop ‘guixr’ and use > GUIX_DAEMON_SOCKET=guix:// instead. Hint hint. ;-) This is what guixr is already doing, see: https://github.com/UMCUGenetics/guix-additions/blob/master/umcu/packages/guix.scm#L95 So I went a little bit further and did this: [roel@hpcguix ~]$ export GUIX_DAEMON_SOCKET="/gnu/daemon-socket/socket" [roel@hpcguix ~]$ export NIX_STATE_DIR=/gnu This means that if I run "guix" on the same machine as where guix-daemon is running, and communicate over the UNIX socket, I should not experience a performance problem, other than a little bit of NFS latency.. Here are the timings: [roel@hpcguix ~]$ time guix environment --ad-hoc coreutils --pure --no-grafts -- true real0m16.293s user0m2.550s sys 0m0.274s [roel@hpcguix ~]$ time guix environment --ad-hoc coreutils --pure --no-grafts -- true real0m15.746s user0m2.411s sys 0m0.260s [roel@hpcguix ~]$ time guix environment --ad-hoc coreutils --pure -- true real0m32.821s user0m4.342s sys 0m0.375s [roel@hpcguix ~]$ time guix environment --ad-hoc coreutils --pure -- true real0m31.140s user0m4.361s sys
Re: python-pbr-minimal
2017-06-18 0:30 GMT+02:00 Leo Famulari : > On Sat, Jun 17, 2017 at 09:32:41PM +0200, Catonano wrote: > > [env]$ ./pre-inst-env guix lint python-pbr-minimal > > guix lint: error: python-pbr-minimal: unknown package > > > > This is after applying Muriithi's patch that updates python-pbr-minimal > to > > 3.0.1 > > > > I didn't try before applying the patch > > > > Note that python-pbr (without the -minimal) works like a charm and it > > inherits from python-pbr-minimal > > python-pbr-minimal is a "private" package variable, not exported, and > thus not typically available to the command-line tools. > > It's the difference between (define-public python-pbr-minimal) and > (define python-pbr-minimal). > yes. I just overlooked the missing "-public" part on the define :-/ I apologize > > You can still pass it as a Scheme expression to some commands: > > guix build --expression='(@@ (gnu packages python ) python-pbr-minimal)' > > However, `guix lint` doesn't honor --expression. > Good to know Thanks
New git signing key for Eric Bavier?
Hi Eric, bav...@member.fsf.org (Eric Bavier) writes: > bavier pushed a commit to branch master > in repository guix. > > commit d56c55fffbe962cf7c44030ae4eac53ffa1e1f85 > Author: Eric Bavier > Date: Thu Jun 15 00:37:17 2017 -0500 > > gnu: tomb: Upgrade to 2.4. This commit, and another recent commit (e0c9530) of yours, were signed using a key that's different from the one you've previously used to sign commits, and different from the key you have registered on Savannah. Can you please confirm that A0C5E3522EF8EF5C64CDB7F0FD73CAC719D32566 is your new key, sign it with your old key if possible, and upload this new key to Savannah? Mark
New git signing key for Jan Nieuwenhuizen?
Hi Jan, > commit ebfb71d45615698040818a68b7dc34996ff4c046 > gpg: Signature made Fri 16 Jun 2017 07:22:49 AM EDT > gpg:using RSA key DB34CB51D25C9408156FCDD6A12F87978D701B99 > gpg: Can't check signature: No public key > Author: Jan Nieuwenhuizen > Date: Thu Jun 15 19:22:00 2017 +0200 > > tests: Allow setting of qemu memory-size for system tests. > > * gnu/system/vm.scm (common-qemu-options): Remove hardcoded "-m 256". > (system-qemu-image/shared-store-script): New keyword argument: > #:memory-size. This commit was signed using a different key than the one you've been using for your other commits, and different from the one you have registered on Savannah. Is DB34CB51D25C9408156FCDD6A12F87978D701B99 your key? Mark
Re: New git signing key for Jan Nieuwenhuizen?
Mark H Weaver writes: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 >> commit ebfb71d45615698040818a68b7dc34996ff4c046 >> gpg: Signature made Fri 16 Jun 2017 07:22:49 AM EDT >> gpg:using RSA key DB34CB51D25C9408156FCDD6A12F87978D701B99 >> gpg: Can't check signature: No public key >> Author: Jan Nieuwenhuizen >> Date: Thu Jun 15 19:22:00 2017 +0200 >> >> tests: Allow setting of qemu memory-size for system tests. >> >> * gnu/system/vm.scm (common-qemu-options): Remove hardcoded "-m 256". >> (system-qemu-image/shared-store-script): New keyword argument: >> #:memory-size. > > This commit was signed using a different key than the one you've been > using for your other commits, and different from the one you have > registered on Savannah. > > Is DB34CB51D25C9408156FCDD6A12F87978D701B99 your key? Yes, that's my new key. Oops. I have updated my key on savannah. janneke -BEGIN PGP SIGNATURE- iQIzBAEBCAAdFiEE2zTLUdJclAgVb83WoS+Hl41wG5kFAllGFv4ACgkQoS+Hl41w G5mcHg/+Pd3AvBLSmoPUrXrhFcFrhQyVdq/d6f2R0JiSkrPy402P003Bp6pzEdD8 OxQzT/ahk7XYRrD0KywCzn6Vd3b7xd+pnBzAJ3TV7lzyXKg97qm6CAAMEdaLNd/3 18dKOUmGI5oTRAvTc7Ubdwc4E1qmI4ckBT3QW2LBDOzPaMY6wiLgIA8vETN2bq+S vIMN9XDQRbtdwwA+k2fXQUB8b5N+bWdc1/YH5DmuqhKUiaHVCHHBc2K5L8YEydPr 0o+MrbwpmlxXjmKQTYFX8L5Wc1DARWECbPIa70TB53uWIxSQdfQQha/Oqd1fMYU0 H9jPBF0PJHjJMyrTwmvMrZFC9Xi4u3UnKdCOPOCz2DXTttOGsGRsP16so34e9UzR DyLD7kR9K/m5djxSYLcIr8mMAfPJ0g/mk1Jyj5Yv9MZVCmFggcfeamirgcvnClxe Nhkfyf7SPeq/qpKIVpIOabthtuuoYiH2imnlLCL+TS6qGZYtBkhxlKUjRYMFWm+F bTwo3c+FdwOeyWdKn4a6QwIzfaJ70o1EeIYS4MUCyB1TKzcsJWE/t5R0SOqz1SX5 ofLsn8amwE0dnoCKQhX92sse+sq4ITj3oD5YpehBl4pvzeIs+7LpLHJSVDSFRfwT N1Xy/PscHohFgcbfD8dZn/xtxRTN8dz1ysrhwLRBl8zc/tRKYPA= =xmQy -END PGP SIGNATURE- -- Jan Nieuwenhuizen | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
