bug#28772: guix system reconfigure after kernel panic user or group not created

[Ludovic Courtès]

Hi Oleg,

Oleg Pykhalov  skribis:

> apologies for not adding logs before.  It's hard to do when I do guix
> commands from Xterm and not from Emacs.  Emacs *shell* or *compilation*
> buffers will eat all memory if they get too much text.

I sympathize…

> The problem
> ===
>
> The bigger problem from my view are files like /etc/group.lock and
> /etc/passwd.lock.  For example:
>
> sudo touch /etc/group.lock
>
> /etc/config.scm
>
> (operating-system
>   ;; …
>   (groups (cons
>  (user-group (name "test"))
>  %base-groups)))
>
> reconfigure log

I think we can avoid the problem by forcefully removing these two lock
files at boot time:

diff --git a/gnu/services.scm b/gnu/services.scm
index 329b7b151..2ef1d8530 100644
--- a/gnu/services.scm
+++ b/gnu/services.scm
@@ -368,6 +368,8 @@ boot."
 #t
 ;; Ignore I/O errors so the system can boot.
 (fail-safe
+ (delete-file "/etc/group.lock")
+ (delete-file "/etc/passwd.lock")
  (delete-file-recursively "/tmp")
  (delete-file-recursively "/var/run")
  (mkdir "/tmp")

>How to make a kernel panic
>
> The problem will be No defined variable IPTABLES-SSH after 'guix system
> reconfigure' and kernel crash after.

[...]

> Make a kernel panic
>
> sudo GUILE_LOAD_PATH=\"$HOME/src/iptables\
> :$GUILE_LOAD_PATH\" guix system reconfigure \
> $HOME/dotfiles/guix/system-magnolia.scm
>
> # Run above again and kernel will panic.

I tried to reproduce it with my user’s shepherd, but that didn’t work:

--8<---cut here---start->8---
ludo@ribbon ~/src/guix$ cat ,t.scm
(define s
  (make 
#:provides '(nothing)
#:start (lambda _ unbound)))

(register-services s)
(start s)
ludo@ribbon ~/src/guix$ herd load root ,t.scm
Loading ,t.scm.
herd: exception caught while executing 'load' on service 'root':
ERROR: Unbound variable: foo
ludo@ribbon ~/src/guix$ herd load root ,t.scm
Loading ,t.scm.
herd: exception caught while executing 'start' on service 'nothing':
ERROR: Unbound variable: unbound
ludo@ribbon ~/src/guix$ herd load root ,t.scm
Loading ,t.scm.
Assertion (null? (lookup-services (canonical-name new))) failed.
herd: exception caught while executing 'load' on service 'root':
ERROR: Throw to key `assertion-failed' with args `()'.
ludo@ribbon ~/src/guix$ echo $?
1
ludo@ribbon ~/src/guix$ herd status

[...]

ludo@ribbon ~/src/guix$ echo $?
0
--8<---cut here---end--->8---

IOW, shepherd caught the exceptions and didn’t die.

What am I missing?

Ludo’.

bug#28772: guix system reconfigure after kernel panic user or group not created

[Oleg Pykhalov]

Hello Ludovic,

l...@gnu.org (Ludovic Courtès) writes:

[...]

> I think we can avoid the problem by forcefully removing these two lock
> files at boot time:
>
> diff --git a/gnu/services.scm b/gnu/services.scm
> index 329b7b151..2ef1d8530 100644
> --- a/gnu/services.scm
> +++ b/gnu/services.scm
> @@ -368,6 +368,8 @@ boot."
>  #t
>  ;; Ignore I/O errors so the system can boot.
>  (fail-safe
> + (delete-file "/etc/group.lock")
> + (delete-file "/etc/passwd.lock")
>   (delete-file-recursively "/tmp")
>   (delete-file-recursively "/var/run")
>   (mkdir "/tmp")
>
>

There is also a '/etc/.pwd.lock'.  Info about this file
https://lists.debian.org/debian-user/2005/07/msg02949.html


I'm not sure if any files are exist.  Days past after reconfigure
failure.

$ sudo find /etc -name '*.lock' # Shows nothing.

[...]

> IOW, shepherd caught the exceptions and didn’t die.
>
> What am I missing?

I'll try to make a reproducible thing later.

Thanks,
Oleg.

bug#28768: guix system vm Failed to install GRUB (EFI)

[Ludovic Courtès]

Marius Bakke  skribis:

> Oleg Pykhalov  writes:
>
>> EFI bootloader configuration causes problems to test with 'system vm'
>> before 'reconfigure'.
>
> This is because `guix system vm` is not currently UEFI enabled, so GRUB
> tries to install for a normal PC BIOS.
>
> The fix would be to make the various QEMU invocations in (gnu system vm)
> take a #:firmware parameter that passes something along the lines of
> '-bios #$(file-append ovmf "/share/ovmf/firmware/ovmf_x64.bin")' to the
> QEMU command-line, and update other scripts to take advantage.

Perhaps by adding it to  records we’d make it less
annoying to pass around?  Or are there other places that would still
need extra care?

Another option, in the meantime is this:

diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index 273a895be..f763b430b 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -565,6 +565,14 @@ environment with the store shared with the host.  MAPPINGS is a list of
   user-file-systems)))
 
   (operating-system (inherit os)
+
+;; XXX: Until we run QEMU with UEFI support (with the OVMF firmware),
+;; force the traditional i386/BIOS method.
+;; See .
+(bootloader (bootloader-configuration
+  (bootloader grub-bootloader)
+  (target "/dev/vda")))
+
 (initrd (lambda (file-systems . rest)
   (apply base-initrd file-systems
  #:volatile-root? #t

That’s what I do manually anyway.

WDYT?  If that’s fine with you, I can commit it.

Thanks,
Ludo’.

bug#28243: Guix cross-compilation doesn't work

[Manolis Ragkousis]

Hello Ricardo,

On 10/10/17 00:34, Ricardo Wurmus wrote:

> Have you been able to confirm this?
> 
> I just wanted to do this on darnassus, but guix-daemon is not running
> there, and I cannot seem to get it to build software as a user, because
> obviously I cannot write to /gnu/store.
> 
> Could you give me a hint how to run guix on Darnassus?

While disabling grafts does solve some of the issue, it's still not able
to cross-build everything and I didn't have time to look at it.

I could start the guix-daemon in darnassus for you to work on, by just
building guix as a user and using ./pre-inst-env, but I am afraid it
will create problems to darnassus. And Richard is using it for testing
things on the Hurd.

May I suggest giving you access to a local Hurd machine I have here so
you can hack as much as you want without worrying about side effects?

Manolis

bug#27476: guix pull fails on powerful server

[Ludovic Courtès]

Hi!

Ricardo Wurmus  skribis:

> The following derivation will be built:
>/gnu/store/z5bhk17nxmdhvj0g4cy038p25mzh1gys-guix-latest.drv
> copying and compiling to 
> '/gnu/store/s3s7xlqa10mvf8v0ypxz8gzw3lcf1x5z-guix-latest' with Guile 2.2.2...
> loading...   25.7% of 635 filesrandom seed for tests: 1506720257
> loading...   99.8% of 635 files
> compiling... 69.1% of 635 filesice-9/threads.scm:289:22: In procedure 
> loop:
> ice-9/threads.scm:289:22: Syntax error:
> guix/scripts/graph.scm:103:10: return: return used outside of 'with-monad' in 
> form (return (package-node-edges a))

The program below crashes with completely surreal backtraces in less
than a minute on my 4-thread laptop:

--8<---cut here---start->8---
(use-modules (ice-9 threads)
 (srfi srfi-1)
 (guix monads)
 (guix store))

(define threads
  (unfold (lambda (x) (> x 100))
  (lambda (x)
(call-with-new-thread
 (lambda ()
   (define monad
 (symbol-append 'foo-monad
(string->symbol (number->string x

   (while #t
 (macroexpand
  `(begin
 (define-monad ,monad
   (bind +)
   (return -))
 (with-monad ,monad
   (return 3))
 (mapm ,monad + '(1 2 3
  1+
  0))

(for-each join-thread threads)
--8<---cut here---end--->8---

Can you check if that also happens on your many-core machine?

The patch below seems to fix the problem: (guix monads) has shared state
(hash tables) used both at expansion-time and run-time, and it wasn’t
protected.

My hypothesis is that this was causing random memory corruption.  The
weird thing, though, is that the errors we were getting were not so
random.  Also, the load phase of ‘guix pull’ is sequential.

Could you test it and report back?

Thanks,
Ludo’.

diff --git a/guix/monads.scm b/guix/monads.scm
index 6ae616aca..c9c5da3bb 100644
--- a/guix/monads.scm
+++ b/guix/monads.scm
@@ -20,6 +20,7 @@
   #:use-module ((system syntax)
 #:select (syntax-local-binding))
   #:use-module (ice-9 match)
+  #:use-module (ice-9 threads)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-9)
   #:use-module (srfi srfi-26)
@@ -117,6 +118,7 @@
   ;; the syntax object of the parameter over which it is templated, and (2)
   ;; the syntax of its body.
   (define-once %templates (make-hash-table))
+  (define-once %template-lock (make-mutex))
 
   (define (register-template! name param body)
 (hash-set! %templates name (cons param body)))
@@ -139,8 +141,9 @@ template instances."
   (syntax-source s))
 
 (define current-info-port
-  ;; Port for debugging info.
-  (const (%make-void-port "w")))
+  ;; Port for debugging info.  Return a fresh port at each call to make
+  ;; sure we're thread-safe.
+  (lambda () (%make-void-port "w")))
 
 (define location-string
   (format #f "~a:~a:~a"
@@ -204,12 +207,14 @@ template instances."
;; Search for an instance of template NAME for this ACTUAL parameter.
;; On success, expand to the identifier of the instance; otherwise
;; expand to #f.
-   (any (matching-instance? #'name #'actual) %template-instances))
+   (with-mutex %template-lock
+ (any (matching-instance? #'name #'actual) %template-instances)))
   ((_ exists? name actual)
;; Likewise, but return a Boolean.
(let ((result (->bool
-  (any (matching-instance? #'name #'actual)
-   %template-instances
+  (with-mutex %template-lock
+(any (matching-instance? #'name #'actual)
+ %template-instances)
  (unless result
(format (current-warning-port)
"~a: warning: no specialization of template '~a' for '~a'~%"
@@ -220,8 +225,9 @@ template instances."
;; Expand to the definitions of all the existing templates
;; specialized for ACTUAL.
#`(begin
-   #,@(hash-map->list (cut instance-definition <> <> #'actual)
-  %templates))
+   #,@(with-mutex %template-lock
+(hash-map->list (cut instance-definition <> <> #'actual)
+%templates)))
 
 (define-syntax define-template
   (lambda (s)

bug#28756: Substitute download progress bar doesn't reach 100%

[宋文武]

l...@gnu.org (Ludovic Courtès) writes:

> Hello,
>
> Leo Famulari  skribis:
>
>> I just upgraded to the latest Guix and, while downloading substitutes, I
>> noticed the progress bar seems to never reach 100%, as shown below. I'm
>> not sure if this started recently or not.
>>
>> Downloading 
>> https://mirror.hydra.gnu.org/guix/nar/gzip/drm4pj1k5mkb5784i0rkqb0bg2z8lmyw-libabw-0.1.1...
>>  libabw-0.1.1  346KiB
>
> Indeed, I’ve noticed too, and I think it relates to the new progress
> reporters.  宋文武, do you experience this as well?  Thoughts?
>

Yes, thanks for the report, commit abaee53c8 should fix it.

I thought close the 'input' port in the 'progress-substitution' will
close the 'progress' port too, but it's not true.

bug#28243: Guix cross-compilation doesn't work

[Ricardo Wurmus]

Hi Manolis,

> May I suggest giving you access to a local Hurd machine I have here so
> you can hack as much as you want without worrying about side effects?

thanks, that’s a very kind offer.  I’d appreciate it.
Attached is my SSH public key.



id_rsa.hurd.pub
Description: Binary data

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net