Re: preparing the next release v0.15.0

[Ludovic Courtès]

Hi!

Ricardo Wurmus  skribis:

>> Can we aim for a v1.0 release? At least for the package manager? I am
>> facing people that say Guix is not mature and therefore insecure and
>> therefore they can justifiably ignore my pleas for installation
>> without even looking.
>
> I would like to leave 1.0 for the future.
>
> Things that I’d like to see in 1.0 include a new manifest format that
> records more information about the version of Guix, retiring of
> GUIX_PACKAGE_PATH, and more.

Ditto.  In terms of programming effort, we’re very close to what we’ve
been targeting for 1.0.

Ludo’.

Re: preparing the next release v0.15.0

[Ludovic Courtès]

Hello!

The plan looks good to me!

Ricardo Wurmus  skribis:

> * breakage since the last core-updates merge.  Since the long overdue
>   merge of the “core-updates” branch we have a few serious problems on
>   i686 (e.g. the gst-plugins-base package) and armhf.  Who would like to
>   lead an effort to investigate and fix these problems?  (Note: you
>   obviously don’t need to fix them yourself, just keep track of these
>   things and coordinate with others who investigate them.)
>
> * collecting new features and big changes in NEWS.  The NEWS file
>   already lists some of the changes that the next release would provide
>   compared to 0.14.0, but we all know that there are more.  Who would
>   like to take charge of the NEWS file?  This involves asking people for
>   changes they would really like to be mentioned and coordinating with
>   other volunteers to search through all commits since v0.14.0 to find
>   noteworthy changes.

Agreed with all this.  :-)

> * Finding a date.  Some of the fixes that need to be included may have
>   to end up on core-updates.  According to my calendar core-updates will
>   be frozen on <2018-08-06 Mon> and be merged on <2018-08-20 Mon> at the
>   latest.  That’s a little late, so if we can solve problems on the
>   master branch with architecture-specific workarounds that don’t result
>   in rebuilding the world (in addition to proper fixes on core-updates)
>   that would be best.  What do you think: is July 27 (2018) realistic?

I would prefer an earlier date even, perhaps around June 30.

WDYT?

Thanks,
Ludo’.

Re: Chrooting into GuixSD

[swedebugia]

On 2018-06-14 17:01, Ricardo Wurmus wrote:

Oleg Pykhalov  writes:


Would any of you like to document this in the manual?

I would because I posted articles like the current one sometime ago.

Excellent!


Maybe a separate ‘users-stories.texi’ should be as was discussed
somewhere on a mailing list (as I remember)?

I think that this particular section on tools that Guix provides to
recover a broken system (and the peculiarities of chrooting into a
GuixSD system) can go right into the manual.  This does not preclude the
option to have a more detailed step by step guide in a
“user-stories.texi”.

+1

Does GuixSD by the way support dropping to a root shell by passing 
"single" as a kernel parameter? I once used a distro that had a feature 
like that.


Ubuntu supports dropping to a root shell (and memtesting RAM?) among 
more options if "recovery" is choosen in the grub menu. I like that 
feature. If something like that had been possible on this old 
GuixSD-installation I would not have had to resort to chrooting.


--
---
Swedebugia

Re: preparing the next release v0.15.0

[Ricardo Wurmus]

Hi Ludo,

> The plan looks good to me!
>
> Ricardo Wurmus  skribis:
>
>> * breakage since the last core-updates merge.  Since the long overdue
>>   merge of the “core-updates” branch we have a few serious problems on
>>   i686 (e.g. the gst-plugins-base package) and armhf.  Who would like to
>>   lead an effort to investigate and fix these problems?  (Note: you
>>   obviously don’t need to fix them yourself, just keep track of these
>>   things and coordinate with others who investigate them.)
>>
>> * collecting new features and big changes in NEWS.  The NEWS file
>>   already lists some of the changes that the next release would provide
>>   compared to 0.14.0, but we all know that there are more.  Who would
>>   like to take charge of the NEWS file?  This involves asking people for
>>   changes they would really like to be mentioned and coordinating with
>>   other volunteers to search through all commits since v0.14.0 to find
>>   noteworthy changes.
>
> Agreed with all this.  :-)

Excellent!  Now we just need volunteers who would like to help with
these tasks :)

I’d like to reiterate that none of these tasks should be done by one
person alone, but we do need someone for each task to keep track of
progress or to report on hurdles.  So if anyone would like to help out,
please do respond here.

>> * Finding a date.  Some of the fixes that need to be included may have
>>   to end up on core-updates.  According to my calendar core-updates will
>>   be frozen on <2018-08-06 Mon> and be merged on <2018-08-20 Mon> at the
>>   latest.  That’s a little late, so if we can solve problems on the
>>   master branch with architecture-specific workarounds that don’t result
>>   in rebuilding the world (in addition to proper fixes on core-updates)
>>   that would be best.  What do you think: is July 27 (2018) realistic?
>
> I would prefer an earlier date even, perhaps around June 30.

June 30 sounds good to me.

--
Ricardo

Re: Chrooting into GuixSD

[Ludovic Courtès]

Hello,

Thorsten Wilms  skribis:

> You can interact with the daemon:
>
>   cd /path/to/guix/root
>   sudo mount --bind /dev dev
>   sudo mount --bind /proc proc
>   sudo mount --bind /sys sys
>   sudo unshare -m sudo chroot . /run/current-system/profile/bin/bash
>   source /etc/profile
>
>   guix-daemon --build-users-group=guixbuild &
>
> If you then `su` to a plain user, do `source /etc/profile` again.
>
> I think all of `guix pull`. `reconfigure` and `package` search,
> install and update worked here with that approach. `guix download`
> failed due to nss certificates, until I added:
>   export SSL_CERT_DIR="$HOME/.guix-profile/etc/ssl/certs"
>   export
> SSL_CERT_FILE="$HOME/.guix-profile/etc/ssl/certs/ca-certificates.crt"
>   export GIT_SSL_CAINFO="$SSL_CERT_FILE"

I haven’t tried, but something that would be equivalent to booting into
the system is:

  chroot /path/to/guixsd
  /gnu/store/*-guile-2.0.3/bin/guile /run/current-system/boot

It may interfere badly with the running system (like when it tries to
bring up ttys), but maybe it’s OK?

Ludo’.

Re: Chrooting into GuixSD

[Thorsten Wilms]

On 15.06.2018 17:01, Ludovic Courtès wrote:

I haven’t tried, but something that would be equivalent to booting into
the system is:

   chroot /path/to/guixsd
   /gnu/store/*-guile-2.0.3/bin/guile /run/current-system/boot

It may interfere badly with the running system (like when it tries to
bring up ttys), but maybe it’s OK?


Well, I tried:

  cd /path/to/guixsd
  chroot . /run/current-system/profile/bin/bash
  source /etc/profile
  guile /run/current-system/boot

---
guile: warning: failed to install locale
GC Warning: pthread_getattr_np or pthread_attr_getstack failed for main 
thread

GC Warning: Couldn't read /proc/stat
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;   or pass the --no-auto-compile argument to disable.
;;; compiling /run/current-system/boot
;;; compiled 
/home/thorwil/.cache/guile/ccache/2.2-LE-8-3.A/gnu/store/8m935f33cg4qzlr9d44ibqmcrwn5c1r4-boot.go

Backtrace:
  12 (apply-smob/1 #)
In ice-9/boot-9.scm:
705:2 11 (call-with-prompt _ _ #)
In ice-9/eval.scm:
619:8 10 (_ #(#(#)))
In ice-9/boot-9.scm:
   2312:4  9 (save-module-excursion _)
  3822:12  8 (_)
In /run/current-system/boot:
1:761  7 (_)
In unknown file:
   6 (primitive-load "/gnu/store/llka25n1x5l1nj82mr7xs8pi1if?")
In ice-9/eval.scm:
619:8  5 (_ #f)
In ./gnu/build/activation.scm:
   439:33  4 (activate-current-system _)
   435:31  3 (boot-time-system)
In ./gnu/build/linux-boot.scm:
 90:3  2 (linux-command-line)
In ice-9/ports.scm:
   439:11  1 (call-with-input-file "/proc/cmdline" # ?)
In unknown file:
   0 (open-file "/proc/cmdline" "r" #:encoding #f # #f)

ERROR: In procedure open-file:
In procedure open-file: No such file or directory: "/proc/cmdline"
---

I _guess_ if I'd bind proc, I would then find out dev and sys has to be 
taken care of, too, leading back to what I had already.



--
Thorsten Wilms

thorwil's design for free software:
http://thorwil.wordpress.com/

Re: Restarting 'staging'

[Ricardo Wurmus]

Hi Marius,

> Now that 'core-updates' is merged, it's time to get 'staging' rolling
> again.  We have lots of minor updates this round, mostly on the
> graphical side.  Is there anything we're missing that's eligible (<1200
> rebuilds)?

The ant-build-system needs to be modified to raise an exception on test
failures.  Currently it uses the old (zero? (system* …)) idiom, which
causes failing tests to be ignored.

This will cause all Java packages and packages depending on them to be
rebuilt.

--
Ricardo

Re: Fwd: Re: Patch file for colorize module

[Sahitihi]

Hi Ricardo,

I have fixed the errors mentioned bellow, Please do check and notify me
for further modifications.
>>> The next steps are to fix the errors in the regular expressions that
>>> swallow certain space characters and to re-read the documentation of
>>> “make-soft-port” to make sure that the procedures in the vector behave
>>> as they should.
>> I will update back once i am through with it.
I have added the comment lines. Patch file is attached. I applied indent
script please do not worry about that. :-)
> After reading the documentation for “make-soft-port” again, I recommend
> adding very short comments above each of the procedures in the vector to
> explain what they are intended to do.
>
Please do inform other modifications I have to do and next tasks to
proceed further.

---
Thanks!!
Sahithi.
From 2d4a335d0262e59784912b6cd922ae9283f17cb4 Mon Sep 17 00:00:00 2001
From: Sahithi Yarlagadda 
Date: Wed, 13 Jun 2018 02:08:27 +0530
Subject: [PATCH] guix: Add coloring soft port.

* guix/ui.scm (handle-string): New procedures.
(colorful-build-output-port): New variable.

guix: Added colorful-build-output-port to build.scm instead of default
---
 guix/scripts/build.scm |   4 +-
 guix/ui.scm| 101 -
 2 files changed, 75 insertions(+), 30 deletions(-)

diff --git a/guix/scripts/build.scm b/guix/scripts/build.scm
index 4dd4fbccd..f6924874d 100644
--- a/guix/scripts/build.scm
+++ b/guix/scripts/build.scm
@@ -733,9 +733,7 @@ needed."
 ;; Set the build options before we do anything else.
 (set-build-options-from-command-line store opts)
 
-(parameterize ((current-build-output-port (if quiet?
-  (%make-void-port "w")
-  (current-error-port
+   (parameterize ((current-build-output-port  colorful-build-output-port))
   (let* ((mode  (assoc-ref opts 'build-mode))
  (drv   (options->derivations store opts))
  (urls  (map (cut string-append <> "/log")
diff --git a/guix/ui.scm b/guix/ui.scm
index 80f1a4d77..5ddc7c959 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -109,7 +109,7 @@
 warning
 info
 guix-main
-colorize-string))
+colorful-build-output-port))
 
 ;;; Commentary:
 ;;;
@@ -201,8 +201,8 @@ information, or #f if it could not be found."
   "Load the user provided Scheme source code FILE."
   (define (error-string frame args)
 (call-with-output-string
-  (lambda (port)
-(apply display-error frame port (cdr args)
+ (lambda (port)
+   (apply display-error frame port (cdr args)
 
   (define tag
 (make-prompt-tag "user-code"))
@@ -490,17 +490,17 @@ FILE."
 (augmented-system-error-handler file)
 
 (set! symlink
-  ;; We 'set!' the global binding because (gnu build ...) modules and similar
-  ;; typically don't use (guix ui).
-  (error-reporting-wrapper symlink (source target) target))
+  ;; We 'set!' the global binding because (gnu build ...) modules and similar
+  ;; typically don't use (guix ui).
+  (error-reporting-wrapper symlink (source target) target))
 
 (set! copy-file
-  ;; Note: here we use 'set!', not #:replace, because UIs typically use
-  ;; 'copy-recursively', which doesn't use (guix ui).
-  (error-reporting-wrapper copy-file (source target) target))
+  ;; Note: here we use 'set!', not #:replace, because UIs typically use
+  ;; 'copy-recursively', which doesn't use (guix ui).
+  (error-reporting-wrapper copy-file (source target) target))
 
 (set! canonicalize-path
-  (error-reporting-wrapper canonicalize-path (file) file))
+  (error-reporting-wrapper canonicalize-path (file) file))
 
 
 (define (make-regexp* regexp . flags)
@@ -898,10 +898,10 @@ replacement if PORT is not Unicode-capable."
 (catch 'encoding-error
   (lambda ()
 (call-with-output-string
-  (lambda (port)
-(set-port-encoding! port encoding)
-(set-port-conversion-strategy! port 'error)
-(display arrow port
+ (lambda (port)
+   (set-port-encoding! port encoding)
+   (set-port-conversion-strategy! port 'error)
+   (display arrow port
   (lambda (key . args)
 "->"
 
@@ -1086,7 +1086,7 @@ converted to a space; sequences of more than one line break are preserved."
   ;; 'texi-fragment->stexi' uses a string port so make sure it's a
   ;; Unicode-capable one (see .)
   (with-fluids ((%default-port-encoding "UTF-8"))
-(stexi->plain-text (texi-fragment->stexi str
+   (stexi->plain-text (texi-fragment->stexi str
 
 (define (package-field-string package field-accessor)
   "Return a plain-text representation of PACKAGE field."
@@ -1338,10 +1338,10 @@ DURATION-RELATION with the current time."
   ;; Return TIME at midnight

Next steps

[Ricardo Wurmus]

Hi Sahithi,

> I have fixed the errors mentioned bellow, Please do check and notify me
> for further modifications.

Excellent, this looks better.  Unfortunately, your patch includes a
couple of unrelated indentation changes.  Please remove those.

Let’s talk about the next steps.


1) Enabling colours only *optionally*.

We modified “guix/scripts/build.scm” to test the changes, but this
really should be optional, because not everybody wants colours.

For example, when the environment variable “NO_COLOR” is set to any
value we should not use colours.  Likewise, when the environment
variable “INSIDE_EMACS” has a value (meaning that this is a shell in
Emacs where people should rather use “guix-build-log-minor-mode”) we
should not print colours.  You can read the values of environment
variables with the “getenv” procedure (it is explained in the manual).

This check could be in “guix/scripts/build.scm” and
“guix/scripts/package.scm” to either use “colorful-build-output-port” or
“(current-error-port)”.

Another option is to do this in the definition of
“colorful-build-output-port” itself, which would either be defined with
“handle-string” or “handle-plain-string” (which uses no colours)
dependent on these environment variables.

This option would be better, because we want to extend the soft port to
also filter lines optionally (when “guix package” is used).  Instead of
swapping out the full port we could configure it with various options
like “color?” and “filter?”.

Time estimate: This should take no longer than 3 days.


2) Fixing the other soft port procedures

Currently we have this:

> +   (vector
> +(lambda (c) (write c (current-error-port)))
> +;; procedure accepting one character for output
> +handle-string
> +;; procedure accepting a string for handle-string procedure
> +(lambda () (display " " (current-error-port)))
> +(lambda () (char-upcase (read-char)))
> +(lambda () (display "@" (current-error-port

Please move the comments *above* the procedures they describe.  Note
that the third procedure is still wrong – according to the manual it
should force or flush the output, not print a space.  Please use
“(lambda () (force-output (current-error-port)))” instead.

The fourth procedure reads a character.  Currently it also turns the
read character to an uppercase character.  We don’t need that, because
we don’t read from the port at all.  You can use “(const #t)” instead.

Time estimate: This should take less than 10 minutes to fix.


3) Filtering all lines between “starting phase” and “phase
succeeded/failed”.

Currently, the port applies colours and passes all other lines through
unmodified.  When using “guix package” it may be good to *hide* other
lines and replace them with a progress indicator.  The first step to
test this would be to replace *any* other line with “.”.  You would then
only see the lines that announce phases, but not the build output.

(Later we would change the dots for a cute little “spinner” animation.)

Can you think of a way to *only* filter lines when “guix package” is
used but not when “guix build” is used?  Maybe we need another variation
of the port…?

Time estimate: You should have some results and an idea how to finish
this after no more than a day of work.


4) Making the colorization prettier.

We repeat “colorize-string” a lot!  Can we do better?  How about using
“match:count” and a list of colours to be applied in order?

Start playing with something like this in the REPL:

--8<---cut here---start->8---


(use-modules (ice-9 match)   ; need this for “match-lambda”
 (srfi srfi-1))  ; need this for “any”

(define str "phase foo failed after 100 seconds")


;; Each list item in “patterns” is a regular expression followed by a
;; number of colours.
(let ((patterns '(("^(starting phase )(.*)"
   BLUE GREEN)
  ("^(phase)(.*)(succeeded after)(.*)(seconds)"
   GREEN BLUE GREEN BLUE GREEN)
  ("^(phase)(.*)(failed after)(.*)(seconds)"
   RED BLUE RED BLUE RED

   ;; See if “any” of the patterns matches, i.e. returns a string and
   ;; not just #f.  We use “match-lambda” to bind the pattern to the
   ;; variable “pattern” and the list of colours to the variable
   ;; “colors”.

   (or (any (match-lambda
 ((pattern . colors)
  ;; TODO: use string-match, match:count, match:substring,
  ;; colorize-string, and=>, etc to see if a pattern matches
  ;; and to transform the string according to “colors”.

  ;; If the pattern does not match return #f to
  ;; automatically try the next, thanks to “any”.
  #f
 ))
 patterns)
   ;; Nothing matched, so return the string without changes.
   str))
--8<---cut here---end--->8---

Take your time to understand this one.  Play around wit