Re: A Critique of Shepherd Design

[raid5atemyhomework]

Good rmoning Mark,

> Hi,
>
> raid5atemyhomework raid5atemyhomew...@protonmail.com writes:
>
> > GNU Shepherd is the `init` system used by GNU Guix. It features:
> >
> > -   A rich full Scheme language to describe actions.
> > -   A simple core that is easy to maintain.
> >
> > However, in this critique, I contend that these features are bugs.
> > The Shepherd language for describing actions on Shepherd daemons is a
> > Turing-complete Guile language. Turing completeness runs afoul of the
> > Principle of Least Power. In principle, all that actions have to do
> > is invoke `exec`, `fork`, `kill`, and `waitpid` syscalls.
>
> These 4 calls are already enough to run "sleep 1000" and wait
> for it to finish, or to rebuild your Guix system with an extra patch
> added to glibc.

I agree.  But this mechanism is intended to avoid stupid mistakes like what I 
committed, not protect against an attacker who is capable of invoking `guix 
system reconfigure` on arbitrary Scheme code (and can easily wrap anything 
nefarious in any `unsafe-turing-complete` or `without-static-analysis` escape 
mechanism).  Seatbelts, not steel walls.

>
> > Yet the language is a full Turing-complete language, including the
> > major weakness of Turing-completeness: the inability to solve the
> > halting problem.
> > The fact that the halting problem is unsolved in the language means it
> > is possible to trivially write an infinite loop in the language. In
> > the context of an `init` system, the possibility of an infinite loop
> > is dangerous, as it means the system may never complete bootup.
>
> Limiting ourselves to strictly total functions wouldn't help much here,
> because for all practical purposes, computing 10^100 digits of Pi is
> just as bad as an infinite loop.

Indeed.  Again, seatbelts, not steel walls.  It's fairly difficult to commit a 
mistake that causes you to accidentally write a program that computes 10^100 
digits of pi, not so difficult to have a brain fart and use `(- count 1)` 
instead of `(+ count 1)` because you were wondering idly whether an increment 
or a decrement loop would be more Scemey or if both are just as Schemey as the 
other.

What I propose would protect against the latter (a much more likely mistake), 
as in-context the recursive loop would be flagged since the recursion would be 
flagged due to being a call to a function that is not a member of a whitelist.  
Hopefully getting recursive loops flagged would make the sysad writing 
`configuration.scm` look for the "proper" way to wait for an event to be true, 
and hopefully lead to them discovering the (hopefully extant) documentation on 
whatever domain-specific language we have for waiting for the event to be true 
instead of rolling their own.

> That said, I certainly agree that Shepherd could use improvement, and
> I'm glad that you've started this discussion.
>
> At a glance, your idea of having Shepherd do more within subprocesses
> looks promising to me, although this is not my area of expertise.

An issue here is that we sometimes pass data across Shepherd actions using 
environment variables, which do not cross process boundaries.  Xref. the 
`set-http-proxy` of `guix-daemon`; the environment variable is used as a global 
namespace that is accessible from both the `set-http-proxy` and `start` actions.

On the other hand, arguably the environment variable table is a global resource 
shared amongst multiple shepherd daemons.  This technique in general may not 
scale well for large numbers of daemons; environment variable name conflicts 
may cause subtle problems later.  I think it would be better if in addition to 
the "value" (typically the PID) each Shepherd service also had a `settings` 
(which can be used to contain anything that satisfies `(lambda (x) (equal? x 
(read (print x` so that it can be easily serialized across each subprocess 
launched by each action) that can be read and modified by each action.  Then 
the `set-http-proxy` action would update this `settings` field for the shepherd 
service, then queue up a `restart` action.  It could by convention be an 
association list.

This would also persist the `http_proxy` setting, BTW --- currently if you 
`herd set-http-proxy guix-daemon ` and then `herd restart 
guix-daemon` later, the HTTP proxy is lost (since the environment variable is 
cleared after `set-http-proxy` restarts the `guix-daemon`).  In short, this 
`set-http-proxy` example looks like a fairly brittle hack anyway, and maybe 
worth avoiding as a pattern.

Then there's actions that invoke other actions.  From a cursory glance at the 
Guix code it looks like only Ganeti and Guix-Daemon have actions that invoke 
actions, and they only invoke actions on their own Shepherd services.  It seems 
to me safe for an action invoked in another action of the same service to *not* 
spawn a new process, but to execute as the same process.  Not sure how safe it 
would be to allow one shepherd service to invoke an action 

Re: [opinion] CVE-patching is not sufficient for package security patching

[Ludovic Courtès]

Hi,

Mark H Weaver  skribis:

> Ultimately, I gave up.  In my opinion, Guix has never achieved usability
> as a desktop system on non-Intel systems.  Therefore, the Guix community
> is unable to attract many developers who want a distro that supports
> non-Intel systems well.  Our community has thus become dominated by
> Intel users, and there's unsufficient political will to adopt policies
> that would enable us to provide a usable system for non-Intel users.

A practical problem that’s been mentioned repeatedly is lack of ci.guix
hardware for non-Intel architectures: please everyone, consider helping
the project find either sponsors or companies that sell fitting
hardware, along with a plan to host it and maintain it over time!

Ludo’.

Re: Fedora/Debian release monitoring inspiration for Guix Data Service

[Ludovic Courtès]

Hi,

Léo Le Bouter  skribis:

> It seems GNU Guix takes a generic approach to updates while Debian or
> Fedora seems to look at specialized rules for each package, I was
> thinking we could import those already existing rules (in Fedora's or
> Debian's) into GNU Guix, I find it a superior approach for reliability
> of notifications even though generic updaters are also very
> complementary.

The updaters in Guix are fairly specialized; only ‘generic-html’ is
generic.

Do take a closer look; it’s an area of the code that’s fun to work with!

Ludo’.

Re: Are gzip-compressed substitutes still used?

[Ludovic Courtès]

Vagrant Cascadian  skribis:

> On 2021-03-17, Léo Le Bouter wrote:
>> Just as a reminder siding with vagrantc here:
>>
>> We must ensure the Debian 'guix' package can still work and upgrade
>> from it's installed version, so ensure that removing gzip doesnt break
>> initial 'guix pull' with it.
>
> ... and I would expect this version to ship in Debian for another ~3-5
> years, unless it gets removed from Debian bullseye before the upcoming
> (real soon now) release!
>
> But if lzip substitutes are still supported, I *think* guix 1.2.0 as
> packaged in Debian still supports that, at least.
>
> Dropping both gzip and lzip would be unfortunate; I don't think it would
> be trivial to backport the zstd patches to guix 1.2.0, as it also
> depends on guile-zstd?

Indeed.  But don’t worry: we wouldn’t drop both gzip and lzip at once!

Ludo’.

Re: Why [bug#47081] Remove mongodb?

[Ludovic Courtès]

Hi Léo,

Léo Le Bouter  skribis:

> On Wed, 2021-03-17 at 17:56 +0100, zimoun wrote:
>> If the removal for security reasons had been discussed on IRC, it
>> could
>> be nice to point the discussion here.  Otherwise, open a discussion
>> on
>> the topic on guix-devel or bug-guix.  The full removal is a radical
>> solution (especially, it should be done with 2 commits: service+doc
>> and
>> then package; well another story).
>
> https://issues.guix.gnu.org/47081 - some of it there: 

Removing a package and its services is not something to do lightly: it
breaks user configs with no recourse.

We must insist on getting more opinions on such matters, and I think
there just wasn’t enough feedback here.  I understand it can be
frustrating to wait for input, but in such a case, please do.  This
project has always strove for consensus.

Remember that the opinion of those who’ve been taking care of security
issues in Guix for years, those who’ve been maintaining MongoDB, those
who wrote the service and its tests, are invaluable; they must have a
say.  I insist: humbly solicit and wait for their feedback.

Now, how do we move forward?  IMO we must look for available options
before we remove MongoDB.  Are there forks of the original
freely-licensed code base maintained around?  That sounds likely.  Are
there backports of the security fixes?  What do the previous
contributors to this code think—Chris, Efraim, Marius, Arun?

Léo, please get involved in reaching consensus on a solution.

Ludo’.

Make guix commands pipeable

[pkill9]

I would like to be able to pipe files into guix commands.

Specifically the `guix system build` command, so I can build a system
configuration on a remote Guix system over SSH, i.e. `cat config.scm |
ssh  guix system build -`, or perhaps using the
`--expression` flag which would make more sense, e.g. `cat config.scm |
ssh  guix system build -e -`.

While you can currently just copy over a file and then use that, it
would be a little cleaner and more stateless to use a pipe.

What do other people think?

Re: Finding the store path of a package

[Ludovic Courtès]

Hi!

Konrad Hinsen  skribis:

>>> does “guix build  -n” fit your use-case?
>>
>> Checking... yes! I hadn't even considered using "build" when building is
>> exactly what I do not want to happen. But yes, it works just fine.
>
> Not quite:
>
> $ guix build -n zziplib
> substitute: 
> /gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bash: 
> warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
> 24,6 MB would be downloaded:
>/gnu/store/9h9153akbd6g2520mcl2d0vgbvyfi5qm-openssl-1.1.1i
>/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31
>/gnu/store/s3dcqzwqaakv1yx37by9chksdbkgih17-glibc-2.31-static
>/gnu/store/fbn395nfpbp4d4fr6jsbmwcx6n10kg16-python-minimal-3.8.2
>/gnu/store/7hizrpdsqf6q3pjgzmi51r5vbzlijkw0-python-minimal-wrapper-3.8.2
>
> My first impression is that "guix build -n" shows the store path of the
> package only if all dependencies are in the store. Otherwise, it shows
> the dependencies.

Yes.  In the presence of grafts, run “guix build PKG”.  That always
gives you the store file name of PKG, 100% reliable!

I regularly do things like:

  ls $(guix build PKG)/bin
  find $(guix build PKG) -name …

But note that ‘guix build’ returns one file name per output, so in case
of multiple-output packages, you have to pipe its output through ‘head’,
‘grep’, or similar.

If you want a variant that does that without building/downloading it,
it’s also possible, though not as easily from the command line.

Ludo’.

Re: A Critique of Shepherd Design

[Ludovic Courtès]

Hi,

raid5atemyhomework  skribis:

> Now, let us combine this with the second feature (really a bug): GNU
> shepherd is a simple, single-threaded Scheme program.  That means that
> if the single thread enters an infinite loop (because of a Shepherd
> service description that entered an infinite loop), then Shepherd
> itself hangs.

You’re right that it’s an issue; in practice, it’s okay because we pay
attention to the code we run there, but obviously, mistakes could lead
to the situation you describe.

It’s a known problem and there are plans to address it, discussed on
this list a few times before.  The Shepherd “recently” switched to
‘signalfd’ for signal handling in the main loop, with an eye on making
the whole loop event-driven:

  https://issues.guix.gnu.org/41507

This will address this issue and unlock things like “socket activation”.

That said, let’s not lie to ourselves: the Shepherd’s design is
simplistic.  I think that’s okay though because there’s a way to address
the main issues while keeping it simple.

Thanks,
Ludo’.

Re: Discrepancy between manual and channels.scm

[Ludovic Courtès]

Hi,

ison  skribis:

> Under "6.7 Declaring Channel Dependencies" the examples show names being
> quoted, such as (name 'some-collection) however this causes guix pull to
> fail. I don't remember the exact error but it was failing the "match"
> statement under "resolve-dependencies" in "guix/channels.scm".
>
> (name "some-collection") also fails.
> (name some-collection) succeeds, without any quoting.

Oops you’re right; I’ll push a fix to the manual shortly.

> I'm not sure if it's the manual which needs fixing or the code, but
> specifying the name without quoting would be inconsistent with how
> channel names are declared in ~/.config/guix/channels.scm.

It’s necessarily different because ‘.guix-channel’ is data whereas
‘channels.scm’ is code.

Thanks!

Ludo’.

Re: Release 1.2.1: status

[Leo Famulari]

On Fri, Mar 19, 2021 at 09:50:55AM +0100, zimoun wrote:
> The release work happens on master.  The branch wip-next-release
> contains fixes, but AFAIK, it is not built by the CI, and these fixes
> are ’core-updates’-like changes; I do not know if it is doable to merge
> on time.

I agree. The scope of ungrafting has grown way too large, and we need to
spend the rest of the time fixing bugs.

Maybe there are some things we could ungraft in time, or that are
failing to work properly as grafts (ImageMagick / broken Inkscape?) and
should thus be ungrafted or somehow adjusted.

>From the wip-next-release branch, we should cherry-pick the tzdata
updates and Qt 4 removal.

I'll rewrite the branch with those commits today, and then see about
getting it built on CI.

Re: gnu: imagemagick/fixed: Redirect old sonames to new sonames.

[Leo Famulari]

On Fri, Mar 19, 2021 at 08:14:03PM -0400, Mark H Weaver wrote:
> Leo Famulari  writes:
> 
> > On Thu, Mar 18, 2021 at 09:40:04AM -0400, Mark H Weaver wrote:
> >> I knew this couldn't be right, but I thought I remembered it having
> >> fewer dependencies.  Oh well.  Sorry for the noise.
> >
> > It's relatively new that ImageMagick is depended on by so many packages.
> > I think we should look into this and see if we can significantly reduce
> > the number.
> 
> The following dependency chain seems to be responsible for most of the
> imagemagick-dependent packages:
> 
> gtk+@3 -> at-spi2-atk -> at-spi2-core -> gtk-doc -> dblatex -> imagemagick

Thanks, that's useful.

But, I think this dependency chain has existed for a long time (notes
about this at the end of my message).

I wonder if there is some newer change that has increased the number of
ImageMagick dependents. Maybe it's just that GTK+ is depended on by a
lot of packages?

Chris (cc-ed), is this something that Guix Data Service can help us
answer?

To clarify, the question is, what changed in the package dependency
graph that caused so many packages to transitively depend on
ImageMagick? Previously, we could update ImageMagick on the master
branch.

--
Exploring the history of the dependency chain, using trusty `git log`:

Dblatex was added in 2014, with a dependency on imagemagick.

Gtk-doc was added in 2015, with a dependency on dblatex.

at-spi2-core became dependent on gtk-doc in 2018. at-spi2-atk has
depended on at-spi2-core since it was added 2013

And gtk+ has depended on at-spi2-atk since 2013.

Re: Make guix commands pipeable

[david larsson]

On 2021-03-20 13:13, pkill9 wrote:

I would like to be able to pipe files into guix commands.

Specifically the `guix system build` command, so I can build a system
configuration on a remote Guix system over SSH, i.e. `cat config.scm |
ssh  guix system build -`, or perhaps using the
`--expression` flag which would make more sense, e.g. `cat config.scm |
ssh  guix system build -e -`.

While you can currently just copy over a file and then use that, it
would be a little cleaner and more stateless to use a pipe.

What do other people think?


I like pipes and this would be nice to have. Though an easy way to wrap 
things in bash and make them pipable at the same time is to use the -t 
check like this:


f(){ [[ ! -t 0 ]] && mapfile -t Args ; if [[ -n "${Args[@]}" ]]; then 
printf 'some_command %s ' "${Args[@]}" ; else printf 'some_command %s ' 
"$@"; fi ; }

printf '%s\n' a b c | f
f a b c

So you could define f as: f(){ [[ ! -t 0 ]] && mapfile -t -d '' Args ; 
if [[ -n "${Args[@]}" ]]; then ssh  guix system build -e 
"${Args[@]//$'\n'/}" ; else ssh  guix system build -e 
"$@"; fi ; }


and then I think you should be able to cat config.scm | f

Notice -d '' above which means use null as delimiter which will most 
often put everything in index 0 of the list. Also notice the parameter 
expansion which removes all newlines from the content.


I do agree though that it would be nicer to just have '-' as a guix 
input arg causing any guix command to read from stdin.


I hope it helps!

Best regards,
David

Re: Release 1.2.1: status

[Leo Famulari]

I suggest we use debbugs to keep track of tasks for the release.

We can create a new bug called "1.2.1 release checklist".

This bug can be made to depend on other bugs using the "block" feature
of debbugs:

https://debbugs.gnu.org/server-control.html

Concretely, this means we send email to debbugs with messages like
"block NNN with YYY", which means that NNN cannot be closed until YYY is
closed.

What do you think? Should I create a new bug ticket for this?

Re: gnu: imagemagick/fixed: Redirect old sonames to new sonames.

[Mark H Weaver]

Hi Leo,

> On Fri, Mar 19, 2021 at 08:14:03PM -0400, Mark H Weaver wrote:
>> The following dependency chain seems to be responsible for most of the
>> imagemagick-dependent packages:
>> 
>> gtk+@3 -> at-spi2-atk -> at-spi2-core -> gtk-doc -> dblatex -> imagemagick

Leo Famulari  writes:
> To clarify, the question is, what changed in the package dependency
> graph that caused so many packages to transitively depend on
> ImageMagick? Previously, we could update ImageMagick on the master
> branch.

I think you've answered your own question, below.

> Exploring the history of the dependency chain, using trusty `git log`:
>
> Dblatex was added in 2014, with a dependency on imagemagick.
>
> Gtk-doc was added in 2015, with a dependency on dblatex.
>
> at-spi2-core became dependent on gtk-doc in 2018. at-spi2-atk has
> depended on at-spi2-core since it was added 2013
>
> And gtk+ has depended on at-spi2-atk since 2013.

As you noted, at-spi2-core became dependent on gtk-doc in August 2018.
That's when the chain above was completed.  My memory of 'imagemagick'
not having many dependents was from before then, I think.

Thanks for looking into it!

  Mark

Re: Release 1.2.1: status

[zimoun]

Hi Leo,

On Sat, 20 Mar 2021 at 14:09, Leo Famulari  wrote:
> On Fri, Mar 19, 2021 at 09:50:55AM +0100, zimoun wrote:
>> The release work happens on master.  The branch wip-next-release
>> contains fixes, but AFAIK, it is not built by the CI, and these fixes
>> are ’core-updates’-like changes; I do not know if it is doable to merge
>> on time.
>
> I agree. The scope of ungrafting has grown way too large, and we need to
> spend the rest of the time fixing bugs.

I agree.  For instance, I have started to see if zstd could be ungrafted
and with my knowledge, my time available considering the other parts to
check, and my CPU available to build a lot, IMHO, it is not possible to
be on time.

> Maybe there are some things we could ungraft in time, or that are
> failing to work properly as grafts (ImageMagick / broken Inkscape?) and
> should thus be ungrafted or somehow adjusted.

I have not check at ImageMagick specifically.  My feeling about the
recent grafts is that they are more than security fixes because they are
often version upgrades.  And version upgrade often implies breakage here
or there.  Therefore, I agree they need to be adjusted and maybe, IMHO,
we could have a gratf freeze starting on April 1rst in order to have
time to check that everything is fine, AFAWCT.

> From the wip-next-release branch, we should cherry-pick the tzdata
> updates and Qt 4 removal.
>
> I'll rewrite the branch with those commits today, and then see about
> getting it built on CI.

Do you mean cherry-pick and then push them to master?


Cheers,
simon

Re: Release 1.2.1: status

[zimoun]

Hi Leo,

On Sat, 20 Mar 2021 at 16:01, Leo Famulari  wrote:
> I suggest we use debbugs to keep track of tasks for the release.
>
> We can create a new bug called "1.2.1 release checklist".
>
> This bug can be made to depend on other bugs using the "block" feature
> of debbugs:
>
> https://debbugs.gnu.org/server-control.html
>
> Concretely, this means we send email to debbugs with messages like
> "block NNN with YYY", which means that NNN cannot be closed until YYY is
> closed.
>
> What do you think? Should I create a new bug ticket for this?

We discussed that and I agree.  We also discussed some tagging and Maxim
did some tests, IIRC.  Please go ahead and let try if it helps to
synchronize. :-)

Cheers,
simon

Re: Release 1.2.1: status

[zimoun]

Hi Andreas,

On Fri, 19 Mar 2021 at 19:31, Andreas Enge  wrote:
> Am Thu, Mar 18, 2021 at 03:28:38PM +0100 schrieb zimoun:
>>   guix weather --display-missing
>
> I am giving it a try, but after about one hour at 100% CPU on one core it
> is still only half way through. Is this normal? I think I will stop it to
> at least redirect the output into a file...

I sympathise.  It depends the architecture from my experience.  Some are
not responding at all.  Which one are you “guix weather”ing?

If your machine is always up, you can try:

  until guix weather; do echo ok; done

even it is not really reliable.


Cheers,
simon

Re: Release 1.2.1: status

[Leo Famulari]

On Sat, Mar 20, 2021 at 11:56:57PM +0100, zimoun wrote:
> > From the wip-next-release branch, we should cherry-pick the tzdata
> > updates and Qt 4 removal.
> >
> > I'll rewrite the branch with those commits today, and then see about
> > getting it built on CI.
> 
> Do you mean cherry-pick and then push them to master?

We can remove Qt 4 on master, but tzdata cannot be changed on the master
branch (too many dependents). We need to build it on CI, on its own
branch, and then deploy the change on master.

Re: Release 1.2.1: status

[zimoun]

Hi Luis,

Thanks for testings and reporting.


On Fri, 19 Mar 2021 at 22:26, Luis Felipe  wrote:
> ‐‐‐ Original Message ‐‐‐
> On Thursday, March 18, 2021 2:28 PM, zimoun  wrote:
>
> [...]
>
>> We are still missing a good story to monitor what is archived on
>> Software Heritage and what is not. Because for now there is rate limit,
>> I am not able to automate… Well, it is a long WIP. :-)
>>
>> To help and avoid this rate limit, if all of us simply run:
>>
>> for pkg in $(guix package -I | cut -f1);
>> do
>> guix lint -c archival $pkg
>> done
>>
>> for all our profiles, it will ensure at least a coverage for the
>> packages using git-fetch that we individually care. Note the option
>> --manifest for “guix lint” is missing… should happen soon if no one
>> beats me. :-)
>
> I tried this with my user profile and the script hit the Software
> heritage limit after reporting the status of 33 packages (most of
> which are not archived). So, for the rest of the packages, you are
> asked to try again later (I have 95 packages in my profile).

There is 2 rate limit: one for saving and one for requesting.

Each time you do “guix lint -c archival ”, Guix requests to SWH via
their API [1] if the package is already in.  AFAIR, it is 120 requests
per hour.

Then if it is not, Guix saves to SWH via their API.  And this rate is
very low, maybe 10 per hour.  Well, if I remember correctly.


1: 

> Is "guix lint -c archival $pkg" supposed to poke Software Heritage to
> archive the $pkg if it is not archived yet? I ask because I ran the
> script later and I got the same output from the first run, i.e.,
> packages reported to be missing from Software Heritage were still
> reported as such, instead of being planned for archive.

Currently, the request/save via “guix lint -c archival” is not optimal.
For instance, the source of the package “libvirt“ is url-fetch so
requesting for it is not necessary because it cannot be saved via their
API.  And I not remember exactly how the ‘tarball’ request is counted.

BTW, the packages using ’url-fetch’ should be ingested by SWH via their
nixguix loader reading the sources.json [2].  And for example:

--8<---cut here---start->8---
$ guix lint -c archival libvirt
gnu/packages/virtualization.scm:1070:5: libvirt@5.8.0: source not archived on 
Software Heritage

$ guix download -H sha256 -f base64 
https://libvirt.org/sources/libvirt-5.8.0.tar.xz

Starting download of /tmp/guix-file.XWwdFj
>From https://libvirt.org/sources/libvirt-5.8.0.tar.xz...
 libvirt-5.8.0.tar.xz  12.5MiB678KiB/s 00:19 [##] 100.0%
/gnu/store/1pgi1bl8p7jv2mhk83kv7raak2b4k1w5-libvirt-5.8.0.tar.xz
4jMoKJsYve3B6Wb2wmQCspgxScZg7YvVLNpv6rDCDFU=
--8<---cut here---end--->8---

and in the same time, the sources.json contains: 

--8<---cut here---start->8---
{
  "type": "url",
  "urls": [
"https://libvirt.org/sources/libvirt-5.8.0.tar.xz";
  ],
  "integrity": "sha256-4jMoKJsYve3B6Wb2wmQCspgxScZg7YvVLNpv6rDCDFU="
},
--8<---cut here---end--->8---

Well, 2 possible explanations:

 1) The tarball is not in SWH; because their loader fails on it or for
 whatever else reasons
 
 2) Or the tarball is archived by SWH but they use another hashing
 (SWH-ID) than the NAR.  Well, with the information in the package, Guix
 is not able to ask to SWH with the correct hash.  That’s the main
 motivation behind disarchive [3].

Last, Guix is not able to deal with hg-fetch or svn-fetch.  In this
message [4] and the 2 follow-up in the thread, there is some explanation
to implement ‘lookup-subversion-revision’ in (guix swh). Maybe for the
next release on Nov. ;-)

Well, the dance SWH needs some love. :-)


Thanks for trying!  It really helps to have this kind of feedback.



2: 
3: 
4: 


> Also, I got a backtrace when checking icecat:
>
> ★★★
> Backtrace:cecat@78.8.0-guix0-preview1 [archival]...

[...]

> ice-9/boot-9.scm:1667:16: In procedure raise-exception:
> In procedure bv-length: Wrong type argument in position 1 (expecting 
> bytevector): #f
> ★★★

Indeed, there is a bug.  Because the source of ’icecat’ raises a case
that is not handled by ’check-archival’ in (guix lint).

Basically in the snippet:

--8<---cut here---start->8---
 (match (lookup-content (content-hash-value hash)
(symbol->string
 (content-hash-algorithm hash)))
--8<---cut here---end--->8---

’lookup-content’ expect a bytevector for ’content-hash’ and in the case
of ’icecat’, it returns #f.  Then rai

Re: A Critique of Shepherd Design

[raid5atemyhomework]

Hello Ludo',

> Hi,
>
> raid5atemyhomework raid5atemyhomew...@protonmail.com skribis:
>
> > Now, let us combine this with the second feature (really a bug): GNU
> > shepherd is a simple, single-threaded Scheme program. That means that
> > if the single thread enters an infinite loop (because of a Shepherd
> > service description that entered an infinite loop), then Shepherd
> > itself hangs.
>
> You’re right that it’s an issue; in practice, it’s okay because we pay
> attention to the code we run there, but obviously, mistakes could lead
> to the situation you describe.
>
> It’s a known problem and there are plans to address it, discussed on
> this list a few times before. The Shepherd “recently” switched to
> ‘signalfd’ for signal handling in the main loop, with an eye on making
> the whole loop event-driven:
>
> https://issues.guix.gnu.org/41507
>
> This will address this issue and unlock things like “socket activation”.
>
> That said, let’s not lie to ourselves: the Shepherd’s design is
> simplistic. I think that’s okay though because there’s a way to address
> the main issues while keeping it simple.


I'm not sure you can afford to keep it simple.  Consider: 
https://issues.guix.gnu.org/47253

In that issue, the `networking` provision comes up potentially *before* the 
network is, in fact, up.  This means that other daemons that require 
`networking` could potentially be started before the network connection is up.

One example of such a daemon is `transmission-daemon`.  This daemon will bind 
itself to port 9091 so you can control it.  Unfortunately, if it gets started 
while network is down, it will be unable to bind to 9091 (so you can't control 
it) but still keep running.  On my system that means that on reboot I have to 
manually `sudo herd restart trannsmission-daemon`.

In another example, I have a custom daemon that I have set up to use the Tor 
proxy over 127.0.0.1:9050.  It requires both `networking` and `tor`.  When it 
starts after `networking` comes up but before the actual network does, it dies 
because it can't access the proxy at 127.0.0.1:9050 (apparently NetworkManager 
handles loopback as well).  Then shepherd respawns it, then it dies again 
(network still not up) enough times that it gets disabled.  This means that on 
reboot I have to manually `sudo herd enable raid5atemyhomework-custom-daemon` 
and `sudo herd restart raid5atemyhomework-custom-daemon`.

On SystemD-based systems, there's a `NetworkManager-network-online.service` 
which just calls `nm-online -s -q --timeout=30`.  This delays network-requiring 
daemons until after the network is in fact actually up.

However in https://issues.guix.gnu.org/47253#1 Mark points out this is 
undesirable in the Guix case since it could potentially stall the 
(single-threaded) bootup process for up to 30 seconds if the network is 
physically disconnected, a bad UX for desktop and laptop users (who might still 
want to run `transmission-daemon`, BTW) because it potentially blocks the 
initialization of X and make the computer unusable for such users for up to 30 
seconds after boot.  I note that I experienced such issues in some very old 
Ubuntu installations, as well.

SystemD can afford to *always* have `nm-online -s -q --timeout=30` because it's 
concurrent.  The `network-online.service` will block, but other services like X 
that don't ***need*** the network will continue booting.  So the user can still 
get to a usable system even if the boot isn't complete because the network 
isn't up yet due to factors beyond the control of the operating system.


Switching to a concurrent design for Shepherd --- *any* concurrent design --- 
is probably best done sooner rather than later, because it risks strongly 
affecting customized `configuration.scm`s like mine that have almost a half 
dozen custom Shepherd daemons.


Thanks
raid5atemyhomework