Re: Installer, ISO9660, etc.

[Danny Milosavljevic]

Aha!

I finally found why I couldn't use the iso9660 image on an usb stick.

Fix:

diff --git a/gnu/build/file-systems.scm b/gnu/build/file-systems.scm
index b6930497d..7f2c5dcb0 100644
--- a/gnu/build/file-systems.scm
+++ b/gnu/build/file-systems.scm
@@ -398,7 +398,7 @@ not valid header was found."
   (match (string-tokenize line)
 (((= string->number major) (= string->number minor)
   blocks name)
- (if (partition? name major minor)
+ (if #t ;(partition? name major minor)
  (loop (cons name parts))
  (loop parts))

Then it works fine on CD, on USB stick, whereever.

Is there a reason why (disk-partitions) second-guesses the Linux kernel and 
filters out some entries from /proc/partitions ?

Re: python-conda: Build is not reproducible

[Frederick Muriithi]

Cool, thanks!

On 9 Jul 2017 11:10 p.m., "Ludovic Courtès"  wrote:

> Hi Frederick,
>
> Frederick Muriithi  skribis:
>
> > On Fri, Jul 7, 2017 at 3:03 PM, Ludovic Courtès  wrote:
> >> Hello,
> >>
> >> This is most likely a consequence of .
> >>
> >> Can you run:
> >>
> >>   diff -ru --no-dereference \
> >> /gnu/store/302bb93kq3170xr57vijil7cya2s74ir-python-conda-4.3.16{,-
> check}
> >>
> >> to confirm that only Python object files are affected?
> >>
> >
> > The output of running that is:
> >
> > Binary files /gnu/store/302bb93kq3170xr57vijil7cya2s74
> ir-python-conda-4.3.16/lib/python3.5/site-packages/conda/
> cli/__pycache__/help.cpython-35.pyc
> > and /gnu/store/302bb93kq3170xr57vijil7cya2s74
> ir-python-conda-4.3.16-check/lib/python3.5/site-packages/
> conda/cli/__pycache__/help.cpython-35.pyc
> > differ
> > Binary files /gnu/store/302bb93kq3170xr57vijil7cya2s74
> ir-python-conda-4.3.16/lib/python3.5/site-packages/conda/
> cli/__pycache__/install.cpython-35.pyc
> > and /gnu/store/302bb93kq3170xr57vijil7cya2s74
> ir-python-conda-4.3.16-check/lib/python3.5/site-packages/
> conda/cli/__pycache__/install.cpython-35.pyc
> > differ
> > Binary files /gnu/store/302bb93kq3170xr57vijil7cya2s74
> ir-python-conda-4.3.16/lib/python3.5/site-packages/conda/
> __pycache__/__init__.cpython-35.pyc
> > and /gnu/store/302bb93kq3170xr57vijil7cya2s74
> ir-python-conda-4.3.16-check/lib/python3.5/site-packages/
> conda/__pycache__/__init__.cpython-35.pyc
> > differ
>
> Thanks for testing.  That indeed corresponds to the bug I mentioned
> above, so it’s not specific to this package, don’t worry.  :-)
>
> Ludo’.
>

Re: cuirass evaluate

[Ludovic Courtès]

Hi Mathieu,

Mathieu Othacehe  skribis:

> Now that Cuirass uses (guix git), I'm trying to robustify specification
> evaluation. Currently Cuirass calls a binary called "evaluate" to get a
> job list from a specification.
>
> I don't like the idea of this extra "evaluate" script because :
>
> * We have to give evaluate almost all arguments given to Cuirass
>   (load-path, package-path, cachedir, spec, database).
> * The script is made available for the user but it's very unclear how to
>   call it. The help says : "Usage evaluate FILE" which is wrong (5
>   arguments are expected).
>
> So my question is, is this a strong requirement to have a separate
> script to parse specification, or can I put this stuff back in main
> Cuirass program ?

It’s a requirement because the evaluation process has side effects on
the Guile that runs it; for instance, it loads tons of modules in it.
Also, the evaluation process may need to load modules that have the same
name as currently-loaded modules, but different content—and Guile
supports only one module with a given name.

That said, this program should be moved to $libexecdir/cuirass/VERSION.

HTH,
Ludo’.

Re: [PATCH] service: Fix arguments passing in the 'restart' action.

[Ludovic Courtès]

Hello,

宋文武  skribis:

> * modules/shepherd/service.scm (action): Use 'apply' to invoke the 'start'
> procedure in the 'restart' action.
> ---
>  modules/shepherd/service.scm | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/modules/shepherd/service.scm b/modules/shepherd/service.scm
> index 26f29aa..72fbc3d 100644
> --- a/modules/shepherd/service.scm
> +++ b/modules/shepherd/service.scm
> @@ -398,7 +398,7 @@ wire."
>   (if running
>   (stop obj)
>   (local-output "~a was not running." (canonical-name obj)))
> - (start obj args)))
> + (apply start obj args)))

Good catch!  Applied, thanks.

Ludo’.

RPC pipelining

[Ludovic Courtès]

Hello Guix!

One of the main sources of slowness when talking to a remote daemon, as
with GUIX_DAEMON_SOCKET=guix://…, is the many RPCs that translate in
lots of network round trips:

--8<---cut here---start->8---
$ GUIX_PROFILING=rpc ./pre-inst-env guix build inkscape -d --no-grafts
/gnu/store/iymxyy5sn0qrkivppl6fn0javnmr3nss-inkscape-0.92.1.drv
Remote procedure call summary: 1006 RPCs
  built-in-builders  ... 1
  add-to-store   ...   136
  add-text-to-store  ...   869
--8<---cut here---end--->8---

In this example we’re making ~1,000 round trips; not good!

Before changing the protocol, an idea that came to mind is to do “RPC
pipelining”: send as many RPC requests at once, then read all the
corresponding responses.

It turns out to necessitate a small change in the daemon, though, but
the attached patch demonstrates it: the client buffers all
‘add-text-to-store’ RPCs, and writes them all at once when another RPC
is made (because other RPCs, which are not buffered, might depend on the
effect of those ‘add-text-to-store’ RPCs) or when the connection is
closed.  In practice, on the example above, it manages to buffer all 869
RPCs and send them all at once.

To estimate the effectiveness of this approach, I introduced delay on
the loopback device with tc-netem(8) and measured execution time (the
first run uses pipelining, the second doesn’t):

--8<---cut here---start->8---
$ sudo tc qdisc add dev lo root netem delay 150ms
$ time GUIX_DAEMON_SOCKET=guix://localhost ./pre-inst-env guix build inkscape 
-d --no-grafts
accepted connection from 127.0.0.1
/gnu/store/iymxyy5sn0qrkivppl6fn0javnmr3nss-inkscape-0.92.1.drv

;;; (flush-pending-rpcs 869)

real0m47.796s
user0m1.307s
sys 0m0.056s
$ time GUIX_DAEMON_SOCKET=guix://localhost guix build inkscape -d --no-grafts
accepted connection from 127.0.0.1
/gnu/store/iymxyy5sn0qrkivppl6fn0javnmr3nss-inkscape-0.92.1.drv

real5m7.226s
user0m1.392s
sys 0m0.056s
$ sudo tc qdisc del dev lo root
--8<---cut here---end--->8---

So the wall-clock time is divided by 6 thanks to ‘add-text-to-store’
pipelining, but it’s still pretty high due to the 136 ‘add-to-store’
RPCs which are still *not* pipelined.

It’s less clear what to do with these.  Buffering them would require
clients to compute the store file name of the files that are passed to
‘add-to-store’, which involves computing the hash of the files itself,
which can be quite costly and redundant with what the daemon will do
eventually anyway.  The CPU cost might be compensated for when latency
is high, but not when latency is low.

Anyway, food for thought!

For now, if those using Guix on clusters are willing to test the patch
below (notice that you need to run the patched guix-daemon as well), I’d
be interested in seeing how representative the above test is!

Ludo’.

diff --git a/guix/store.scm b/guix/store.scm
index b15da5485..1ba22cf2d 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -40,6 +40,7 @@
   #:use-module (ice-9 regex)
   #:use-module (ice-9 vlist)
   #:use-module (ice-9 popen)
+  #:use-module (ice-9 format)
   #:use-module (web uri)
   #:export (%daemon-socket-uri
 %gc-roots-directory
@@ -322,7 +323,7 @@
 
 (define-record-type 
   (%make-nix-server socket major minor
-buffer flush
+buffer flush pending-rpcs
 ats-cache atts-cache)
   nix-server?
   (socket nix-server-socket)
@@ -332,6 +333,10 @@
   (buffer nix-server-output-port) ;output port
   (flush  nix-server-flush-output);thunk
 
+  ;; List of pending 'add-text-to-store' RPC arguments.
+  (pending-rpcs nix-server-pending-rpcs
+set-nix-server-pending-rpcs!)
+
   ;; Caches.  We keep them per-connection, because store paths build
   ;; during the session are temporary GC roots kept for the duration of
   ;; the session.
@@ -509,7 +514,7 @@ for this connection will be pinned.  Return a server object."
   (let ((conn (%make-nix-server port
 (protocol-major v)
 (protocol-minor v)
-output flush
+output flush '()
 (make-hash-table 100)
 (make-hash-table 100
 (let loop ((done? (process-stderr conn)))
@@ -521,8 +526,17 @@ for this connection will be pinned.  Return a server object."
   (force-output (nix-server-output-port server))
   ((nix-server-flush-output server)))
 
+(define (flush-pending-rpcs server)
+  (let ((len (length (nix-server-pending-rpcs server
+(when (> len 0)
+ 

Re: FW: [oss-security] accepting new members to (linux-)distros lists

[Ludovic Courtès]

Leo Famulari  skribis:

> On Wed, Jul 05, 2017 at 01:33:05PM -0400, Mark H Weaver wrote:
>> l...@gnu.org (Ludovic Courtès) writes:
>> > The real question is about our commitment to contribute back.
>> > Presumably only one or two of us would be on that list, so they would
>> > largely have that responsibility individually, even if the rest of us
>> > could of course help out as far as the embargo etc. permits.
>> >
>> > Long story short, I would be super happy if you or Mark were on that
>> > list.
>> >
>> > How do you feel about it?
>> 
>> It might be that joining linux-distros is the right thing to do, but I
>> don't have the spare capacity to contribute back at this time.  Also, I
>> have mixed feelings about promising to keep security flaws a secret for
>> however long I'm asked to do so (which apparently exceeded the time
>> specified in the mailing list rules for Stack Clash).  I'm not yet
>> prepared to make such a promise.
>
> I also don't have the time to contribute back to the linux-distros list
> at the level required in the list membership criteria.
>
> The work I do for Guix is my best effort and I think it's understood
> that, as a volunteer, there will be times when I am unavailable. My
> job's schedule is irregular and has to take precedence.
>
> On the other hand, it seems like the linux-distros mailing list
> community is trying to formally assign tasks for members to fulfill for
> the benefit of their community.
>
> Currently, I can't commit to doing any task by myself for the entire
> Linux distro community. Also, I don't think there is anybody on the list
> who knows me and could vouch for me alone.
>
> If there were another Guix member who could split the task, and had
> someone to vouch for them, that could make it possible. But based on the
> replies in the last 10 days, it doesn't seem to be the case.

There’s a handful of Guix contributors who’ve been active on security
fixes, so it would be great if one of them could team up with you!

Ludo’.

Re: FW: [oss-security] accepting new members to (linux-)distros lists

[Ludovic Courtès]

Leo Famulari  skribis:

> On Thu, Jun 29, 2017 at 12:48:22PM +0800, Alex Vong wrote:
>> Leo Famulari  writes:
>> 
>> [...]
>> > But, the "Stack Clash" issues took us by surprise and we spent a few
>> > days writing and testing our fixes. We are committed to supporting
>> > 32-bit platforms where these bugs are apparently easy to exploit.
>> > Without access to the exploits or detailed discussion, it was very
>> > difficult to know if our fixes actually worked. So, we could have
>> > responded more quickly and effectively with early notice.
>> [...]
>> 
>> Should we bring this discussion to nix devs as well? I am sure they are
>> facing the same issue of not having early access to vulnerabilities. It
>> will be insightful to know how they dealt with it in the past and their
>> opinions on joining the list.
>
> If somebody who has a relationship with the Nix team would like to
> discuss it with them, I'd be happy to hear the result, but I don't
> really have time for it right now. Also, we would not be able to discuss
> embargoed bugs from linux-distros with them, according to the list
> policy.
>
> Besides, I think our present situation and practices regarding security
> updates is very different from Nix's. They have different tools for
> shipping security updates, and they do the "stable" branch thing.

Indeed.  We can learn by working with each other in general, but for
this particular topic I think it wouldn’t be that helpful.  In addition
to having different tools and practices, Nix and Guix are simply
different distros.

Ludo’.

Re: 34/44: gnu: Add texlive-fonts-ec.

[Mark H Weaver]

Ricardo Wurmus  writes:

> This message seems to indicate that:
>
>> svn: E000110: Unable to connect to a repository at URL 
>> 'svn://www.tug.org/texlive/tags/texlive-2017.1/Master/texmf-dist/fonts/source/jknappen/ec'
>> svn: E000110: Unknown hostname 'www.tug.org'
>
> I’ve tried downloading this directory again and I get the same hash as
> before.  Maybe it was a temporary problem?

You're right.  It worked on a second attempt.

Given the error message above, I would have expected 'svn' to return a
non-zero status code, and for that error to be noticed by Guix before
trying to verify the hash.  I guess something failed to propagate the
error.  I looked at the relevant code in Guix, but didn't see anything
obviously wrong with it.  Oh well, I suppose it doesn't matter much,
since checking the hash is the ultimate test.

 Thanks,
   Mark

Re: FW: [oss-security] accepting new members to (linux-)distros lists

[Leo Famulari]

On Mon, Jul 10, 2017 at 05:53:24PM +0200, Ludovic Courtès wrote:
> Leo Famulari  skribis:
> > If there were another Guix member who could split the task, and had
> > someone to vouch for them, that could make it possible. But based on the
> > replies in the last 10 days, it doesn't seem to be the case.
> 
> There’s a handful of Guix contributors who’ve been active on security
> fixes, so it would be great if one of them could team up with you!

Yes, I'd be willing to apply for membership if I had a partner.

There is also the issue of finding someone to vouch for us. According
to the membership criteria [0]:

"9. Have someone already on the private list, or at least someone else
who has been active on oss-security for years but is not affiliated with
your distro nor your organization, vouch for at least one of the people
requesting membership on behalf of your distro (then that one
vouched-for person will be able to vouch for others on your team, in
case you'd like multiple people subscribed)"

I don't think I know anyone who meets those criteria.

I understand that people may not want to discuss that in public. If we
were ready to apply and this was the final missing piece of the puzzle,
I hope somebody would at least contact me in private to discuss it.

[0]
http://seclists.org/oss-sec/2017/q2/638


signature.asc
Description: PGP signature

core-updates summer 2017

[Leo Famulari]

I just started a new evaluation of core-updates and am reconfiguring my
GuixSD system based on it.

Let's use this thread to discuss the state of the branch.


signature.asc
Description: PGP signature

Re: Installer, ISO9660, etc.

[Ludovic Courtès]

Danny Milosavljevic  skribis:

> Aha!
>
> I finally found why I couldn't use the iso9660 image on an usb stick.
>
> Fix:
>
> diff --git a/gnu/build/file-systems.scm b/gnu/build/file-systems.scm
> index b6930497d..7f2c5dcb0 100644
> --- a/gnu/build/file-systems.scm
> +++ b/gnu/build/file-systems.scm
> @@ -398,7 +398,7 @@ not valid header was found."
>(match (string-tokenize line)
>  (((= string->number major) (= string->number minor)
>blocks name)
> - (if (partition? name major minor)
> + (if #t ;(partition? name major minor)
>   (loop (cons name parts))
>   (loop parts))
>
> Then it works fine on CD, on USB stick, whereever.
>
> Is there a reason why (disk-partitions) second-guesses the Linux kernel and 
> filters out some entries from /proc/partitions ?

It has this:

  (define (partition? name major minor)
;; Select device names that end in a digit, like libblkid's 'probe_all'
;; function does.  Checking for "/sys/dev/block/MAJOR:MINOR/partition"
;; doesn't work for partitions coming from mapped devices.
(and (char-set-contains? char-set:digit (last-character name))
 (> major 2)))  ;ignore RAM disks and floppy disks

What’s the /dev name in your case?

Ludo’.

Re: Installer, ISO9660, etc.

[Danny Milosavljevic]

> What’s the /dev name in your case?

In the working case as a CD, it's /dev/sr0.
In the non-working case on a USB flash drive, it's /dev/sda and /dev/sda1.

There's a dummy partition in the iso image as well, so I'm not sure why 
/dev/sda1 doesn't work as well... let's see...

When fdisk says that the starting sector is "1", does that mean the first 
sector (i.e. there's no sector before it) or does that mean the sector after 
the MBR?

On the guix rescue console, I tried comparing the first few bytes of /dev/sda 
and /dev/sda1 and it seems they are different.

That would mean that the ISO filesystem support code can't find the primary 
volume descriptor any more because it's reading it from the wrong position 
because it reads from the device file "/dev/sda1" and that file shifts it from 
where it would have been if it read "/dev/sda".

But this "partition?" thing would mean that having a partition table is now 
mandatory, right?  Is that really supposed to be the case?  The partition table 
has a magic mark (0x55 0xAA) in order for it *not* to be mandatory (i.e. if you 
want to use the whole disk without partitioning anything, go right ahead) and I 
can still remember using (floppy) disks which had no partition table.

Re: core-updates on Hurd

[rennes]

Hello,

On 07/09/2017 05:10:47 PM, Ludovic Courtès wrote:


The problem was that ‘struct dirent64’ is slightly different on  
GNU/Hurd

than on GNU/Linux.

Rennes, could you confirm that it works for you?


The solution works, Thanks

my recent messages

[ng0]

Hi,

I want to apologize. While I did not mean to attack or
make anyones work look bad, it happened nevertheless.
Some people will always read between the lines, even when
I never have any subtext and always express what I mean.

Since Summer/Spring '15 I worked on creating a system,
most of the recent energy spend on GuixSD as a base.
It is good to deconstruct and re-evaluate ideas from
other points of view, this is still going on.
Ultimately the decision doesn't depend on myself as
we are a small team already.

What I think caused my disappointment and annoyance
is just being at a turning point of personal events
(with the addition of lots of bureaucracy)
and the fact that I only started a couple of months
ago to reduce the amount of hours I work per week.
Before that it was (I once had to calculate this)
85 - 98 hours a week, and taking one day off was an
exception.
I am currently trying to get back into making music
and get some other interests I have back into my
life.
I hope that the next vacation can happen within the
next 3 years, vacation usually (in the ideal case)
means getting somewhere where I can absolutely ignore
that technology exists.

My criticism to some point is still valid, I should
just express what I meant to say when I am more
relaxed.

Sorry.
-- 
ng0
GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588
GnuPG: https://n0is.noblogs.org/my-keys
https://www.infotropique.org https://krosos.org


signature.asc
Description: PGP signature

Re: core-updates summer 2017

[Kei Kebreau]

Leo Famulari  writes:

> I just started a new evaluation of core-updates and am reconfiguring my
> GuixSD system based on it.
>
> Let's use this thread to discuss the state of the branch.

Would this be the time to enable the ACL tests as previously discussed?


signature.asc
Description: PGP signature