bug#22883: Authenticating Git checkouts: step #1

[Ludovic Courtès]

Hello Guix!

Ludovic Courtès  skribis:

> To begin with, I propose the attached script: when given a commit range,
> it authenticates each commit, meaning that it ensures commits have a
> valid signature and that that signature was made by one of the
> authorized keys.  Sample session:
>
> $ time ./pre-inst-env guile -e git-authenticate 
> build-aux/git-authenticate.scm d68de958b60426798ed62797ff7c96c327a672ac 
> 099ce5d4901706dc2c5be888a5c8cbf8fcd0d576
> Authenticating d68de95 to 099ce5d (7938 commits)...
> Signing statistics:
>   BCA689B636553801C3C62150197A5888235FACAC   1454
>   3CE464558A84FDC69DB40CFB090B11993D9AEBB5   1025
>   BBB02DDF2CEAF6A80D1DE643A2A06DF2A33A54FA941
>
> [...]
>
> real  2m21.272s
> user  1m38.741s
> sys   0m59.546s

I’ve now committed this file:

  b3011dbbd2 doc: Mention "make authenticate".
  787766ed1e git-authenticate: Keep a local cache of previously-authenticated 
commits.
  785af04a75 git: 'commit-difference' takes a list of excluded commits.
  1e43ab2c03 Add 'build-aux/git-authenticate.scm'.

Commit 787766ed1e takes care of caching (one of the limitations I
mentioned in my previous message).

Commit b3011dbbd2 adds instructions for contributors on how to
authenticate a checkout (copied below).  It’s a bit bumpy so I would
very much welcome feedback and suggestions on how to improve this!

Thanks in advance!

Ludo’.

--8<---cut here---start->8---
If you want to hack Guix itself, it is recommended to use the latest
version from the Git repository:

 git clone https://git.savannah.gnu.org/git/guix.git

   How do you ensure that you obtained a genuine copy of the repository?
Guix itself provides a tool to “authenticate” your checkout, but you
must first make sure this tool is genuine in order to “bootstrap” the
trust chain.  To do that, run:

 git verify-commit `git log --format=%H build-aux/git-authenticate.scm`

   The output must look something like:

 gpg: Signature made Fri 27 Dec 2019 01:27:41 PM CET
 gpg:using RSA key 3CE464558A84FDC69DB40CFB090B11993D9AEBB5
 ...
 gpg: Signature made Fri 27 Dec 2019 01:25:22 PM CET
 gpg:using RSA key 3CE464558A84FDC69DB40CFB090B11993D9AEBB5
 ...

...  meaning that changes to this file are all signed with key
‘3CE464558A84FDC69DB40CFB090B11993D9AEBB5’ (you may need to fetch this
key from a key server, if you have not done it yet).

   From there on, you can authenticate all the commits included in your
checkout by running:

 make authenticate

   The first run takes a couple of minutes, but subsequent runs are
faster.

 Note: You are advised to run ‘make authenticate’ after every ‘git
 pull’ invocation.  This ensures you keep receiving valid changes to
 the repository
--8<---cut here---end--->8---


signature.asc
Description: PGP signature

bug#38167: guix pull takes over 8 GiB of memory to finish if there are no substitutes

[Ludovic Courtès]

Hi!

Danny Milosavljevic  skribis:

> On Mon, 11 Nov 2019 21:42:24 +0100
> Ludovic Courtès  wrote:
>
>> Also, it would be great if you could identify which derivation build
>> requires that much memory, if this was happening sequentially.  (I
>> suspect “guix-packages.drv” is the one that eats up the most memory.)
>
> How do I do that?

You could look at the output of “guix processes” and find out which
session child process is consuming memory.

> I've attached the process tree, filtered by "grep guix", of the latest such
> occurence (if this E-Mail still makes it before I have no memory left :P).

AFAICS this excludes the actual build process (which runs “guile”, not
“guix”).

HTH!

Ludo’.

bug#38764: emacs-darkroom: buggy display behavior and errors

[Christopher Howard]

Hello, with emacs-26.2, emacs-darkroom, in darkroom-tentative-mode,
often gives buggy display behavior and errors, though in sporadic
fashion that is difficult to reproduce mechanically. With frequent use,
the three following problems are often noticed:

1) Sometime I get an "outside of point" error, which prevents me from
disabling darkroom-tentative-mode or using other keyboard meta
commmands. I can escape by using the gtk menu.

2) Sometimes when I disable darkroom-tentative-mode, the text shrinks
very small to fit into the size of the window it happens to be in.

3) Sometimes enabling darkroom-tentative-mode centers the text but does
not resize it, or resizes it but does not center it.

I believe this behavior might be somehow related to the use of non-
standard monospace fonts or font sizes in custom emacs options.

-- 
Christopher Howard
Enterprise Solutions Manager
Alaska Satellite Internet
PO Box 70, Ester, AK 99725
3239 La Ree Way, Fairbanks, AK 99709
907.451.0088
1.888.396.5623
www.alaskasatelliteinternet.com

bug#38167: guix pull takes over 8 GiB of memory to finish if there are no substitutes

[Ludovic Courtès]

Hi!

Danny Milosavljevic  skribis:

> USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND
> dannym   19221 20.8 87.4 9404812 6884184 pts/0 Tl   20:34   2:40 
> /gnu/store/sc7z07gim1iq5zvfz1amdwf2irxrzifg-guile-2.2.6/bin/guile 
> --no-auto-compile /home/dannym/.config/guix/current/bin/guix pull

Oh, that’s an RSS of 6 GiB for ‘guix pull’ itself?  Weird, I don’t see
how that can happen.

Could it be that ~/.cache/guix/checkouts/THE-THING is too big, which in
turn causes libgit2 to consume too much memory somehow?

What happens if you attach strace to this process at the moment where
it’s consuming a lot of memory?  Is it traversing Git pack files or
something like that?

TIA,
Ludo’.

bug#22883: Authenticating a Git checkout

[Ricardo Wurmus]

Ludovic Courtès  writes:

> Hello,
>
> Just a note for later…
>
> l...@gnu.org (Ludovic Courtès) skribis:
>
>> With the quick-hack libgit2 bindings attached, I can run this program,
>> which authenticates HEAD:
>
> [...]
>
>> So I think we can go from here.  Our repo would contain a Scheme list of
>> authorized OpenPGP fingerprints, and we’d check whether the fingerprint
>> that shows up in ‘valid-signature’ above is among them
>
> Storing the list of authorized keys in a file in the repo is
> inconvenient: simply to retrieve it, you’d need to make a checkout.  So
> for each commit we verify, we have to check out the whole repo, which is
> inefficient.
>
> While reading
> , I
> realized we could store in empty Git commit messages, which would
> address the above problem (we could use a custom object type too, but
> that would be less convenient.)
>
> So the special commit could look like:
>
>   Authorization
>
>   (commit-authorizations
> (authorization-commit (KEY1 KEY2 …))
> (files ("hydra.gnu.org.pub") (KEY1 KEY2 …))
> (files _ (KEY1 KEY2 …))) ;all other files
>
> That way, to authenticate a commit, we first fetch the latest
> authorization commit, read the authorization rules from there, and make
> sure that the changes it makes match the rules.
>
> Thoughts?

Does this *have* to be baked into git?  Or are we like the carpenter
apprentice who just learned how to use a hammer and considers everything
to be a kind of nail…?

I see the appeal of having everything in git as that’s where the commits
are that should be authenticated, but using special commit messages
seems to me like shoehorning update authorization into a code revision
tool.

You mentioned that checking signatures on commits is also kinda slow
because it’s sequential and not cached.  I don’t know what I really
want, but is there perhaps a way to aggregate signatures on past commits
so that the client’s work is reduced…?

(I’m very glad you’re thinking about this problem and that you’ve come up
with practical steps forward!  I don’t know if my thoughts on this topic
are useful.)

--
Ricardo

bug#22883: Authenticating Git checkouts: step #1

[Ricardo Wurmus]

Hi Ludo,

> I’ve now committed this file:
>
>   b3011dbbd2 doc: Mention "make authenticate".
>   787766ed1e git-authenticate: Keep a local cache of previously-authenticated 
> commits.
>   785af04a75 git: 'commit-difference' takes a list of excluded commits.
>   1e43ab2c03 Add 'build-aux/git-authenticate.scm'.
>
> Commit 787766ed1e takes care of caching (one of the limitations I
> mentioned in my previous message).
>
> Commit b3011dbbd2 adds instructions for contributors on how to
> authenticate a checkout (copied below).  It’s a bit bumpy so I would
> very much welcome feedback and suggestions on how to improve this!

This is great!

Thank you for the instructions.  I thought I had all keys, but
apparently at least one of them is missing.  “make authenticate” fails
for me with this error:

Throw to key `srfi-34' with args `(#)'.

I previously downloaded the gpg keyring from Savannah:

https://savannah.gnu.org/project/memberlist-gpgkeys.php?group=guix

Looks like Hartmut used to use a different key, which I don’t have.

--
Ricardo

bug#38613: Disabling bytecompilation on a list of files.

[Brett Gilio]

Maxim Cournoyer  writes:

> Brett Gilio  writes:
>
>
> [...]
>
>> Leo and Maxim,
>>
>> Do you think this feature addition to the emacs-build-system is good to
>> be pushed, then? I think it looks quite fine but I want to double check
>> before potentially doing something dumb.
>
> LGTM!
>
> Thank you!
>
> Maxim
>

Done! Thank you Leo and Maxim.

-- 
Brett M. Gilio
GNU Guix, Contributor | GNU Project, Webmaster
[DFC0 C7F7 9EE6 0CA7 AE55 5E19 6722 43C4 A03F 0EEE]
 

bug#38755: icecat retains reference on clang

[Ludovic Courtès]

Hi,

Ricardo Wurmus  skribis:

> This should do the trick:
>
> diff --git a/gnu/packages/gnuzilla.scm b/gnu/packages/gnuzilla.scm
> index d5d9839e1a..e9458037a5 100644
> --- a/gnu/packages/gnuzilla.scm
> +++ b/gnu/packages/gnuzilla.scm
> @@ -1023,7 +1023,11 @@ from forcing GEXP-PROMISE."
> (format #t "configure flags: ~s~%" flags)
> (apply invoke bash
>(string-append srcdir "/configure")
> -  flags
> +  flags)
> +   ;; Remove references to store items
> +   (substitute* "toolkit/content/buildconfig.html"
> + (("/gnu/store") ""))

This should not be a literal “/gnu/store”, though.  Perhaps we should
just use ‘remove-store-references’ instead of ‘substitute*’?

Thanks,
Ludo’.