Re: On raw strings in commit field

[zimoun]

On Tue, 04 Jan 2022 at 06:23, Liliana Marie Prikler  
wrote:

>> The content can be one file, some files, folders, etc.  or Git
>> objects as Git commit object or Git tree object or whatever. 
>> Therefore, Git commit hash only depends on the content itself, i.e.,
>> Git commit object; as explained by the pointer provided earlier in
>> the thread,
>> 
>>     
>
> At some point in there (you can figure out yourself where), you are
> mistakenly equating file hashes and commit hashes, which you make
> comparison to other tools which only regard the files as content.  One
> of them is immutable for all I know, the other is subject to very
> observable changes.

Incorrect.

I quote myself:

then, for what my opinion is worth on that matter, my probably
wrong understanding of your words is that perhaps you are
missing a point about content-addressability.

Loop.

Re: On raw strings in commit field

[zimoun]

Hi Liliana,

On Tue, 04 Jan 2022 at 09:51, zimoun  wrote:
> On Tue, 04 Jan 2022 at 06:23, Liliana Marie Prikler 
>  wrote:
>
>>> The content can be one file, some files, folders, etc.  or Git
>>> objects as Git commit object or Git tree object or whatever. 
>>> Therefore, Git commit hash only depends on the content itself, i.e.,
>>> Git commit object; as explained by the pointer provided earlier in
>>> the thread,
>>> 
>>>     
>>
>> At some point in there (you can figure out yourself where), you are
>> mistakenly equating file hashes and commit hashes, which you make
>> comparison to other tools which only regard the files as content.  One
>> of them is immutable for all I know, the other is subject to very
>> observable changes.

Let pick one commit in the Git history, for instance:
e598e46913c661bc92df813d537eeb6be5a86471. 

--8<---cut here---start->8---
$ git --no-pager show e598e46913c661bc92df813d537eeb6be5a86471

commit e598e46913c661bc92df813d537eeb6be5a86471
Author: Tobias Geerinckx-Rice 
Date:   Tue Oct 26 02:01:41 2021 +0200

gnu: darkhttpd: Update to 1.13.

* gnu/packages/web.scm (darkhttpd): Update to 1.13.
[source]: Use GIT-FETCH and GIT-FILE-NAME.
[arguments]: Don't explicitly return #t from phases.

diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index dc5a9d61a8..2bd3c4ea13 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -5791,28 +5791,28 @@ (define-public surfraw
[...]
--8<---cut here---end--->8---

Let copy somewhere the content of this Git commit:

$ mkdir -p /tmp/kikoo
$ cp .git/objects/e5/98e46913c661bc92df813d537eeb6be5a86471 \
 /tmp/kikoo/content

Now, let run inside a container:

$ cd /tmp/kikoo
$ guix shell -C coreutils gzip
[env]$ printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" \
  | cat - content | gzip -d | sha1sum

gzip: stdin: unexpected end of file
e598e46913c661bc92df813d537eeb6be5a86471  -

Explain me.  « Git commit hash only depends on the content itself, i.e.,
Git commit object », as I wrote.

Instead of taking a superior tone «(you can figure out yourself where)»,
I would prefer that you correctly read the messages I wrote.  Maybe,
that’s why my previous email is probably is bit harsh, sorry.


Specifically, the content of this Git commit is just:

--8<---cut here---start->8---
$ printf "\x1f\x8b\x08\x00\x00\x00\x00\x00" |cat - content | gzip -d

commit 662tree 44207c0d8c9e885b156bb98562221beb2ab8f7bf
parent b8fc7c23596b542b81306829b31cf255d908fa65
author Tobias Geerinckx-Rice  1635206501 +0200
committer Tobias Geerinckx-Rice  1635206568 +0200
gpgsig -BEGIN PGP SIGNATURE-
 
 iIMEABYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCYXdFqA0cbWVAdG9iaWFz
 LmdyAAoJEA2w/4hPVW15OoAA+gIzCyrlUEBUFdLp0CBFW1GxjVzYiSFMmk5aNDNu
 OngEAQDoisl2dQK7lLvIl/2xXDqoJ2CAoiqZ1DbGNyg3Yt/DCA==
 =RH+I
 -END PGP SIGNATURE-

gnu: darkhttpd: Update to 1.13.

* gnu/packages/web.scm (darkhttpd): Update to 1.13.
[source]: Use GIT-FETCH and GIT-FILE-NAME.
[arguments]: Don't explicitly return #t from phases.
--8<---cut here---end--->8---

The files and folders are in a Git tree object, referred in the Git
commit object by ’tree 44207c0d8c9e885b156bb98562221beb2ab8f7bf’.

I let you run this command from the Guix git repo:

$ git cat-file -p 44207c0d8c9e885b156bb98562221beb2ab8f7bf


Please re-read all your answers and mines.  I hope you will see where
you were incorrect.


Cheers,
simon

Re: guix fails to build on aarch64

[Pierre Langlois]

Hi there,

Akira Kyle  writes:

> Ricardo Wurmus  writes:
>> > test-name: file-needed/recursive
>> > location:
>> > /tmp/guix-build-guix-1.3.0-14.2a621f1.drv-0/source/tests/gremlin.scm:70
>> > source:
>> …
>> > + (and (zero? (close-pipe pipe))
>> > +  (lset= string=?
>> > + (pk 'truth ground-truth)
>> > + (pk 'needed needed)
>> > actual-value: #f
>> > result: FAIL
>
>> Did the logs not contain the values for truth and needed?  That would
>> mean that the test already failed to close the pipe.
>
> I've been trying to debug failing guix tests on aarch64. At the end of
> logfile for the gremlin test suite there's the following which may be
> related to why the truth and needed values were not printed:
>
> a.out: stripping RUNPATH to
> ("/gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib"
> "/gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib"
> "/gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib/gcc/aarch64-unknown-linux-gnu/10.3.0/../../.."
> "/gnu/store/40lx91wz35qci25qzi9xfqvxwby85xha-profile/lib") (removed
> ("/foo" "/bar"))
> a.out: warning: RUNPATH contains bogus entries:
> ("/gnu/store/skfhr2f8b7jnnpiarrrcjn6qx0xzfw00-glibc-2.33/lib"
> "/gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib"
> "/gnu/store/47snyrq6pq6896m9dysp2s5vx53m6x08-gcc-10.3.0-lib/lib/gcc/aarch64-unknown-linux-gnu/10.3.0/../../.."
> "/gnu/store/40lx91wz35qci25qzi9xfqvxwby85xha-profile/lib")
> a.out: error: depends on 'libgcc_s.so.1', which cannot be found in RUNPATH ()
> WARNING: (test-gremlin): imported module (guix build utils) overrides
> core binding `delete'

I've also been trying to fix this test but without much luck.  It
does look similar to this issue for ppc64 [0], where the `ldd/ld.so'
beaviour isn't the same as what the gremlin guile module does. However
the patch proposed there isn't fixing the issue for me either on
aarch64.

[0]: https://issues.guix.gnu.org/52940,

Maybe we can compare notes and a solution will come up :-).  So the test
fails because 'truth', the result from `ldd', has ld-linux-aarch64.so
listed while 'needed', from the gremlin guile module doesn't:

--8<---cut here---start->8---
(truth ;; result from `ldd libguile.so'
 ("/gnu/store/...-gcc-10.3.0-lib/lib/libgcc_s.so.1"
  "/gnu/store/...-glibc-2.33/lib/ld-linux-aarch64.so.1"   ;; This isn't
  ;; in gremlins
  "/gnu/store/...-glibc-2.33/lib/libc.so.6"
  "/gnu/store/...-glibc-2.33/lib/libcrypt.so.1"
  "/gnu/store/...-glibc-2.33/lib/libdl.so.2"
  "/gnu/store/...-glibc-2.33/lib/libm.so.6"
  "/gnu/store/...-glibc-2.33/lib/libpthread.so.0"
  "/gnu/store/...-guile-3.0.7/lib/libguile-3.0.so.1"
  "/gnu/store/...-libffi-3.3/lib/libffi.so.7"
  "/gnu/store/...-libgc-8.0.4/lib/libgc.so.1"
  "/gnu/store/...-libunistring-0.9.10/lib/libunistring.so.2"))

(needed  ;; result from gremlin
 ("/gnu/store/...-gcc-10.3.0-lib/lib/libgcc_s.so.1"
  "/gnu/store/...-glibc-2.33/lib/libc.so.6"
  "/gnu/store/...-glibc-2.33/lib/libcrypt.so.1"
  "/gnu/store/...-glibc-2.33/lib/libdl.so.2"
  "/gnu/store/...-glibc-2.33/lib/libm.so.6"
  "/gnu/store/...-glibc-2.33/lib/libpthread.so.0"
  "/gnu/store/...-guile-3.0.7/lib/libguile-3.0.so.1"
  "/gnu/store/...-libffi-3.3/lib/libffi.so.7"
  "/gnu/store/...-libgc-8.0.4/lib/libgc.so.1"
  "/gnu/store/...-libunistring-0.9.10/lib/libunistring.so.2"))
--8<---cut here---end--->8---

Digging a bit more I started comparing x86_64 and aarch64 binaries and
noticed that libguile.so didn't have the same dynamic section:

--8<---cut here---start->8---
# On aarch64:
$ objdump -x 
/gnu/store/pqw0c33k2h8n2snpchnyvx7w617kk31s-guile-3.0.7/lib/libguile-3.0.so.1.4.0



/gnu/store/pqw0c33k2h8n2snpchnyvx7w617kk31s-guile-3.0.7/lib/libguile-3.0.so.1.4.0:
 file format elf64-littleaarch64
...
Dynamic Section:

  NEEDED   libgc.so.1   

  NEEDED   libpthread.so.0  

  NEEDED   libffi.so.7  

  NEEDED   libunistring.so.2

  NEEDED   libcrypt.so.1

  NEEDED   libdl.so.2  

Re: Release v1.4 (or 2.0): process and schedule ?

[Maxim Cournoyer]

Hello,

Ludovic Courtès  writes:

> Hello!
>
> Maxim Cournoyer  skribis:
>
>> About the current status, I'm nearing on pushing a version-1.4.0 branch
>> which is based on master with a few more (core-ish) updates.  There's
>> still a few days ahead of that, so if you manage to get many of this
>> kind of problems fixed & merged in master they can easily be included in
>> the next release.
>
> Before the first RC, I’d like to push the latest ‘guix style’ changes:
> .
>
> Do we enter string freeze before the first RC?  I forgot how we did it
> last time.

I remember the string freeze was a bit early last time, given the
release also took time to be polished and shipped, but I think it also
was useful.  Integrating translation changes should occur not too late
as they can introduce build issues that need to be resolved, blocking a
release.

Thank you,

Maxim

RFC: new syntax for inline patches

[Ricardo Wurmus]

Hi Guix,

does this pattern look familiar to you?

  (arguments
(list
#:phases
'(modify-phases %standard-phases
  (add-after 'unpack 'i-dont-care
(lambda _
  (substitute* "this-file"
(("^# some unique string, oh, careful! gotta \\(escape\\) this\\." 
m)
 (string-append m "\nI ONLY WANTED TO ADD THIS LINE!\n"

This is a lot of boilerplate just to add a line to a file.  I’m using
“substitute*” but actually I don’t want to substitute anything.  I just
know that I need to add a line somewhere in “this-file”.

Or maybe it’s a CMakeLists.txt file that inexplicably wants to download
stuff?  I should patch that file but it’s a multi-line change.  So I’m
trying to do the same as above with several different anchor strings to
comment out lines.

We have a lot of packages like that.  And while this boilerplate pattern
looks familiar to most of us now, it is really unclear.  It is
imperative and abuses regular expression matching when really it should
have been a patch.

There are a few reasons why we don’t use patches as often:

1. the source code is precious and we prefer to modify the original
sources as little as necessary, so that users can get the source code as
upstream intended with “guix build -S foo”.  We patch the sources
primarily to get rid of bundled source code, pre-built binaries, or
code that encroaches on users’ freedom.

2. the (patches …) field uses patch files.  These are annoying and
inflexible.  They have to be added to dist_patch_DATA in gnu/local.mk,
and they cannot contain computed store locations.  They are separate
from the package definition, which is inconvenient.

3. snippets feel like less convenient build phases.  Snippets are not
thunked, so we can’t do some things that we would do in a build phase
substitution.  We also can’t access %build-inputs or %outputs.  (I don’t
know if we can use Gexps there.)

I feel that the first point is perhaps a little overvalued.  I have
often felt annoyed that I had to manually apply all this build phase
patching to source code obtained with “guix build -S”, but I never felt
that source code I got from “guix build -S” was too far removed from
upstream.

It may not be possible to apply patches with computed store locations —
because when we compute the source derivation (which is an input to the
package derivation) we don’t yet know the outputs of the package
derivation.  But perhaps we can still agree on a more declarative way to
express patches that are to be applied before the build starts; syntax
that would be more declarative than a serious of brittle substitute*
expressions that latch onto hopefully unique strings in the target
files.

(We have something remotely related in etc/committer.scm.in, where we
define a record describing a diff hunk.)

Here’s a colour sample for the new bikeshed:

  (arguments
(list
  #:patches
  #~(patch "the-file"
 ((line 10)
  (+ "I ONLY WANTED TO ADD THIS LINE"))
 ((line 3010)
  (- "maybe that’s better")
  (+ (string-append #$guix " is better"))
  (+ "but what do you think?")

-- 
Ricardo

Re: Release v1.4 (or 2.0): process and schedule ?

[Chris Marusich]

Maxim Cournoyer  writes:

> About the current status, I'm nearing on pushing a version-1.4.0 branch
> which is based on master with a few more (core-ish) updates.  There's
> still a few days ahead of that, so if you manage to get many of this
> kind of problems fixed & merged in master they can easily be included in
> the next release.

There is a problem that currently prevents "guix pull" from succeeding
for powerpc64le-linux on master.  I'd like to resolve it before the
release if possible.  The problem is here, including a patch to fix it:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=52940

This patch is relatively simple, but it will rebuild many packages for
all architectures because it modifies guix/build/gremlin.scm, which is
used by the gnu-build-system.  How can I integrate this change into the
1.4.0 release?

Normally I would commit such a change to core-updates.  However, if I do
that, then the change probably won't make it into master or the
version-1.4.0 branch in time.  Is there an opportunity to put the change
somewhere so that it will make it into the release?  I'm not sure.

Here's my proposal.  Since the bug may very well be benign, I could
apply a simple work-around to master that just skips the failing test on
powerpc64le-linux.  I could then apply the actual fix to core-updates.
Later, after master has been merged to core-updates, I could re-enable
the test on core-updates.  This would allow "guix pull" to succeed for
powerpc64le-linux on master without rebuilding the world, and the
correct fix would still be applied on core-updates for a later release.

Do you think that would work?

-- 
Chris

PGP: https://savannah.gnu.org/people/viewgpg.php?user_id=106836


signature.asc
Description: PGP signature

Re: On raw strings in commit field

[Liliana Marie Prikler]

Hi simon,

> Please re-read all your answers and mines.  I hope you will see where
> you were incorrect.
I don't think there's anything to see here.  Believe it or not, but
you've so far been boiling my water multiple times only to then throw
it into my face as I attempt to put the rice in.  Admittedly, that's a
little frustrating.  However, I am a big girl and I can handle getting
hot and wet.

> [F]or what my opinion is worth on that matter, my probably wrong
> understanding of your words is that perhaps you are missing a point
> about content-addressability.

Am Dienstag, dem 04.01.2022 um 14:15 +0100 schrieb zimoun:
> Let pick one commit in the Git history, for instance:
> e598e46913c661bc92df813d537eeb6be5a86471.  [...]
> 
> Explain me.  « Git commit hash only depends on the content itself,
> i.e., Git commit object », as I wrote.
That's exactly the point.  For the entirety of this discussion, I've
been assuming the content (i.e. "the content content" or "the content
without meta-content" or however else you want to term it) to be "that
which is hashed by Guix", which if using git-fetch is the working
directory (using Git parlance correctly here, hopefully, correct me if
not) sans the Git subdirectory.  I am sure you have at least a rough
understanding of how that ought to work yourself, but for a more in-
depth analysis of what goes into that, see Timothy's message
> Note for all of this that my scripts treat the SHA256 hash as *the*
> identifier for a source.  That is, if a tag is mutated and a someone
> adjusts the origin URI to point to the commit that the tag used to
> refer to, I would not notice.  Similarly, for tricking peer review:
> fixing the URI to match the hash is invisible to me.  It’s only when we
> fix the hash to match the URI that I notice.
from 87ee5pspza@ngyro.com

> Instead of taking a superior tone «(you can figure out yourself
> where)», I would prefer that you correctly read the messages I
> wrote.  Maybe, that’s why my previous email is probably is bit harsh,
> sorry.
I apologize, I had not intended that to be a superior tone.  I wanted
this to be a less authoritarian version of « you have to figure out
yourself where », leaving open some room for you to not bother any
longer, but my attempt failed.

For the sake of transparency, you are (in my opinion at least) making a
leap here in that you think I somehow care about the hash of a commit
message, which in fact I couldn't care less about other than the
obvious fact that it changes with it.  I can't pinpoint where exactly
along these lines you might have mistakenly got that impression or I
might mistakenly have conveyed it, but you appear to have a rather
convincing reason for you to do so.  Therefore, before going off on yet
another tangent I wanted you to make sure whether that is in fact the
case.


In short, not all content addressing schemes are equal and the content
by which Git addresses its commits is completely irrelevant to Guix (by
virtue of it deleting its means to do so anyway).  I hope that cleared
up any misconceptions.  If not, feel free to ask.

Cheers

Re: On raw strings in commit field

[zimoun]

Hi Liliana,

On Tue, 4 Jan 2022 at 20:45, Liliana Marie Prikler
 wrote:

> I don't think there's anything to see here.  Believe it or not, but
> you've so far been boiling my water multiple times only to then throw
> it into my face as I attempt to put the rice in.  Admittedly, that's a
> little frustrating.  However, I am a big girl and I can handle getting
> hot and wet.

I apologize.  I was not my intent.

For the rest, we said that we have to say.

Cheers,
simon

Re: On raw strings in commit field

[Liliana Marie Prikler]

Am Montag, dem 03.01.2022 um 18:14 -0500 schrieb Mark H Weaver:
> 
> > If you are talking specifically about the uncountability of real
> > numbers, that'd be quite deep down (as in an uncountability of push
> > actions to a particular Git repo, particularly if we also allow
> > reinitialization).
> 
> Hmm.  I think that the set of push actions to a particular Git repo
> is countable, even if we allow reinitialization.  What makes you
> think that it's uncountable?  I'd be very curious to see your
> argument for that.
I can always add another by force-pushing a tree with a single commit,
garbage collecting and then pushing regular commits on top, which
should be able to reuse previously existing hashes.  Even if not, since
we identify a repo by URL, I could ask the forge to delete it and then
push a completely reinitialized repo under the same name.

Speaking about destructive operations, there is one course of actions
that an upstream could take if they deem 2^160 to be too small an
address space, which would be sane for consumers using tags, but insane
for those using commits.  And that would be squashing all the commits
between two tags to a single one, correctly reapplying tags and (force-
)pushing the resulting repo.  Though again, the blow for commit users
might be softened through git vanity this time :)

I'm not saying the above should have any impact on what we feed to git-
fetch, but it's a fun shower thought.
> 

Re: Formalizing teams

[adriano]

Il giorno lun, 03/01/2022 alle 16.22 +0100, Ludovic Courtès ha scritto:


> This brings a related but slightly different topic: how to let people
> filter incoming patches and bug reports in general.
> 
> How does it even work on git*.com?  Do they let you subscribe to
> issues/merge requests that match certain patterns or touch certain
> files?

I don't remember exactly about Github and Gitlab

But I do remember about the Canonical issue tracking system for Ubuntu

When you file a bug there, you are asked to include a list of
packages/areas of the system you think your bug is about

That automatically informs the right people/teams

Also on Stackoverflow, you can accompany your question with some labels
(tags) and every tag is "followed" by some concerned people

Re: On raw strings in commit field

[Mark H Weaver]

Hi Liliana,

Liliana Marie Prikler  writes:

> Am Montag, dem 03.01.2022 um 18:14 -0500 schrieb Mark H Weaver:
>> 
>> > If you are talking specifically about the uncountability of real
>> > numbers, that'd be quite deep down (as in an uncountability of push
>> > actions to a particular Git repo, particularly if we also allow
>> > reinitialization).
>> 
>> Hmm.  I think that the set of push actions to a particular Git repo
>> is countable, even if we allow reinitialization.  What makes you
>> think that it's uncountable?  I'd be very curious to see your
>> argument for that.

> I can always add another by force-pushing a tree with a single commit,
> garbage collecting and then pushing regular commits on top, which
> should be able to reuse previously existing hashes.  Even if not, since
> we identify a repo by URL, I could ask the forge to delete it and then
> push a completely reinitialized repo under the same name.

Sorry, but this is not even close to a valid argument that the set of
possible push actions to a Git repo is uncountable.  In fact, it's quite
easy to prove that the set is countable.  Any mathematician will know this.

I'll also note that the Cantor-style diagonalization argument, which you
have repeatedly claimed to have, is still nowhere to be found.  To make
matters worse, you gaslit me earlier by claiming that I was failing to
see it in your earlier message, when it's clearly not there.  If you
have such an argument, you're apparently unwilling to share it with us.

   Mark

-- 
Disinformation flourishes because many people care deeply about injustice
but very few check the facts.  Ask me about .

Re: Guix Package Search API Server

[Tissevert]

Hi,

This JSON file sounds nice and useful. By the way, it may only be a
transient error but the instance of hpcguix-web running at UMC Utrecht
mentioned in the github repos seems to be down
(https://hpcguix.op.umcutrecht.nl/). Is it supposed to be so ?

Tissevert

Le Mon, 03 Jan 2022 17:27:56 +0100,
Ludovic Courtès  a écrit :

> Hi!
> 
> Sai Karthik  skribis:
> 
> > Nice! I didn't knew about this earlier. Does hpc.guix.info exposes
> > API ?  
> 
> It runs hpcguix-web¹, which exposes a big (but gzipped) JSON file at
> .  Everything else happens in
> client-side JavaScript code.
> 
> Ludo’.
> 
> ¹ https://github.com/UMCUGenetics/hpcguix-web
>