Re: Hashing a procedure object reliably

[Vivien Kraus]

Hello,

Le vendredi 11 avril 2025 à 09:49 -0400, Olivier Dion a écrit :
> But some fields seems to
> not be taken into account and you will end up with fetching
> substitute
> in the store.  I want to avoid this element of surprise in my case.

>From what I understand, here (problem A), you want to avoid a hash
collision (two different codes hashing to the same value)…

> Like mentioned in the comment (look for the `(procedure? value)'
> clause), this does _not_ yield the same hash for a procedure if the
> module where this procedure came from is compiled.

And here (problem B), you want to avoid having equivalent code (source
vs bytecode) hashing to different values.

Did I summarize it correctly?

Solving A is easy: generate a new unique identifier each time you hash
any functional value (or refuse to cache the result).

I don’t have an answer for B, but others may.

However, considering problem B, it may be more complex than just source
and bytecode. What if the GOOPS object creator uses different
dependencies, but they turn out to have the same behavior? I’m thinking
about a non-free/free replacement situation. Since the object creator
has a clearer vision of the functions and dependencies they use, maybe
you could ask them to provide a version number of their relevant
functional code, and hash this version number instead?

Best regards,

Vivien

Re: Hashing a procedure object reliably

[Zelphir Kaltstahl]

On 11.04.25 21:49, Olivier Dion wrote:

Hi Guile users!

I have a hashing procedure that can hash many things like so:

   (define (hash value)
 (cond
  ((pair? value)
   (let loop ((elements value)
  (acc (srfi:hash '()))
  (k 1))
 (if (null? elements)
 acc
 (loop (cdr elements)
   (+ acc (* k (hash (car elements
   (1+ k)
  ((vector? value)
   (let loop ((acc (srfi:hash (vector)))
  (k 0))
 (if (= k (vector-length value))
 acc
 (loop (+ acc (* (1+ k) (hash (vector-ref value k
   (1+ k)
  ((bytevector? value)
   (let loop ((acc (srfi:hash #vu8()))
  (k 0))
 (if (= k (bytevector-length value))
 acc
 (loop (+ acc (* (1+ k) (hash (bytevector-u8-ref value k
   (1+ k)
  ;; FIXME:  The hashing of a procedure is not the same after compilation.
  ;; ((procedure? value)
  ;;  (match (program-address-range value)
  ;;((start . stop)
  ;; ;; Adding the hash of 'procedure so that it does not give the same 
hash
  ;; ;; for an identical bytevector.
  ;; (hash (+ (srfi:hash 'procedure)
  ;;  (hash (pointer->bytevector (make-pointer start)
  ;; (- stop start
  ((or (number? value)
   (string? value)
   (symbol? value)
   (boolean? value))
   (srfi:hash value))
  ((instance? value)
   (let ((class (class-of value)))
 (fold
  (lambda (slot acc)
(+ (if (hash-slot? slot)
   (let ((name (slot-definition-name slot)))
 (* (hash name) (hash (slot-ref value name
   0)
   acc))
  (hash (class-name class))
  (class-slots class
  (else
   0)))

Like mentioned in the comment (look for the `(procedure? value)'
clause), this does _not_ yield the same hash for a procedure if the
module where this procedure came from is compiled.  And it's not
possible to pass the procedure (a reference) to `compile' to get a
compiled version.  I assume that the compiler would need lexical context
for couple of its optimization anyway, so even if it did worked, we
would not get the same bytecode.

Note that hashing the source code won't work either because:

   1. If not compiled, there is no source location available, thus not
   usable in a REPL.

   2. Hashing of the source does not make sense in my point of view.
   Adding a newline to a function does not change its bytecode and should
   yield the same hash.

So I wonder if somebody has an idea on how to have reliable hash of
procedure. That is, a given procedure will yield the same hash in
different Guile processes, compiled or not.

My goal here is that I have GOOPS object.  The object is used to produce
a pure result which I can store in a cache on the disk, given the hash
of the object.  I can then re-fetch the result on disk in another Guile
process if the hashes match.  As you can see in the above code, GOOPS
instances get hashed by folding over their slots, which can include
procedures.

If you have ever used Guix and change the value of a package field and
ask to build it, usually it will rebuild it.  But some fields seems to
not be taken into account and you will end up with fetching substitute
in the store.  I want to avoid this element of surprise in my case.

Thanks,
Olivier
  


I don't have a solution to this, but when I read the first lines of your 
message, I thought already: "But Guile does not provide code/syntax location!"


It would be great, if Guile did, and we could have similar things to what Racket 
has with typed Racket and such.


Best regards, Zelphir

--
repositories:https://notabug.org/ZelphirKaltstahl,https://codeberg.org/ZelphirKaltstahl

Hashing a procedure object reliably

[Olivier Dion]

Hi Guile users!

I have a hashing procedure that can hash many things like so:

  (define (hash value)
(cond
 ((pair? value)
  (let loop ((elements value)
 (acc (srfi:hash '()))
 (k 1))
(if (null? elements)
acc
(loop (cdr elements)
  (+ acc (* k (hash (car elements
  (1+ k)
 ((vector? value)
  (let loop ((acc (srfi:hash (vector)))
 (k 0))
(if (= k (vector-length value))
acc
(loop (+ acc (* (1+ k) (hash (vector-ref value k
  (1+ k)
 ((bytevector? value)
  (let loop ((acc (srfi:hash #vu8()))
 (k 0))
(if (= k (bytevector-length value))
acc
(loop (+ acc (* (1+ k) (hash (bytevector-u8-ref value k
  (1+ k)
 ;; FIXME:  The hashing of a procedure is not the same after compilation.
 ;; ((procedure? value)
 ;;  (match (program-address-range value)
 ;;((start . stop)
 ;; ;; Adding the hash of 'procedure so that it does not give the same 
hash
 ;; ;; for an identical bytevector.
 ;; (hash (+ (srfi:hash 'procedure)
 ;;  (hash (pointer->bytevector (make-pointer start)
 ;; (- stop start
 ((or (number? value)
  (string? value)
  (symbol? value)
  (boolean? value))
  (srfi:hash value))
 ((instance? value)
  (let ((class (class-of value)))
(fold
 (lambda (slot acc)
   (+ (if (hash-slot? slot)
  (let ((name (slot-definition-name slot)))
(* (hash name) (hash (slot-ref value name
  0)
  acc))
 (hash (class-name class))
 (class-slots class
 (else
  0)))

Like mentioned in the comment (look for the `(procedure? value)'
clause), this does _not_ yield the same hash for a procedure if the
module where this procedure came from is compiled.  And it's not
possible to pass the procedure (a reference) to `compile' to get a
compiled version.  I assume that the compiler would need lexical context
for couple of its optimization anyway, so even if it did worked, we
would not get the same bytecode.

Note that hashing the source code won't work either because:

  1. If not compiled, there is no source location available, thus not
  usable in a REPL.

  2. Hashing of the source does not make sense in my point of view.
  Adding a newline to a function does not change its bytecode and should
  yield the same hash.

So I wonder if somebody has an idea on how to have reliable hash of
procedure. That is, a given procedure will yield the same hash in
different Guile processes, compiled or not.

My goal here is that I have GOOPS object.  The object is used to produce
a pure result which I can store in a cache on the disk, given the hash
of the object.  I can then re-fetch the result on disk in another Guile
process if the hashes match.  As you can see in the above code, GOOPS
instances get hashed by folding over their slots, which can include
procedures.

If you have ever used Guix and change the value of a package field and
ask to build it, usually it will rebuild it.  But some fields seems to
not be taken into account and you will end up with fetching substitute
in the store.  I want to avoid this element of surprise in my case.

Thanks,
Olivier
 
-- 
Olivier Dion
oldiob.ca

Re: Hashing a procedure object reliably

[Olivier Dion]

On Fri, 11 Apr 2025, Vivien Kraus  wrote:
> Hello,
>
> Le vendredi 11 avril 2025 à 09:49 -0400, Olivier Dion a écrit :
>> But some fields seems to
>> not be taken into account and you will end up with fetching
>> substitute
>> in the store.  I want to avoid this element of surprise in my case.

Side note, my problem is outside of Guix entirely.  I just gave this as
an example of caching behavior I want to avoid in my project.

> From what I understand, here (problem A), you want to avoid a hash
> collision (two different codes hashing to the same value)…

Right.  We don't want `(lambda (x y) (+ x y))' to have the same hash as
`(lambda (x y) (* x y))'.  Otherwise, users would get a big suprise to
see that it get the same cached result.

>> Like mentioned in the comment (look for the `(procedure? value)'
>> clause), this does _not_ yield the same hash for a procedure if the
>> module where this procedure came from is compiled.
>
> And here (problem B), you want to avoid having equivalent code (source
> vs bytecode) hashing to different values.
>
> Did I summarize it correctly?

I think the definition of equivalent here is that the pure behavior of
the procedures are the same.  Say we're hashing `(lambda (x y) (+ x y))'
and `(lambda (u v) (+ u v))'.  These procedures does not have equivalent
sources, but I do have equivalent semantics.  This is why the bytecode
is hashed instead, because it represents the semantic of the procedure
at the machine level.  Alas, If you rehash the same procedure after
compilation (often auto-compilation), you get a different hash of
course.

> Solving A is easy: generate a new unique identifier each time you hash
> any functional value (or refuse to cache the result).

This assumes the same hashing ordering from process to process.  Hash
must be identical for every invokation of Guile.  But really A is not a
problem here I think.

> I don’t have an answer for B, but others may.

I do have an idea.  I'm willing to have `define#' and `lambda#'
syntax-transformers.  Users would use it to define procedures that are
important to hash.  The idea would be to evaluate the source of the
procedure at expansion time and hash this procedure bytecode.  The
resulting hash is thus computed at expansion time and stored as a
procedure property.

Here's the implementation:

--8<---cut here---start->8---
  (use-modules
   (ice-9 match)
   (rnrs bytevectors)
   ((srfi srfi-69) #:prefix srfi:)
   (system foreign)
   (system vm program)
   )

  (eval-when (expand load compile)
(define (procedure-bytecode proc)
  (match (program-address-range proc)
((start . stop)
 (pointer->bytevector (make-pointer start)
  (- stop start)
(define (hash-bytevector bv)
  (let loop ((acc (srfi:hash #vu8()))
 (k 0))
(if (= k (bytevector-length bv))
acc
(loop (+ acc (* (1+ k) (srfi:hash (bytevector-u8-ref bv k
  (1+ k)
(define (hash-expression formals body body*)
  (false-if-exception
   (hash-bytevector
(procedure-bytecode
 (eval `(lambda (,@formals) ,body ,@body*)
   (current-module)))

  (define-syntax define#
(lambda (stx)
  (syntax-case stx ()
((_ (name formals ...) body body* ...)
 (identifier? #'name)
 (let ((the-hash (hash-expression (syntax->datum #'(formals ...))
  (syntax->datum #'body)
  (syntax->datum #'(body* ...)
   (with-syntax ((set-hash! (if the-hash
#`(set-procedure-property! name 'hash 
#,the-hash)
#'(begin
 #'(begin
 (define* (name formals ...)
   body body* ...)
 set-hash!)))

  (define-syntax lambda#
(lambda (stx)
  (syntax-case stx ()
((_ (formals ...) body body* ...)
 (let ((the-hash (hash-expression (syntax->datum #'(formals ...))
  (syntax->datum #'body)
  (syntax->datum #'(body* ...)
   #`(let ((proc (lambda (formals ...) body body* ...)))
   #,(if the-hash
 #`(set-procedure-property! proc 'hash #,the-hash)
 #'(begin))
   proc))

  (define (procedure-hash proc)
(or (procedure-property proc 'hash) 0))

  (define# (foo x y)
(+ x y))

  (define# (bar x y)
(+ x y))

  (pk
   (procedure-hash foo)
   (procedure-hash bar)
   (procedure-hash (lambda# (u v) (+ u v)))
   )
--8<---cut here---end--->8---

> However, considering problem B, it may be more complex than just
> source and bytecode. What if the GOOPS object creator uses different
> dependencies, but they turn out to have the same behavior? 

First example of g-golf (hello world) not working

[Zelphir Kaltstahl]

Hello Guile Users!

Yesterday I decided to give g-golf a try. Unfortunately I did not get very far 
at all. I tried running the hello world example, and it already crashed:


main.scm

 Copyright (C) 2022 - 2023
 Free Software Foundation, Inc.

 This file is part of GNU G-Golf

 GNU G-Golf is free software; you can redistribute it and/or modify
 it under the terms of the GNU Lesser General Public License as
 published by the Free Software Foundation; either version 3 of the
 License, or (at your option) any later version.

 GNU G-Golf is distributed in the hope that it will be useful, but
 WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 Lesser General Public License for more details.

 You should have received a copy of the GNU Lesser General Public
 License along with GNU G-Golf.  If not, see
.


;;; Commentary:

;;; Code:

(eval-when (expand load eval)
  (use-modules (oop goops))

  (default-duplicate-binding-handler
'(merge-generics replace warn-override-core warn last))

  (use-modules (g-golf))

  (g-irepository-require "Gtk" #:version "4.0")
  (for-each (lambda (name)
  (gi-import-by-name "Gtk" name))
  '("Application"
"ApplicationWindow"
"Box"
"Label"
"Button")))


(define (activate app)
  (let ((window (make 
  #:title "Hello"
  #:default-width 320
  #:default-height 240
  #:application app))
(box(make 
  #:margin-top 6
  #:margin-start 6
  #:margin-bottom 6
  #:margin-end 6
  #:orientation 'vertical))
(label  (make 
  #:label "Hello, World!"
  #:hexpand #t
  #:vexpand #t))
(button (make 
  #:label "Close")))

(connect button
 'clicked
 (lambda (b)
   (close window)))

(set-child window box)
(append box label)
(append box button)
(present window)))


(define (main args)
  (let ((app (make 
   #:application-id "org.gtk.example")))
(connect app 'activate activate)
(let ((status (g-application-run app args)))
  (exit status


Running this results in the following error:


guix shell: checking the environment variables visible from shell '/bin/bash'...
guix shell: All is good!  The shell gets correct environment variables.
;;; note: source file /home/hans/dev/guile/gui/g-golf/example-01/main.scm
;;;   newer than compiled 
/home/hans/.cache/guile/ccache/3.0-LE-8-4.6/home/hans/dev/guile/gui/g-golf/example-01/main.scm.go
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;   or pass the --no-auto-compile argument to disable.
;;; compiling /home/hans/dev/guile/gui/g-golf/example-01/main.scm
**
GLib-GObject:ERROR:../glib-2.82.1/gobject/gbinding.c:874:g_binding_class_init: 
assertion failed: (gobject_notify_signal_id != 0)
Bail 
out!GLib-GObject:ERROR:../glib-2.82.1/gobject/gbinding.c:874:g_binding_class_init:
 assertion failed: (gobject_notify_signal_id != 0)
make: *** [Makefile:18: run] Error 134


I am running it in a guix shell, which has the package "guile" and the package 
"guile-g-golf" installed.


channels.scm
(list (channel
(name 'guix)
(url"https://git.savannah.gnu.org/git/guix.git";)
(branch "master")
(commit
  "f0c0769189d11debf7b237a02695c44c9773d52a")
(introduction
  (make-channel-introduction
"9edb3f66fd807b096b48283debdcddccfea34bad"
(openpgp-fingerprint
  "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA")


manifest.scm
(specifications->manifest
 '("guile"
   "guile-g-golf"
   "make"
   "bash"))


Using time machine:


guix time-machine --channels=guix-env/channels.scm -- shell --check 
--manifest=guix-env/manifest.scm -- bash -c guile -L . main.scm


What is going wrong?

Is the example code meant to be run in a different way?

Does anyone know how to deal with this error and make it work?

Best regards,
Zelphir

--
repositories:https://notabug.org/ZelphirKaltstahl,https://codeberg.org/ZelphirKaltstahl

Re: Hashing a procedure object reliably

[Thompson, David]

Hi Olivier,

On Fri, Apr 11, 2025 at 9:50 AM Olivier Dion  wrote:
>
> Like mentioned in the comment (look for the `(procedure? value)'
> clause), this does _not_ yield the same hash for a procedure if the
> module where this procedure came from is compiled.  And it's not
> possible to pass the procedure (a reference) to `compile' to get a
> compiled version.  I assume that the compiler would need lexical context
> for couple of its optimization anyway, so even if it did worked, we
> would not get the same bytecode.
>
> Note that hashing the source code won't work either because:
>
>   1. If not compiled, there is no source location available, thus not
>   usable in a REPL.
>
>   2. Hashing of the source does not make sense in my point of view.
>   Adding a newline to a function does not change its bytecode and should
>   yield the same hash.
>
> So I wonder if somebody has an idea on how to have reliable hash of
> procedure. That is, a given procedure will yield the same hash in
> different Guile processes, compiled or not.

Hashing by procedure content is not something you can do.  Procedures
are *opaque* objects.  As you've seen, the same procedure can be
represented in many different ways at runtime. What does it even mean
for two procedures to be the same, anyway?

Since procedures are opaque, the only hash that makes sense is hashq,
which hashes by object identity. If (eq? a b) is #t then the hash code
will be the same.

- Dave

Re: Hashing a procedure object reliably

[tomas]

On Fri, Apr 11, 2025 at 11:34:37AM -0400, Olivier Dion wrote:

[...]

> I think the definition of equivalent here is that the pure behavior of
> the procedures are the same.  Say we're hashing `(lambda (x y) (+ x y))'
> and `(lambda (u v) (+ u v))'.  These procedures does not have equivalent
> sources, but I do have equivalent semantics [...]

This problem in its general form is, alas, undecidable. You might try to
restrict it in some useful ways, if I remember correctly.

Cheers
-- 
t 


signature.asc
Description: PGP signature

Re: First example of g-golf (hello world) not working

[David Pirotte]

Hello Zelphir,

> Yesterday I decided to give g-golf a try. Unfortunately I did not get
> very far at all. I tried running the hello world example, and it
> already crashed:

You can't run the upstream version of the g-golf distributed examples
in guix, due to a known problem (in guix, this has nothing to do with
guile nor g-golf itself), they must be wrapped so they are being run in
an 'appropriate way' (that is using the --no-graft guix shell command).

Guix now two separate g-golf examples packages:

g-golf-gtk-4-examples
g-golf-adw-1-examples

You should install those package and run them from your profile bin
dir (iiuc, not a guix user).

To see and learn how to wrap your own g-golf app, in guix, you may do
(I am quoting a sentence of the package description):

Run @command{guix edit g-golf-gtk-4-examples} for inspiration
how to wrap G-Golf applications when writing a Guix package.

Cheers,
David


pgpTvqzCD3EZN.pgp
Description: OpenPGP digital signature