Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

Maxime Devos  writes:
> ```
> #!r6rs
> (library (foo)
>(export foo)
>(import (rnrs base))
>(define foo
>  (cons 'a 1)))
> ```
> Here you are demonstrating how R6RS libraries have the _same_ problem. You 
> should at least have included a version number next to (rnrs
> base). Who knows, maybe R8RS will rename ‘cons’ to ‘make-pair’?

Keep in mind that adding a version number does not resolve
backwards compatibility. It just moves it so source maintenance instead
of package maintenance — and then adds more complexity in keeping the
system compatible with different versions of itself.

You’ll then have to test every version of Scheme with every other
version, else you get breakage the moment a library updates to a newer
version of Scheme while another is still on the old version.

You can use methods to shift the load of backwards compatibility to
different groups, but you cannot remove it.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de


signature.asc
Description: PGP signature

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

"Thompson, David"  writes:
>  wrote:
>> I will suggest that if a consensus is reached on what new code must
>> adhere to (e.g. no (guile) namespace pollution, no new procedures
>> written in C), it should be documented in the reference manual.
>
> Right, there is no consensus about this right now.

If there were consensus, there would be less talk about it :-)

>> In the specific context of this patch I feel like the risks of namespace
>> pollution and C maintenance burden are low since both are near identical
>> to what already exists.
>
> I agree that the risks are low in this case. It doesn't make sense to
> have destructive list operations without nondestructive equivalents,
> especially since mutation of pairs is strongly discouraged amongst
> seasoned Schemers.  If it were my call (and it's not, to be clear),
> I'd say that they should just be implemented in Scheme, at least.
> There really is no good reason to implement Scheme procedures in C
> anymore.

I think harmonizing the existing codebase and moving to Scheme should be
seen as orthogonal.

This does not increase the work when moving delete to Scheme by much
(asking in the conversion "was there a good reason not to have a
non-mutating version" might actually take up more time than just doing
the conversion alongside), so I would suggest to merge the patch.

There are nuances which could turn harmonizing into expanding the C in
the codebase (and thus creating a moving target for reducing the C
core), but I think this patch is purely harmonizing

>> It's quite possible as a first-time contributor I am missing something
>> that already exists. If so, oops again!
>
> Nope! Not missing a thing. There's literally no way for you to know
> that there's a desire amongst some Guilers to discourage implementing
> new standard library procedures in C and expanding the default
> namespace.

> I really hope I haven't discouraged you from future contributions by
> using your patch as a case study. Thank you again for hacking on
> Guile!

I want to second this: thank you for your patch!

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de


signature.asc
Description: PGP signature

Performance test results Guile 3.0.8 / 3.0.9 / 3.0.10

[Dr. Arne Babenhauserheide]

Hi,


I ran and evaluated ecraven’s r7rs-benchmarks for Guile 3.0.8 to 3.0.10
and found 3.0.10 to be around 3% faster than 3.0.8 and 3.0.9.

I’m actually calculating the geometric mean of the slowdown of different
tests compared to the fastest:


=== Guile--V3.0.10 Geometric Mean slowdown (successful tests / total tests) ===

1.0241580763853801 (55 / 57)
--
=== Guile--V3.0.9 Geometric Mean slowdown (successful tests / total tests) ===

1.0601747419740355 (55 / 57)
--
=== Guile--V3.0.8 Geometric Mean slowdown (successful tests / total tests) ===

1.0568441538253395 (55 / 57)


This was only run once, so take it with a grain of salt. Or better
still: run it yourself and post the result, so we can compare!


To reproduce on Guix:

export PROGRAMS=/tmp # adjust this
cd $PROGRAMS || exit 1
git clone https://github.com/ecraven/r7rs-benchmarks
git clone https://git.savannah.gnu.org/git/guile.git
hg clone https://hg.sr.ht/~arnebab/wisp
export VERSIONS="v3.0.10 v3.0.9 v3.0.8";
 cd $PROGRAMS/r7rs-benchmarks;
 for i in $VERSIONS; do
 (cd $PROGRAMS/guile;
  git checkout $i;
  guix shell -D guile gperf sed guile -- \
   bash -x -c 'make clean; autoreconf -i; ./configure CFLAGS="$CFLAGS 
-march=native"; make -j6');
 GUILE=$PROGRAMS/guile/meta/guile ./bench guile all;
 cat results.Guile >> results.Guile--$i && rm results.Guile;
 done
 rm all.csv;
 for i in $VERSIONS; do
 grep -a -h '+!CSVLINE' results.Guile--$i | sed s/guile/guile--$i/g | sed 
's/+!CSVLINE!+//' >> all.csv;
 done
 for i in $VERSIONS; do
 $PROGRAMS/wisp/examples/evaluate-r7rs-benchmark.w 
$PROGRAMS/r7rs-benchmarks/all.csv guile--$i 2>/dev/null;
 done | grep -A2 "Geometric Mean slowdown"


More detailed output (without the grep, and reducing redundant best time
output):


=== Best times ===

(("ack:3:12:2" . 3.026162621)
 ("array1:100:500" . 2.570030061)
 ("browse:2000" . 4.439979993)
 ("bv2string:1000:1000:100" . 2.772243723)
 ("cat:50" . 37.996525162)
 ("chudnovsky:50:500:50:1000" . 0.13607684)
 ("compiler:2000" . 2.829384225)
 ("conform:500" . 6.061795319)
 ("cpstak:40:20:11:1" . 15.376498663)
 ("ctak:27:16:8:1" . 29.210273056)
 ("deriv:1000" . 6.459855722)
 ("destruc:600:50:4000" . 3.844580263)
 ("diviter:1000:100" . 3.47571884)
 ("divrec:1000:100" . 5.923372727)
 ("dynamic:500" . 5.790662373)
 ("earley:1" . 4.740488712)
 ("fft:65536:100" . 3.663431697)
 ("fib:40:5" . 4.166439342)
 ("fibfp:35.0:10" . 12.587807545)
 ("gcbench:20:1" . 0.99936)
 ("graphs:7:3" . 11.161914468)
 ("lattice:44:10" . 6.287998907)
 ("matrix:5:5:2500" . 3.700863832)
 ("maze:20:7:1" . 2.330800762)
 ("mazefun:11:11:1" . 4.386609981)
 ("mbrot:75:1000" . 26.094648762)
 ("mbrotZ:75:1000" . 26.299364033)
 ("mperm:20:10:2:1" . 7.026928755)
 ("nboyer:5:1" . 1.970066148)
 ("nqueens:13:10" . 7.917849131)
 ("ntakl:40:20:12:1" . 3.363077275)
 ("nucleic:50" . 6.278230475)
 ("paraffins:23:10" . 2.959535594)
 ("parsing:2500" . 5.61743579)
 ("peval:2000" . 5.08106331)
 ("pi:50:500:50:100" . 0.082752902)
 ("pnpoly:100" . 16.038586581)
 ("primes:1000:1" . 3.248385311)
 ("puzzle:1000" . 7.033796782)
 ("quicksort:1:2500" . 4.122845352)
 ("ray:50" . 8.519810821)
 ("read1:2500" . 6.36043769)
 ("sboyer:5:1" . 1.591647452)
 ("scheme:10" . 7.302776431)
 ("simplex:100" . 5.685521242)
 ("slatex:500" . 10.970135207)
 ("string:50:100" . 0.677046739)
 ("sum1:25" . 5.613144143)
 ("sum:1:20" . 1.710376816)
 ("sumfp:100.0:500" . 22.795371313)
 ("tail:50" . 12.964143459)
 ("tak:40:20:11:1" . 2.135131049)
 ("takl:40:20:12:1" . 3.367369662)
 ("triangl:22:1:50" . 2.935341486)
 ("wc:inputs/bib:50" . 12.688892668))

=== Guile--V3.0.10 times ===

(("ack:3:12:2" "3.026162621")
 ("array1:100:500" "2.807721003")
 ("browse:2000" "4.439979993")
 ("bv2string:1000:1000:100" "2.772243723")
 ("cat:50" "38.846298461")
 ("chudnovsky:50:500:50:1000" "0.147779112")
 ("compiler:2000" "2.866082384")
 ("conform:500" "6.061795319")
 ("cpstak:40:20:11:1" "16.299212988")
 ("ctak:27:16:8:1" "29.419785896")
 ("deriv:1000" "6.459855722")
 ("destruc:600:50:4000" "3.886062447")
 ("diviter:1000:100" "4.1036992")
 ("divrec:1000:100" "6.254394067")
 ("dynamic:500" "6.237513291")
 ("earley:1" "4.752008016")
 ("equal" "ULIMITKILLED")
 ("fft:65536:100" "3.663431697")
 ("fib:40:5" "4.36791794")
 ("fibc" "ULIMITKILLED")
 ("fibfp:35.0:10" "12.716975291")
 ("gcbench:20:1" "0.99936")
 ("graphs:7:3" "11.161914468")
 ("lattice:44:10" "6.420426816")
 ("matrix:5:5:2500" "3.813956798")
 ("maze:20:7:1" "2.330800762")
 ("mazefun:11:11:1" "4.953506464")
 ("mbrot:75:1000" "26.225870661")
 ("mbrotZ:75:1000" "26.356033154")
 ("mperm:20:10:2:1" "9.663579251")
 ("nboyer:5:1" "2.005036256")
 ("nqueens:13:10" "7.917849131")
 ("ntakl:40:20:12:1" "3.371785352")
 ("nucleic:50" "6.327269619")
 ("paraffins:23:10" "2.959535594")
 ("parsing:2500" "5.61743579")
 ("peval:2000" "5.19038936")
 ("pi:50:500:50:100" "0.08354783")
 ("pnpoly:100" "16.0855

RE: The Guile junk drawer and a C plea

[Maxime Devos]

>Maxime Devos  writes:
>> ```
>> #!r6rs
>> (library (foo)
>>(export foo)
>>(import (rnrs base))
>>(define foo
>>  (cons 'a 1)))
>> ```
> Here you are demonstrating how R6RS libraries have the _same_ problem. You 
> should at least have included a version number next to (rnrs
> base). Who knows, maybe R8RS will rename ‘cons’ to ‘make-pair’?

>Keep in mind that adding a version number does not resolve
backwards compatibility. It just moves it so source maintenance instead
of package maintenance — and then adds more complexity in keeping the
system compatible with different versions of itself.

>You’ll then have to test every version of Scheme with every other
version, else you get breakage the moment a library updates to a newer
version of Scheme while another is still on the old version.

No it doesn’t, because of _the version number_.  If you put (6) next to the 
module name, that’s the _exact_ version number, it’s not ‘6 or later’ or ‘maybe 
6 whatever’, it’s ‘exactly 6’. You could load multiple versions of a module at 
the same time! (Not actually implemented currently, but it _could_ relatively 
easily be implemented.)

Also, what source maintenance? Software, that isn’t subject to changing 
external demands, doesn’t rot, and a _precise_ version number is included which 
implies the lack of changing external demand.

In terms of implementation, realistically there would be a single module 
version that contains the actual implementation, and various other version 
variants that only say ‘take these binding from that module (vN), with those 
renamings’ (and the occasional ‘this binding was removed / has different 
arguments / ..., let’s provide a small wrapper).

>You can use methods to shift the load of backwards compatibility to
different groups, but you cannot remove it.

I’m pretty sure you can remove backwards compatibility, just replace ‘cons’ by 
‘make-pair’ in Guile code and see what happens.

I think it’s better though to preserve backwards compatibility, e.g. by 
implementing version number support for modules with a _well-defined_ API.

Note the emphasis on _well-defined_ -- if some API (in terms of semantics and 
not just bikeshedding over naming and argument order) is just what we have 
currently instead of something that is expected to remain the same (except 
perhaps for pure additions), and hence could easily change in the future, then 
version numbers are just going to lead to new problems. 

But in the case of _(guile)_, for most things in (guile) this isn’t really the 
case. Likewise, (rnrs ...) is pretty well-defined – there are some changes to 
RnRS stuff, but it’s mostly just additions, removals and reorderings – the 
semantics appear to have remained pretty much the same.

Best regards,
Maxime Devos

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

Maxime Devos  writes:
> No it doesn’t, because of _the version number_. If you put (6) next to
> the module name, that’s the _exact_ version number, it’s not ‘6 or
> later’ or ‘maybe 6 whatever’, it’s ‘exactly 6’. You could load
> multiple versions of a module at the same time! (Not actually
> implemented currently, but it _could_ relatively easily be
> implemented.)

Yes, that’s what I mean.

And then I use a second library which gives me a comparator, so I pass
equal? from (rnrs base 7) to hash-table from (rnrs something 6).

Either this is allowed, then I just doubled my testing area, or this
is forbidden, then programs break on update of a library and need to
update all their libraries to the new dependency version.

> Also, what source maintenance? Software, that isn’t subject to changing 
> external demands, doesn’t rot, and a _precise_ version number
> is included which implies the lack of changing external demand.

I am writing about not breaking on update.

> In terms of implementation, realistically there would be a single module 
> version that contains the actual implementation, and various
> other version variants that only say ‘take these binding from that module 
> (vN), with those renamings’ (and the occasional ‘this binding
> was removed / has different arguments / ..., let’s provide a small wrapper).

I’m seeing in Guix how much more complex this approach makes packaging
Haskell-programs. How Rust programs spawn 500 MiB of
source-with-dependency for small features.

>>You can use methods to shift the load of backwards compatibility to
>> different groups, but you cannot remove it.
>
> I’m pretty sure you can remove backwards compatibility

You cannot remove *the load of backwards compatibility*.

> But in the case of _(guile)_, for most things in (guile) this isn’t really 
> the case. Likewise, (rnrs ...) is pretty well-defined – there are some
> changes to RnRS stuff, but it’s mostly just additions, removals and 
> reorderings – the semantics appear to have remained pretty much
> the same.

Yes, and that is how it should be.

It’s when you don’t actually need version numbers. And if you added
them, you’d still have to ensure compatibility between different
versions, because libraries you use will have to be updated, even if
your own code just keeps working.

That’s why I say that the load of backwards compatibility cannot be
removed: it can only be shifted on other people.

But when the culture shifts so people say “hey, we have versioned APIs
now, let’s change everything around to fit this new style; it won’t
break existing clients (until we remove the old version)” (can you say
that you never saw people do that? I did see it), then it causes serious
problems.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de


signature.asc
Description: PGP signature

Re: Performance test results Guile 3.0.8 / 3.0.9 / 3.0.10

[Dr. Arne Babenhauserheide]

Hi,

"Dr. Arne Babenhauserheide"  writes:
> === Guile--V3.0.10 Geometric Mean slowdown (successful tests / total tests) 
> ===
>
> 1.0241580763853801 (55 / 57)
> --
> === Guile--V3.0.9 Geometric Mean slowdown (successful tests / total tests) ===
>
> 1.0601747419740355 (55 / 57)
> --
> === Guile--V3.0.8 Geometric Mean slowdown (successful tests / total tests) ===
>
> 1.0568441538253395 (55 / 57)

a repeated test with more consistent system load gave a similar result:

=== Guile--V3.0.10 Geometric Mean slowdown (successful tests / total tests) ===

1.0197861099786674 (54 / 57)
--
=== Guile--V3.0.9 Geometric Mean slowdown (successful tests / total tests) ===

1.0478708351165291 (54 / 57)
--
=== Guile--V3.0.8 Geometric Mean slowdown (successful tests / total tests) ===

1.0476417362114068 (54 / 57)

Best wishes,
Arne

-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de


signature.asc
Description: PGP signature

Re: [BUG] Eval sets incorrect runtime metainformation

[Philip McGrath]

On 6/29/24 19:05, Maxime Devos wrote:

> Instead of run-time reflection on values, I think an IDE implementing
> jump to definition should use source location and binding information
> from syntax objects. That's how DrRacket does it.
> ...
> Since syntax objects from the expander encode all the details of scope,
> this works mostly automatically. DrRacket makes this functionality
> available as a library, so navigation features can also be used from
> Emacs with racket-xp-mode [2] or other editors with the Language Server
> Protocol [3].

I suppose this could work, but it seems the wrong layer of abstraction 
and a bit round-about to me, it also only works on languages that use 
Scheme-style syntax. Instead, I’d propose compiling the relevant code to 
Tree-IL (without optimisations, should work with most languages).  (For 
Scheme, this is essentially macro expansion, but the resulting object 
won’t technically be a syntax object.)


In Tree-IL, all syntax has been expanded (in tree form), which 
eliminates a lot of complications. Yet, source location information 
remains available (albeit not documented ...)! And the original variable 
names remain available (together with an unshadow-ified version, so for 
local variables you can move upwards to find the corresponding 
definition and in particular its source location.


A large upside is that this should work for most languages (not only 
Scheme) and is relatively straightforward to implement,


I hadn't realized that other languages in Guile might compile to Tree-IL 
directly instead of generating syntax objects. Is that common and/or 
encouraged? It seems like it would require the new language's compiler 
to do a lot of work that could otherwise be delegated to Guile, and it 
would make it difficult to implement parts of the new language using macros.


a small downside 
is that information on definitions like (let-syntax ((f [...])) ...) 
aren’t available – you only see the return of (f stuff), not ‘f’ itself.


(But this downside applies to the ‘syntax’ route as well, unless you are 
doing complicated (but possible!) DIY partial macro expansion shenanigans.)




This and other features, like correctly tracking the binding of `else` 
in `cond`, require cooperation from the macro expander. The explanation 
of origin tracking in 
 might be one 
place to start reading about how Racket supports this (but I am not an 
expert!). Similar expander features can also enable other tools, like 
DrRacket's macro stepper: 



Philip

Re: GNU Guile 3.0.10 released

[Maxim Cournoyer]

Hi Andy and other Guile maintainers,

Andy Wingo  writes:

> We are pleased to announce GNU Guile release 3.0.10, the latest in the
> 3.0 stable release series.
>
> Guile 3.0.10 is a bug-fix release which also adds optimizations, support
> for a new WebAssembly back-end, new custom port facilities, a new
> optional front-end syntax, and more.  See the NEWS extract at the end of
> the mail for full details.

[...]

Wow, thank you for writing this detailed release news
entry, and for making the release happen!  It seems to be packed with
many interesting improvements, I look forward to using it.

-- 
Thanks,
Maxim

Re: The Guile junk drawer and a C plea

[Maxim Cournoyer]

Hi David,

"Thompson, David"  writes:

> Hi Richard and all other Guilers, too,
>
> What follows is not code review, but your patch felt like an
> opportunity to provide some commentary about the trajectory of Guile
> development that I've wanted to share for awhile.
>
> First, I think Guile's default environment is a total mess.  It's the
> very definition of a junk drawer.  There's over 1000 names in the
> (guile) module!  Contrast this with R7RS-small's (scheme base) module
> that only has 200ish.  Guile is an old project and I'm sure stuff just
> accumulated over the years, but having so much in the default
> environment makes it hard to know what a program actually uses because
> many things that ought to be explicit imports are not. This makes it a
> challenge to move Guile in a more "least authority" direction.  As a
> rule, I think Guile should *not* add any additional names to the
> default environment without an extremely good reason. Because (guile)
> is imported implicitly, new names can cause clashes with existing code
> that require #:replace to suppress the warning about shadowing core
> bindings.  For example, the newish 'spawn' procedure collides with
> 'spawn' in (goblins core) in the Goblins project.  I think Guile needs
> a (multi-year, multi-major version) plan to deprecate cruft and move
> the good stuff into different modules.  Give a hoot, don't pollute
> (the default environment)!

You probably know this, but in case you or others don't, you can have a
clean, explicit environment by writing a R7RS-small compatible library
via 'define-library' in a .sld file.  I think that's a good standard and
portable way to resolve the messy default environment situation, which
we should embrace more.  On a related note, I have some patches
improving R7RS support in Guile on the tracker/mailing list, such as
#71304, which allowed me to import R7RS SRFI implementations as-is.

-- 
Thanks,
Maxim