Re: summary: lilypond, lambda, and local-eval

[Mark H Weaver]

I wrote:
> The  would include a list of lexical variable
> names ( ...), which must exactly correspond to the closure slots of
> the `case-lambda', in order to implement `local-eval'.  We _might_ also
> need to include some information about how those variables are stored,

Sorry, this paragraph should have ended here.  The following unfinished
caveat should have been deleted:

> e.g. a flag telling whether they are boxed.  (Usually they will all be
> boxed, but

I was thinking about an edge case where the result of (the-environment)
never escapes the local block, and thus a clever optimizer might be able
to avoid boxing some of the lexicals.  However, I later realized that
this could only happen if the returned environment was never passed to
`local-eval', in which case none of this matters anyway.

 Mark

Re: summary: lilypond, lambda, and local-eval

[Mark H Weaver]

I wrote:
> For now, I will describe a method that I suspect would do the right
> thing without any new compiler interfaces, though not as efficiently or
> robustly: Simply compile the same general-purpose dispatcher as before,
> except replace the #f (from the first case-lambda clause) with the
> expanded local expression:

Although this should result in the same set of captured lexicals, it
does not necessarily guarantee that the closure slots will be in the
same order.  This could be perhaps be solved by always sorting the
captured lexicals by name, but that would slow down the compiler.
Depending on how the compiler works, it might be sufficient to move the
 case to end of the case-lambda, but that's
definitely fragile.

So, I guess this all shows that `local-eval' really shouldn't be
implemented this way, but rather by creating a new internal interface to
the compiler that ensures that the closure slots are exactly the same as
before.

> Most passes of the compiler will pretend that (the-environment) is
> replaced by tree-il code corresponding to the following standard scheme
> code:

I should also mention that perhaps, instead of simply "pretending", it
might make sense to actually replace (the-environment) with the standard
scheme code I gave as early as possible, so that later passes will never
even see `the-environment' tree-il nodes.  It need only be late enough
so that the list of visible lexical variable names is known at that
point.

Apologies for sending multiple messages so quickly.
Obviously this is a work-in-progress :)

  Mark

Re: summary: lilypond, lambda, and local-eval

[David Kastrup]

Mark H Weaver  writes:

> I wrote:
>> For now, I will describe a method that I suspect would do the right
>> thing without any new compiler interfaces, though not as efficiently or
>> robustly: Simply compile the same general-purpose dispatcher as before,
>> except replace the #f (from the first case-lambda clause) with the
>> expanded local expression:
>
> Although this should result in the same set of captured lexicals, it
> does not necessarily guarantee that the closure slots will be in the
> same order.  This could be perhaps be solved by always sorting the
> captured lexicals by name, but that would slow down the compiler.
> Depending on how the compiler works, it might be sufficient to move the
>  case to end of the case-lambda, but that's
> definitely fragile.
>
> So, I guess this all shows that `local-eval' really shouldn't be
> implemented this way, but rather by creating a new internal interface to
> the compiler that ensures that the closure slots are exactly the same as
> before.
>
>> Most passes of the compiler will pretend that (the-environment) is
>> replaced by tree-il code corresponding to the following standard scheme
>> code:
>
> I should also mention that perhaps, instead of simply "pretending", it
> might make sense to actually replace (the-environment) with the standard
> scheme code I gave as early as possible, so that later passes will never
> even see `the-environment' tree-il nodes.  It need only be late enough
> so that the list of visible lexical variable names is known at that
> point.
>
> Apologies for sending multiple messages so quickly.
> Obviously this is a work-in-progress :)

And I consider this _very_ exciting work in progress.  One of the things
that the current development line of GCC markets is "compiler plugins".
Here GUILE has an opportunity to offer similar functionality in a
natural, Scheme-like manner with little complexity exposed to the user
of this feature, and apparently not all that much complexity needed to
get added to the compiler: it is more a matter of factoring the
complexity that has to be there anyway in a proper way.  Which actually
might make the compiler code easier to understand und modify even if you
don't end up using local-eval.

Being able to employ this in Lilypond to simplify things would certainly
be a nice side benefit, but this has the potential to simplify and
facilitate quite more complex scenarios with simple tools.

It would be _much_ _much_ simpler to use than GCC plugins.  And the
better it integrates with the compiler as a whole, the less reason would
be there _not_ to use it whenever it might be useful.

-- 
David Kastrup

[PATCH] Implement `the-environment' and `local-eval' in evaluator

[Mark H Weaver]

Here's an improved version of the preliminary evaluator-only
implementation of `the-environment' and `local-eval'.  I renamed the
primitives to the Guile 1.8 names, fixed the expansion within
`local-eval' to use `expand' instead of `expand-top-sequence', made the
module handling more robust, and various other minor improvements.

I plan to fully support these primitives in the compiler as well, in a
future version of this patch.

This is still a _preliminary_ patch.  In particular:

* The compiler currently fails ungracefully if it encounters
  (the-environment).

* The lexical environment object is currently non-opaque list structure.

* I still wouldn't be surprised if `local-eval' does the wrong thing if
  (current-module) is different from what it was when the associated
  `primitive-eval' was called.

* I manually removed the psyntax-pp.scm patch from the output of
  git-format-patch (though the header change summary still mentions it),
  since it was so huge.  I guess you'll need to manually regenerate that
  file yourself, since the Makefiles don't do it automatically:

 cd guile/module; make ice-9/psyntax-pp.scm.gen

Here's an example session:

  mhw:~/guile$ meta/guile
  GNU Guile 2.0.3.72-c6748
  Copyright (C) 1995-2011 Free Software Foundation, Inc.
  
  Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
  This program is free software, and you are welcome to redistribute it
  under certain conditions; type `,show c' for details.
  
  Enter `,help' for help.
  scheme@(guile-user)> (define env1 (primitive-eval '(let-syntax ((foo 
(syntax-rules () ((foo x) (quote x) (let ((x 1) (y 2)) (the-environment)
  scheme@(guile-user)> (local-eval 'x env1)
  $1 = 1
  scheme@(guile-user)> (local-eval 'y env1)
  $2 = 2
  scheme@(guile-user)> (local-eval '(foo (1 2)) env1)
  $3 = (1 2)
  scheme@(guile-user)> (define env2 (local-eval '(let-syntax ((bar 
(syntax-rules () ((bar x) (foo x) (let ((x 1) (z 3)) (the-environment))) 
env1))
  scheme@(guile-user)> (local-eval 'x env2)
  $4 = 1
  scheme@(guile-user)> (local-eval '(bar (1 2)) env2)
  $5 = (1 2)
  scheme@(guile-user)> (local-eval '(foo (1 2)) env2)
  $6 = (1 2)
  scheme@(guile-user)> (local-eval 'z env2)
  $7 = 3
  scheme@(guile-user)> (local-eval '(set! x (+ x 10)) env2)
  $8 = 11
  scheme@(guile-user)> (local-eval 'x env1)
  $9 = 1

  Mark


>From c6748349a833cd61b380259ca8b9d81d7f14128f Mon Sep 17 00:00:00 2001
From: Mark H Weaver 
Date: Wed, 14 Dec 2011 03:12:43 -0500
Subject: [PATCH] Implement `the-environment' and `local-eval' in evaluator

PRELIMINARY WORK, not ready for commit.
---
 libguile/expand.c   |5 +
 libguile/expand.h   |   13 +
 libguile/memoize.c  |   18 +
 libguile/memoize.h  |5 +-
 module/ice-9/eval.scm   |   31 +
 module/ice-9/psyntax-pp.scm |23299 ++-
 module/ice-9/psyntax.scm|   26 +-
 module/language/tree-il.scm |8 +
 8 files changed, 12095 insertions(+), 11310 deletions(-)

diff --git a/module/ice-9/eval.scm b/module/ice-9/eval.scm
index c0fa64c..7d6e6c1 100644
--- a/module/ice-9/eval.scm
+++ b/module/ice-9/eval.scm
@@ -213,6 +213,8 @@
 ;;; `eval' in this order, to put the most frequent cases first.
 ;;;
 
+(define local-eval #f)  ;; This is set! from within the primitive-eval block
+
 (define primitive-eval
   (let ()
 ;; We pre-generate procedures with fixed arities, up to some number of
@@ -357,6 +359,14 @@
;; Finally, eval the body.
(eval body env)))
 
+;; FIXME: make this opaque!!
+(define (make-lexical-environment module eval-env memoizer-env expander-env)
+  (list ' module eval-env memoizer-env expander-env))
+(define lexical-environment:module cadr)
+(define lexical-environment:eval-env caddr)
+(define lexical-environment:memoizer-env cadddr)
+(define (lexical-environment:expander-env env) (car (cr env)))
+
 ;; The "engine". EXP is a memoized expression.
 (define (eval exp env)
   (memoized-expression-case exp
@@ -459,6 +469,12 @@
   (eval exp env)
   (eval handler env)))
 
+(('the-environment (memoizer-env . expander-env))
+ (let ((module (capture-env (if (pair? env)
+(cdr (last-pair env))
+env
+   (make-lexical-environment module env memoizer-env expander-env)))
+
 (('call/cc proc)
  (call/cc (eval proc env)))
 
@@ -468,6 +484,21 @@
   var-or-spec
   (memoize-variable-access! exp #f))
   (eval x env)
+
+(set! local-eval
+  (lambda (exp env)
+"Evaluate @var{exp} within the lexical environment @var{env}."
+(let ((module (lexical-environment:module env))
+  (eval-env (lexical-environment:eval-env env))
+  

Re: summary: lilypond, lambda, and local-eval

[Andy Wingo]

Hi Mark,

This is an interesting plan.  I still have some doubts, but perhaps you
can make it work.

First, a couple of notes:

On Fri 16 Dec 2011 08:35, Mark H Weaver  writes:

> (quote )  ;; simple list structure
> (quote )  ;; simple list structure

Perhaps syntax-objects can give you what you need here. 

> )  ;; XXX not sure how best to represent this

The expander needs to serialize representations of modules in syntax
objects, and for that purpose it uses the module name.  "Anonymous"
modules actually have generated names, lazily.  Use `module-name'.

I like the "modeling the-environment as a scheme expression" approach;
it gives `the-environment' some semantics.  I used to ignore people who
yammered on about semantics and languages, but I have grown to respect
that approach to language development.  So thank you there.

Also, note that forms that `set!' all bound lexicals in the
`the-environment' will cause them all to be boxed, so there is no boxed
bitvector needed.

> How to implement `local-eval'
> =
>
> When `local-eval' is called on a lexical environment that was created by
> compiled code, it will do the following:
>
> * Macroexpand the local expression within .
>
> * Compile the expanded expression within .
>   (I'll explain how to do this below)
>
> * Make a copy of the closure from the lexical environment object, but
>   replace its code (the dispatcher) with the newly compiled code.
>
> * Call the newly created closure.

We are really far from considering efficiency here :)  Would you always
use the compiler for this task?  (I think I would.)  But otherwise, this
sounds sensible, with a caveat:: hygiene, nested definitions, and the
macro expander.

What does local-eval really mean?

What are the meanings of these expressions:

  ;; Toplevel
  (local-eval '(define foo 42) (the-environment))

  ;; Lexical, tail context
  (local-eval '(define foo 42) (let ((x 100)) (the-environment)))

  ;; Lexical, tail context -- but with a definition
  (local-eval '(begin (define foo 42) foo) (let ((x 100)) (the-environment)))

  ;; Lexical, tail context -- but with a definition, and nested reference
  (local-eval '(begin (define foo 42) (bar))
  (let ((x 100)) (define (bar) foo) (the-environment)))

  ;; Lexical, not a definition context
  (local-eval '(define foo 42) (let ((x 100)) not-a-definition 
(the-environment)))
  
What about this one:

  ;; Keeping in mind that `or' expands to (let ((t ...)) (if t t ...)),
  ;; hygienically
  (local-eval 't '(let ((t 42)) (or #f (the-environment 

Can you pass syntax objects into `local-eval'?

Are let-syntax / letrec-syntax / nested define-syntax forms present in a
local environment?  Currently these things never leave the expander.  I
suppose as long as they are understood to be opaque, it would be OK.

That's all I really wanted to write about local-eval, I think, so no
promised other mail.  Thanks for thinking about these things :)

Andy
-- 
http://wingolog.org/

Re: [PATCH] Implement `the-environment' and `local-eval' in evaluator

[David Kastrup]

Mark H Weaver  writes:

> Here's an improved version of the preliminary evaluator-only
> implementation of `the-environment' and `local-eval'.  I renamed the
> primitives to the Guile 1.8 names, fixed the expansion within
> `local-eval' to use `expand' instead of `expand-top-sequence', made the
> module handling more robust, and various other minor improvements.
>
> I plan to fully support these primitives in the compiler as well, in a
> future version of this patch.
>
> This is still a _preliminary_ patch.  In particular:
>
> * The compiler currently fails ungracefully if it encounters
>   (the-environment).
>
> * The lexical environment object is currently non-opaque list structure.
>
> * I still wouldn't be surprised if `local-eval' does the wrong thing if
>   (current-module) is different from what it was when the associated
>   `primitive-eval' was called.

Before anyone even _defines_ what the "right thing" would be, there is
little point in worrying about this.  I don't think that `local-eval'
1.8 documented any behavior for this case (well, it did not document any
behavior for a lot of cases).

So it probably makes sense to look afterwards what will happen without
special precautions, and unless that is spectacularly useless or
inconsistent, call it the "right thing" by definition.

-- 
David Kastrup

Re: summary: lilypond, lambda, and local-eval

[David Kastrup]

Andy Wingo  writes:

> What are the meanings of these expressions:

I found it amusing to see what my definitions using
with-current-continuation will produce here.

>   ;; Toplevel
>   (local-eval '(define foo 42) (the-environment))

guile> (my-eval '(define foo 42) (my-env))
guile> foo
42

>   ;; Lexical, tail context
>   (local-eval '(define foo 42) (let ((x 100)) (the-environment)))

guile> (my-eval '(define foo 42) (let ((x 100)) (my-env)))

Backtrace:
In standard input:
   1: 0* [my-eval (define foo 42) ...
   1: 1*  (let* ((x 100)) (# (define foo 42)))
   1: 2   (begin #)
   1: 3   [# ...
   1: 4*   (define foo 42)

standard input:1:11: In procedure memoization in expression (define foo 42):
standard input:1:11: In file "standard input", line 0: Bad define placement 
(define foo 42).
ABORT: (syntax-error)
guile> 

>   ;; Lexical, tail context -- but with a definition
>   (local-eval '(begin (define foo 42) foo) (let ((x 100)) (the-environment)))

guile> (my-eval '(begin (define foo 42) foo) (let ((x 100)) (my-env)))

Backtrace:
In standard input:
   6: 0* [my-eval (begin (define foo 42) foo) ...
   6: 1*  (let* ((x 100)) (# (begin # foo)))
   6: 2   (begin #)
   6: 3   [# ...
   6: 4*   (begin (define foo 42) foo)
   6: 5*   (define foo 42)

standard input:6:18: In procedure memoization in expression (define foo 42):
standard input:6:18: In file "standard input", line 5: Bad define placement 
(define foo 42).
ABORT: (syntax-error)
guile> 

>   ;; Lexical, tail context -- but with a definition, and nested reference
>   (local-eval '(begin (define foo 42) (bar))
>   (let ((x 100)) (define (bar) foo) (the-environment)))

guile> (my-eval '(begin (define foo 42) (bar)) (let ((x 100)) (define (bar) 
foo) (my-env)))

Backtrace:
In standard input:
  13: 0* [my-eval (begin (define foo 42) (bar)) ...
  13: 1*  (let* ((x 100)) (letrec (#) (# #)))
In unknown file:
   ?: 2   (letrec ((bar #)) (# (begin # #)))
In standard input:
...
  13: 3   [# ...
  13: 4*   (begin (define foo 42) (bar))
  13: 5*   (define foo 42)

standard input:13:18: In procedure memoization in expression (define foo 42):
standard input:13:18: In file "standard input", line 12: Bad define placement 
(define foo 42).
ABORT: (syntax-error)

>   ;; Lexical, not a definition context
>   (local-eval '(define foo 42) (let ((x 100)) not-a-definition 
> (the-environment)))

guile> (my-eval '(define foo 42) (let ((x 100)) "hello" (my-env)))

Backtrace:
In standard input:
  12: 0* [my-eval (define foo 42) ...
  12: 1*  (let* ((x 100)) "hello" (# (define foo 
42)))
  12: 2   (begin #)
  12: 3   [# ...
  12: 4*   (define foo 42)

standard input:12:11: In procedure memoization in expression (define foo 42):
standard input:12:11: In file "standard input", line 11: Bad define placement 
(define foo 42).
ABORT: (syntax-error)
guile> 

> What about this one:
>
>   ;; Keeping in mind that `or' expands to (let ((t ...)) (if t t ...)),
>   ;; hygienically
>   (local-eval 't '(let ((t 42)) (or #f (the-environment 

Assuming that the second quote mark is a typo.

guile> (my-eval 't (let ((t 42)) (or #f (my-env
42
guile> 

Now of course, the continuation based approach that just hijacks the
expander and jumps in and out of it is not really a measure of how
things should work.  But it makes clear that (the-environment) is a bit
of a chimera: it captures content at a level conceptually relevant for
(define), but returns a value and has to be placed accordingly, like in
a function call or at the providing side of a binding construct.

If those different syntactic aspects prove to be too hard to conciliate,
it might help to look at the kind of interface that some other chimeras
like call-with-values or call-with-current-continuation have taken.

-- 
David Kastrup

Re: summary: lilypond, lambda, and local-eval

[Mark H Weaver]

Hi Andy, thanks for the quick feedback.  I'll respond to the rest of
your email later, but just a few quick notes for now:

> What are the meanings of these expressions:
>
>   ;; Toplevel
>   (local-eval '(define foo 42) (the-environment))
>
>   ;; Lexical, tail context
>   (local-eval '(define foo 42) (let ((x 100)) (the-environment)))
>
>   ;; Lexical, tail context -- but with a definition
>   (local-eval '(begin (define foo 42) foo) (let ((x 100)) (the-environment)))
>
>   ;; Lexical, tail context -- but with a definition, and nested reference
>   (local-eval '(begin (define foo 42) (bar))
>   (let ((x 100)) (define (bar) foo) (the-environment)))
>
>   ;; Lexical, not a definition context
>   (local-eval '(define foo 42) (let ((x 100)) not-a-definition 
> (the-environment)))

All of these will raise errors, because my current implementation of
`local-eval' (I just posted a new version of the evaluator-only patch)
uses `expand' instead of `expand-top-sequence' to expand the local
expression, therefore definitions are not allowed.

> What about this one:
>
>   ;; Keeping in mind that `or' expands to (let ((t ...)) (if t t ...)),
>   ;; hygienically
>   (local-eval 't '(let ((t 42)) (or #f (the-environment

Yes, even my first patch handles this properly (though you must use
`primitive-eval', since the compiler currently barfs if it encounters
`the-environment'):

scheme@(guile-user)> (local-eval 't (primitive-eval '(let ((t 42)) (or #f 
(the-environment)
$1 = 42

> Can you pass syntax objects into `local-eval'?

I believe so, but I haven't verified that this works as it should.
I don't have much experience using syntax-case.

The expression passed to `local-eval' simply gets passed along to
`expand' within psyntax.scm, using the saved "expander environment":
i.e. the values of the `r', `w', and `mod' in psyntax, at the point
where (the-environment) was expanded.  This "expander environment" is
stored in the tree-il representation of (the-environment).

Here's what I currently see:

scheme@(guile-user)> (local-eval #'t (primitive-eval '(let ((t 42)) (or #f 
(the-environment)
ERROR: In procedure memoize-variable-access!:
ERROR: Unbound variable: t

This is the correct behavior, no?

> Are let-syntax / letrec-syntax / nested define-syntax forms present in a
> local environment?

Yes:

scheme@(guile-user)> (define env1 (primitive-eval '(let-syntax ((foo 
(syntax-rules () ((foo x) (quote x) (let ((x 1) (y 2)) (the-environment)
scheme@(guile-user)> (local-eval '(foo (1 2)) env1)
$3 = (1 2)
scheme@(guile-user)> (define env2 (local-eval '(let-syntax ((bar (syntax-rules 
() ((bar x) (foo x) (let ((x 1) (z 3)) (the-environment))) env1))
scheme@(guile-user)> (local-eval '(bar (1 2)) env2)
$5 = (1 2)
scheme@(guile-user)> (local-eval '(foo (1 2)) env2)
$6 = (1 2)

> Currently these things never leave the expander.  I suppose as long as
> they are understood to be opaque, it would be OK.

Agreed.  However, it should be noted that if we compile a form
containing (the-environment) into a .go file, the "expander environment"
will have to be stored within the .go file as an embedded constant.
This means that if the representations of r, w, or mod within psyntax
changes, any .go files containing uses of (the-environment) will have to
be recompiled, or else `local-eval' could fail ungracefully.

This is indeed an unpleasant complication.  Automatic recompilation
could be achieved using a versioning scheme, using a separate
expander-environment-version number.  With some care, we could perhaps
avoid needless recompilation of .go files that do not use
(the-environment) when the expander environment version changes, though
it might be a bit of a pain to do the necessary bookkeeping to determine
whether a .go file contains any expander environments.  I'll have to
think more about this.

 Mark

SXML documentation

[Ian Price]

Hi guilers,

The documentation for this in the manual is pretty sparse. Is it
possible to lift the documentation for the procedures in the source up
to the manual where people who don't read the source can find it?

As an example, the function 'find-string-from-port?' in (sxml ssax
input-parse) has the signature
  find-string-from-port? _ _ . _ 
which is not particularly meaningful. Or how would I know what the
'ssax:xml->sxml' argument 'namespace-prefix-assig' is if I didn't know
already?

-- 
Ian Price

"Programming is like pinball. The reward for doing it well is
the opportunity to do it again" - from "The Wizardy Compiled"

Re: summary: lilypond, lambda, and local-eval

[Hans Aberg]

On 16 Dec 2011, at 11:33, Mark H Weaver wrote:

> Here's what I currently see:
> 
> scheme@(guile-user)> (local-eval #'t (primitive-eval '(let ((t 42)) (or #f 
> (the-environment)
> ERROR: In procedure memoize-variable-access!:
> ERROR: Unbound variable: t
> 
> This is the correct behavior, no?

This is what I get when I play around with the following variation of David's 
code in Guile 2.0.3:
(define (xxx)
  (let* ((x 2))
(set! x (+ x 3))
(interaction-environment)))

(eval '(begin (set! x (+ x 5)) x) (xxx))

My guess (correct?) is that one wants some variation of 
(interaction-environment) that can cause x in the eval expression to bind to 
the environment returned by (xxx).

Might eval be changed to accommodate for that (without introducing the name 
local-eval)?

Hans

Re: summary: lilypond, lambda, and local-eval

[David Kastrup]

Hans Aberg  writes:

> On 16 Dec 2011, at 11:33, Mark H Weaver wrote:
>
>> Here's what I currently see:
>> 
>> scheme@(guile-user)> (local-eval #'t (primitive-eval '(let ((t 42))
>> (or #f (the-environment)
>> ERROR: In procedure memoize-variable-access!:
>> ERROR: Unbound variable: t
>> 
>> This is the correct behavior, no?
>
> This is what I get when I play around with the following variation of David's 
> code in Guile 2.0.3:
> (define (xxx)
>   (let* ((x 2))
> (set! x (+ x 3))
> (interaction-environment)))
>
> (eval '(begin (set! x (+ x 5)) x) (xxx))
>
> My guess (correct?) is that one wants some variation of
> (interaction-environment) that can cause x in the eval expression to
> bind to the environment returned by (xxx).
>
> Might eval be changed to accommodate for that (without introducing the
> name local-eval)?

It would likely help with unasking the question of what to do when
(current-module) is different at the time of local-eval.  I don't know,
however, what the _lexical_ effects of switching the current module are
supposed to be.  If it is supposed to be a noop, then lexical
environments and modules are presumably orthogonal, and eval should
likely be allowed to take both (currently, local-eval is like taking a
lexical environment and using primitive-eval in it).

-- 
David Kastrup

Re: [PATCH] Implement `the-environment' and `local-eval' in evaluator

[Peter TB Brett]

David Kastrup  writes:

>> * I still wouldn't be surprised if `local-eval' does the wrong thing if
>>   (current-module) is different from what it was when the associated
>>   `primitive-eval' was called.
>
> Before anyone even _defines_ what the "right thing" would be, there is
> little point in worrying about this.  I don't think that `local-eval'
> 1.8 documented any behavior for this case (well, it did not document any
> behavior for a lot of cases).
>
> So it probably makes sense to look afterwards what will happen without
> special precautions, and unless that is spectacularly useless or
> inconsistent, call it the "right thing" by definition.

Maybe it makes even more sense (at this stage) to state that the
behaviour in this case is undefined?

   Peter

-- 
Peter Brett 
Remote Sensing Research Group
Surrey Space Centre

Re: [PATCH] Implement `the-environment' and `local-eval' in evaluator

[David Kastrup]

Peter TB Brett  writes:

> David Kastrup  writes:
>
>>> * I still wouldn't be surprised if `local-eval' does the wrong thing if
>>>   (current-module) is different from what it was when the associated
>>>   `primitive-eval' was called.
>>
>> Before anyone even _defines_ what the "right thing" would be, there is
>> little point in worrying about this.  I don't think that `local-eval'
>> 1.8 documented any behavior for this case (well, it did not document any
>> behavior for a lot of cases).
>>
>> So it probably makes sense to look afterwards what will happen without
>> special precautions, and unless that is spectacularly useless or
>> inconsistent, call it the "right thing" by definition.
>
> Maybe it makes even more sense (at this stage) to state that the
> behaviour in this case is undefined?

In my opinion it does not make sense at this stage to state anything.
Declaring the behavior as being undefined is premature when further work
might discover that it makes eminent sense to define it in a particular
way.

-- 
David Kastrup

Re: SXML documentation

[Andy Wingo]

On Fri 16 Dec 2011 11:52, Ian Price  writes:

> Hi guilers,
>
> The documentation for this in the manual is pretty sparse. Is it
> possible to lift the documentation for the procedures in the source up
> to the manual where people who don't read the source can find it?
>
> As an example, the function 'find-string-from-port?' in (sxml ssax
> input-parse) has the signature
>   find-string-from-port? _ _ . _ 
> which is not particularly meaningful. Or how would I know what the
> 'ssax:xml->sxml' argument 'namespace-prefix-assig' is if I didn't know
> already?

It probably needs to be rewritten.  It was part of an experiment to do
literate documentation, and it's pretty illiterate in the end.

Any patches to improve this situation are most welcome.

Andy
-- 
http://wingolog.org/

Re: summary: lilypond, lambda, and local-eval

[Hans Aberg]

On 16 Dec 2011, at 13:43, David Kastrup wrote:

>>> Here's what I currently see:
>>> 
>>> scheme@(guile-user)> (local-eval #'t (primitive-eval '(let ((t 42))
>>> (or #f (the-environment)
>>> ERROR: In procedure memoize-variable-access!:
>>> ERROR: Unbound variable: t
>>> 
>>> This is the correct behavior, no?
>> 
>> This is what I get when I play around with the following variation of 
>> David's code in Guile 2.0.3:
>> (define (xxx)
>>  (let* ((x 2))
>>(set! x (+ x 3))
>>(interaction-environment)))
>> 
>> (eval '(begin (set! x (+ x 5)) x) (xxx))
>> 
>> My guess (correct?) is that one wants some variation of
>> (interaction-environment) that can cause x in the eval expression to
>> bind to the environment returned by (xxx).
>> 
>> Might eval be changed to accommodate for that (without introducing the
>> name local-eval)?
> 
> It would likely help with unasking the question of what to do when
> (current-module) is different at the time of local-eval.  I don't know,
> however, what the _lexical_ effects of switching the current module are
> supposed to be.  If it is supposed to be a noop, then lexical
> environments and modules are presumably orthogonal, and eval should
> likely be allowed to take both (currently, local-eval is like taking a
> lexical environment and using primitive-eval in it).

Would you not it work as though you inserted code in the place where then 
environment? - Then the syntactical rules should be captured as well.

In addition, there should be a way to communicate with the surrounding 
environment, wherefrom the code is inserted. The only truly safe way would be 
to make that explicit somehow, if not merely returning a value suffices.

Hans

Re: [PATCH] Implement `the-environment' and `local-eval' in evaluator

[Mark H Weaver]

Peter TB Brett  writes:

> David Kastrup  writes:
>
>>> * I still wouldn't be surprised if `local-eval' does the wrong thing if
>>>   (current-module) is different from what it was when the associated
>>>   `primitive-eval' was called.
>>
>> Before anyone even _defines_ what the "right thing" would be, there is
>> little point in worrying about this.  I don't think that `local-eval'
>> 1.8 documented any behavior for this case (well, it did not document any
>> behavior for a lot of cases).
>>
>> So it probably makes sense to look afterwards what will happen without
>> special precautions, and unless that is spectacularly useless or
>> inconsistent, call it the "right thing" by definition.
>
> Maybe it makes even more sense (at this stage) to state that the
> behaviour in this case is undefined?

To my mind, top-level (module) variables are conceptually part of every
lexical environment placed within that module.  For example, code like
this:

(define (factorial x)
  (if (zero? x) 1
  (* x (factorial (- x 1)

should act as one would naturally expect (assuming that factorial is not
later set!), whether it is placed within a local block or placed at the
top-level of some module.

It would be crazy for any lexical environment "search path" starting
from within factorial's definition block to lead to a different module
than the one where `factorial' was defined.

In other words, in the following example, any variable lookup that would
lead to `foo2' should also lead to the `foo1' (assuming it's not
shadowed by a lexical binding).

(define foo1 #f)
(let ((foo2 #f))
   )

As an interesting case, suppose that you define the following macro in
module A:

(define foo 'module-a)
(define-syntax alt-environment
  (syntax-rules ()
((_) (the-environment

and then evaluate the following within module B:

(define foo 'module-b)
(local-eval 'foo (alt-environment))

What should the result be?

My guess is that it should return 'module-a, because I think
conceptually it should act as though the local-expression passed to
`local-eval' were put in place of (the-environment), wherever that
might be.

Thoughts?

   Mark

Re: [PATCH] Implement `the-environment' and `local-eval' in evaluator

[Andy Wingo]

On Fri 16 Dec 2011 16:27, Mark H Weaver  writes:

> To my mind, top-level (module) variables are conceptually part of every
> lexical environment placed within that module.

Agreed.

> (define foo 'module-a)
> (define-syntax alt-environment
>   (syntax-rules ()
> ((_) (the-environment

> and then evaluate the following within module B:

> (define foo 'module-b)
> (local-eval 'foo (alt-environment))

> What should the result be?

> My guess is that it should return 'module-a, because I think
> conceptually it should act as though the local-expression passed to
> `local-eval' were put in place of (the-environment), wherever that

Dunno, I could make an argument either way :)  Another question is how
would local environments relate to procedural macros.

Andy
-- 
http://wingolog.org/

Re: [PATCH] Implement `the-environment' and `local-eval' in evaluator

[Hans Aberg]

On 16 Dec 2011, at 16:27, Mark H Weaver wrote:

> As an interesting case, suppose that you define the following macro in
> module A:
> 
> (define foo 'module-a)
> (define-syntax alt-environment
>  (syntax-rules ()
>((_) (the-environment
> 
> and then evaluate the following within module B:
> 
> (define foo 'module-b)
> (local-eval 'foo (alt-environment))
> 
> What should the result be?
> 
> My guess is that it should return 'module-a, because I think
> conceptually it should act as though the local-expression passed to
> `local-eval' were put in place of (the-environment), wherever that
> might be.
> 
> Thoughts?

I thought it should be a way to capture the environment when (the-environment) 
is evaluated, returning a reference to it. So
  (define foo 'module-a)
  (define bar (the-environment))  ; Capture environment, and save reference in 
bar.
Now in
  (define foo 'module-b)
  (local-eval 'foo bar)
bar refers to the captured environment and 'foo is inserted into that; that is 
'module-a.

It would need to capture all dynamic syntactic rules as well, otherwise the 
code cannot be run safely.

Hans

Re: [PATCH] Implement `the-environment' and `local-eval' in evaluator

[Mark H Weaver]

Andy Wingo  writes:
>> (define foo 'module-a)
>> (define-syntax alt-environment
>>   (syntax-rules ()
>> ((_) (the-environment
>
>> and then evaluate the following within module B:
>
>> (define foo 'module-b)
>> (local-eval 'foo (alt-environment))
>
>> What should the result be?
>
>> My guess is that it should return 'module-a, because I think
>> conceptually it should act as though the local-expression passed to
>> `local-eval' were put in place of (the-environment), wherever that
>
> Dunno, I could make an argument either way :)

Having thought more about this, I'm fully convinced that 'module-a
should be the answer.

The real reason is that we need to macroexpand the new local expression
within the expander-environment captured by (the-environment), and this
expander-environment really determines the lexical environment that
symbols are looked up in, in the same way that free variable references
from a macro definition are resolved in the lexical environment of the
macro definition instead of the place where the macro is used.

If we don't use the expander-environment captured by (the-environment),
what other expander-environment would we use, and how would we capture
this other environment?

I guess the other option would be that, if (the-environment) is found
within a macro definition, then we should instead use the expander
environment from where the currently-expanding macro was _used_.  But
what if that use is itself within another macro definition?  I don't
think it makes sense to just go up one level, that seems very inelegant.
No, it should be one extreme or the other, so the other logical choice
would be to go all the way up the macro expansion stack to the top,
where the use of the _initial_ macro was found.

But regardless of whether we go up one level or all the way to the top,
this other option would cause serious problems.  In the general case,
(the-environment) could be nested quite deeply within `let's,
`let-syntax's, and worse of all, `lambda's!

For example, consider the following example:

(define-syntax blah
  (syntax-rules ()
((_) (lambda ()
   (let-syntax (...)
 (let ()
   
   (the-environment)))

If we were to use this crazy alternate rule, then `local-eval' should
evaluate its expression in the environment where the above macro was
used, i.e. where the procedure was _defined_.  I hope we can all agree
that this would be madness.

No, the only sane option is to do the straightforward thing: we use the
expander environment captured by (the-environment), even if that's found
within a macro.  And that also means that the module used by
`local-eval' should be the module found within the expander environment,
i.e. the module where the macro containing (the-environment) was
_defined_.

Does this make sense?

  Best,
   Mark

Re: [PATCH] Implement `the-environment' and `local-eval' in evaluator

[Mark H Weaver]

I wrote:
> Having thought more about this, I'm fully convinced that 'module-a
> should be the answer.

On second thought, I don't think my argument was very solid.  My nephew
demanded my attention while I was working on that email, and so I rushed
it out when I should have put it aside and pondered some more.

So, in cases where (the-environment) is found within a macro template,
I guess I'm still undecided about whether the captured expander
environment should be at the point of (the-environment), or whether it
should be at the point of the initial macro use that led to that
expansion.

However, I'm fairly sure that those are the only two sane options, and I
feel quite certain that the module used by `local-eval' should be taken
from the captured expander environment, whatever that may be.

Do people agree with that those are the two sane options?
Any thoughts on which one is more useful?

I guess we should try to come up with examples of useful macros that use
(the-environment), and hope that these examples point to a clear winner.

For that matter, in the unlikely case that both options seem genuinely
useful, it wouldn't be hard to implement both, since they would both
translate to the same `tree-il' type.  The only difference would be in
the expander, since it puts the  into the tree-il
node, and that piece of code is only a few lines.

Thoughts?

 Best,
  Mark

Re: "exec" procedures don't handle the optional parameters unbounded situation?

[Ludovic Courtès]

Hello!

Nala Ginrut  skribis:

> Since Guile provides flexible optional arguments handling, why we can not
> do this:
> let (execlp "ls")  equal to (execlp "ls" "") if the second argument is
> unbounded?

Because it would differ from what the underlying system call allows, and
would also be an incompatible change.

Thanks,
Ludo’.

Re: "exec" procedures don't handle the optional parameters unbounded situation?

[David Kastrup]

l...@gnu.org (Ludovic Courtès) writes:

> Hello!
>
> Nala Ginrut  skribis:
>
>> Since Guile provides flexible optional arguments handling, why we can not
>> do this:
>> let (execlp "ls")  equal to (execlp "ls" "") if the second argument is
>> unbounded?
>
> Because it would differ from what the underlying system call allows, and
> would also be an incompatible change.

Incompatible?  You really think there are programs relying on this being
an error?  I agree that a system call wrapper should not be too clever,
but compatibility is hardly a concern here.

-- 
David Kastrup