Re: Anything better for delayed lexical evaluation than (lambda () ...)?
Marco Maggi writes: > David Kastrup wrote: >> Hi, if I have something read that is evaluated later, the >> lack of procedure-environment in Guilev2 implies that I >> have to wrap the stuff in (lambda () ...) in order to >> capture the lexical environment for evaluation. > > Sorry to step in without an answer. What are you trying to > do? What I understand is that a Scheme program reads some > expressions and tries to evaluate them in a specific context > of the program. Are you looking for a way to do something > like the following chunk I found on the Net? > > (define x 0) > (define clo > (let ((x 1)) > (lambda () '( > (local-eval 'x (procedure-environment clo)) > => 1 It is more like (define (myeval what) (let* ((x 1) (clo (procedure-environment (lambda () #f (local-eval (read (open-input-string what)) clo))) (myeval "(+ x 3)") Basically a string evaluation of a string that will be captured with read-hash-extend in our application. In practice, _both_ the environment created by (let* ((x 1)) ...) as well as the string to be interpreted later are written by the user, but they are spliced together at quite different points of time since the environment from which the string for myeval gets delivered is available only when the definition is being executed, not yet at its definition time. Basically I need to evaluate dynamic code in a given lexical environment rather than at top and/or module level. For a language that is supposed to be a building block for extension languages, not really a concept that is all that unusual I would think. -- David Kastrup
Re: Anything better for delayed lexical evaluation than (lambda () ...)?
David Kastrup writes: > Marco Maggi writes: > >> David Kastrup wrote: >>> Hi, if I have something read that is evaluated later, the >>> lack of procedure-environment in Guilev2 implies that I >>> have to wrap the stuff in (lambda () ...) in order to >>> capture the lexical environment for evaluation. >> >> Sorry to step in without an answer. What are you trying to >> do? What I understand is that a Scheme program reads some >> expressions and tries to evaluate them in a specific context >> of the program. Are you looking for a way to do something >> like the following chunk I found on the Net? >> >> (define x 0) >> (define clo >> (let ((x 1)) >> (lambda () '( >> (local-eval 'x (procedure-environment clo)) >> => 1 > > It is more like > (define (myeval what) > (let* ((x 1) > (clo (procedure-environment (lambda () #f > (local-eval (read (open-input-string what)) clo))) > > (myeval "(+ x 3)") > > Basically a string evaluation of a string that will be captured with > read-hash-extend in our application. > > In practice, _both_ the environment created by (let* ((x 1)) ...) as > well as the string to be interpreted later are written by the user, but > they are spliced together at quite different points of time since the > environment from which the string for myeval gets delivered is available > only when the definition is being executed, not yet at its definition > time. > > Basically I need to evaluate dynamic code in a given lexical environment > rather than at top and/or module level. > > For a language that is supposed to be a building block for extension > languages, not really a concept that is all that unusual I would think. To come back to the original request: > Is it possible to have a shortcut (make-closure ...) or so for that > purpose? The reason is that if ... is a call to a > procedure-with-setter, (lambda () ...) actually does not cut it for > capturing the semantics of ..., and I need > (make-procedure-with-setter (lambda () ...) > (lambda (x) (set! ... x))) I now implement this more or less as (define clo #t) (define (myeval what) (let* ((x 1)) (set! clo (list (cons 'x (lambda () x (primitive-eval (read (open-input-string what) (myeval "(+ ((assq-ref clo 'x)) 3)") But of course if I want to translate something like (set! x 7) (also when x is something like (myprop k 'g) or so) with that technique, it falls down again. So in short, doing that sort of stuff by prewrapping all conceivable evaluation candidates into (lambda () ...) and doing source code location association at runtime to figure out which lambda to call is quite icky and more restricted than actually capturing an environment. See http://git.savannah.gnu.org/gitweb/?p=lilypond.git;a=blob;f=scm/parser-ly-from-scheme.scm;h=0e697d22bda657f3e970efa0281b01a0cd56360c;hb=HEAD> for the actual current source code that pushes small lambda capsules into the variable "closures" instead of just capturing a single local procedure-environment for _all_ parts of the string that is going to be parsed and interpreted at run time including small Scheme scraps. This is not hypothetical, but bonafide code running in a production environment. -- David Kastrup
Re: Anything better for delayed lexical evaluation than (lambda () ...)?
David Kastrup writes: > Basically I need to evaluate dynamic code in a given lexical environment > rather than at top and/or module level. > > For a language that is supposed to be a building block for extension > languages, not really a concept that is all that unusual I would think. Guile 2 is an excellent base for building extension languages, but not in the way that you'd like to do it. Unfortunately, I see no way to support `procedure-environment' on arbitrary procedures without abandoning our optimizing compiler and going back to a simple evaluator. I suspect it would be possible to implement a special form that captures its lexical environment in such a way that arbitrary code could later be evaluated within that lexical environment. The presence of this special form would impose onerous constraints on the optimizer within the top-level form containing it. In fact, I can't think of an optimization that would still be possible, because the compiler would have to assume the worst: that some other thread could, at any time, mutate any lexical variable or call any lexical procedure visible from the special form. It gets even worse when you consider first-class continuations. I believe that this is the wrong approach, though it may be worth considering for the sake of allowing Lilypond to continue using its existing implementation strategy. In general, the _right_ way to build a custom extension language using Guile 2 is to write a compiler that converts your language into one of the other languages that Guile 2 supports. If there's something about Lilypond's language that you believe would make compilation impractical, let's talk about it. Maybe the Guile experts on this list can find a clever solution, or else maybe we can enhance Guile to support Lilypond's language in a straightforward manner. I would be glad to help with this. In the long run, it might be less work for us Guile hackers to implement a nice compiler for Lilypond than to implement and forever maintain the "capture-lexical-environment" special form, and it would almost certainly have better results. Regards, Mark
Re: Anything better for delayed lexical evaluation than (lambda () ...)?
Mark H Weaver writes: > David Kastrup writes: >> Basically I need to evaluate dynamic code in a given lexical environment >> rather than at top and/or module level. >> >> For a language that is supposed to be a building block for extension >> languages, not really a concept that is all that unusual I would think. > > Guile 2 is an excellent base for building extension languages, but not > in the way that you'd like to do it. Unfortunately, I see no way to > support `procedure-environment' on arbitrary procedures without > abandoning our optimizing compiler and going back to a simple > evaluator. Sure. In an optimizing compiler, I would expect "procedure-environment" to only contain actually used parts of the environment, and that would make (procedure-environment (lambda () '())) fabulously useless. And of course I am not interested in the environment of that procedure, but rather in that where procedure-environment is called. > I suspect it would be possible to implement a special form that > captures its lexical environment in such a way that arbitrary code > could later be evaluated within that lexical environment. The > presence of this special form would impose onerous constraints on the > optimizer within the top-level form containing it. In fact, I can't > think of an optimization that would still be possible, because the > compiler would have to assume the worst: that some other thread could, > at any time, mutate any lexical variable or call any lexical procedure > visible from the special form. It gets even worse when you consider > first-class continuations. We are calling the Lilypond parser in that "top-level form". The optimizer is not much of a worry. > I believe that this is the wrong approach, though it may be worth > considering for the sake of allowing Lilypond to continue using its > existing implementation strategy. Uh, we are not talking about "implementation strategy" but language features. > In general, the _right_ way to build a custom extension language using > Guile 2 is to write a compiler that converts your language into one of > the other languages that Guile 2 supports. Lilypond is not Scheme. It has syntax ambiguities that are resolved by lexical tie-ins and thus depend on the context. You can't easily compile it in advance. And you are _totally_ putting the cart before the horse here. Lilypond is not supposed to be an extension language for Guile, but Guile is supposed to be an extension language for Lilypond. The acronym Guile stands for "GNU's Ubiquitous Intelligent Language for Extension". You are losing sight of what Guile is supposed to be. As an extension language, it does not make sense that it dictates the lexical and the program structure of the system it is supposed to be extending. > If there's something about Lilypond's language that you believe would > make compilation impractical, let's talk about it. Its syntax and semantics. http://git.savannah.gnu.org/cgit/lilypond.git/tree/lily/parser.yy> If I call a function with an optional argument of type integer? before an argument of type ly:music? and I encounter #x, then the value of x decides whether this argument will be used as the optional argument or as the following argument. The rest of the parsing has to follow. > Maybe the Guile experts on this list can find a clever solution, or > else maybe we can enhance Guile to support Lilypond's language in a > straightforward manner. > > I would be glad to help with this. In the long run, it might be less > work for us Guile hackers to implement a nice compiler for Lilypond than > to implement and forever maintain the "capture-lexical-environment" > special form, and it would almost certainly have better results. You are working from the premise that Guile should govern the architecture of the system it is supposed to be extending. That Lilypond is one of the few serious systems actually using Guile as an extension language does not make it a good idea to turn it into a system that Guile uses as its extension. The hard way. With lots of work. You can't expect that kind of investment to be done for every application that considers using Guile. -- David Kastrup
