Mark, thank you very much for explaining at lenght, I appreciate that !
2018-05-30 3:07 GMT+02:00 Mark H Weaver <m...@netris.org>: > Hi, > > Catonano <caton...@gmail.com> writes: > > > 2018-05-29 17:01 GMT+02:00 Mark H Weaver <m...@netris.org>: > > > what's the problem with macroexpand-1 and syntax-case ? > > > > In Guile 1.x, 'macroexpand-1' performed a single macro expansion step at > > the top-level form of an expression, using its old non-hygienic macro > > expander. There are several problems with trying to provide such an > > interface in a Hygienic macro expander, and especially in the > > 'syntax-case' expander with its support for 'datum->syntax'. For one > > thing, our modern macro expander doesn't even work with the plain > > S-expressions which 'macroexpand-1' accepted and produced. It works > > with "syntax objects", which effectively annotate every identifier with > > extra information needed to determine which binding it references, and > > also extra information needed to implement 'datum->syntax'. This in > > turn requires detailed knowledge of the lexical environment in which > > expansion is taking place, whereas 'macroexpand-1' provides no way for > > the user to provide this information. > > > > Mark > > > > I have been reading this document about the scheme higienic macros > > https://www.cs.indiana.edu/~dyb/pubs/bc-syntax-case.pdf > > > > I stopped reading it when I read that the implementation relies on a > > previously bootstrapped version of another macro expansion > > implementation. > > That's not an inherent limitation of the 'syntax-case' design. It's > merely an unfortunate attribute of the psyntax _implementation_ of > 'syntax-case', apparently because they didn't care enough about > bootstrapping issues to write psyntax without the benefit of macros. > > 'syntax-case' could certainly be implemented without using a > pre-existing macro expander. > > > But Racket has some facilities to step and debug macros, as you can > > see here https://docs.racket-lang.org/macro-debugger/index.html > > > > Aren' t Racket macros higienyc ? > > Yes, of course, and we could certainly implement similar macro stepping > facilities in Guile. But that's not what you asked about in your > previous message. You asked about 'macroexpand-1', and my answer was > specifically about that. I don't see any procedure similar to > 'macroexpand-1' in the document you referenced above. > My bad I assumed that macroexpand-1 was the building block for Racket macro stepping and inspecting tools I' m interested in macro stepping and inspecting facilities, not in macroexpand-1 per se > > In this question I've been promptly suggested a quick solution to > > perform a single macro expansion step > > > > https://stackoverflow.com/questions/50073207/macro- > expansion-in-guile-scheme/50515880#50515880 > > For posterity, here's the quick solution suggested in the link above: > > (define-syntax (expand1 stx) > (syntax-case stx () > [(_expand1 form) > (syntax-case #'form () > [(id . more) > (identifier? #'id) > (let ([transformer (syntax-local-value #'id)]) > (with-syntax ([expansion (transformer #'form)]) > #''expansion))] > [_ > #''form])])) > > This is just a toy, and not very useful in practice. > Here's the equivalent formulation for Guile: > > (use-modules (system syntax) > (srfi srfi-11)) > > (define (syntax-local-value id) > (let-values (((type value) (syntax-local-binding id))) > value)) > > (define-syntax expand1 > (lambda (stx) > (syntax-case stx () > [(_expand1 form) > (syntax-case #'form () > [(id . more) > (identifier? #'id) > (let ([transformer (syntax-local-value #'id)]) > (with-syntax ([expansion (transformer #'form)]) > #''expansion))] > [_ > #''form])]))) > > (I usually prefer to avoid using square brackets in this way, but for > sake of comparison, I used them in the definition of 'expand1' above.) > > Anyway, it works the same way as in Racket for this simple example: > > scheme@(guile-user)> (expand1 (or 1 2 3)) > $2 = (let ((t 1)) (if t t (or 2 3))) > > This is surprising to me When I saw that example made in Racket for the first time I instantly identified "syntax-local-value" as problematic Will Guile have anything equivalent ? I asked myself Now you show me the "(system syntax)" namespace (or module) I didn't suspect it existed Does the manual mention it anywhere ? I didn' t see it Or maybe does it belong to any scheme standard ? Do any more (system ....) namespaces exist ? How would I know ? > So, what's the problem? The first problem is that when quoting the > resulting expansion, the binding information associated with identifiers > in the syntax objects are lost, so hygiene is lost. For example: > > scheme@(guile-user)> (expand1 (or 1 2 t)) > $3 = (let ((t 1)) (if t t (or 2 t))) > > Moving on, let's use this to try to investigate how 'define-record-type' > works from SRFI-9 in Guile: > > scheme@(guile-user)> ,use (srfi srfi-9) > scheme@(guile-user)> (expand1 (define-record-type <box> > (box value) > box? > (value unbox set-box!))) > > $4 = (%define-record-type #f (define-record-type <box> (box value) box? > (value unbox set-box!)) <box> (box value) box? (value unbox set-box!)) > > scheme@(guile-user)> (expand1 (%define-record-type #f > (define-record-type <box> (box value) box? (value unbox set-box!)) <box> > (box value) box? (value unbox set-box!))) > While compiling expression: > Wrong type to apply: (%define-record-type guile-user) > scheme@(guile-user)> > > So what went wrong here? The problem is that '%define-record-type' is a > private macro, used internally within (srfi srfi-9), and therefore not > bound in the (guile-user) module where I'm working. If we had been > working with syntax objects, each identifier within the expression would > have been annotated with the specific binding that it refers to, but as > I noted above, that information has been stripped. > > The awkward error message is because this toy implementation doesn't > check if the identifier is a macro or not. > > One way we could try to improve this is to write 'expandN', which > performs N macro expansion steps, keeping them as syntax objects during > the intermediate steps: > > (use-modules (system syntax) > (srfi srfi-11)) > > (define (syntax-local-type id) > (let-values (((type value) (syntax-local-binding id))) > type)) > > (define (syntax-local-value id) > (let-values (((type value) (syntax-local-binding id))) > value)) > > (define-syntax expandN > (lambda (stx) > (syntax-case stx () > ((_expandN n form) > (let ((n (syntax->datum #'n))) > (and (number? n) (integer? n))) > (let ((n (syntax->datum #'n))) > (if (positive? n) > (syntax-case #'form () > ((id . _) > (and (identifier? #'id) > (eq? 'macro (syntax-local-type #'id))) > (let ((transformer (syntax-local-value #'id))) > (with-syntax ((expansion (transformer #'form)) > (n-1 (datum->syntax #'id (- n 1)))) > #'(expandN n-1 expansion)))) > (_ > #''form)) > #''form)))))) > > Unfortunately, this is not quite right, because it fails to add "marks" > to the identifiers introduced by the macro transformers, and thus is not > fully hygienic, and variable capture may occur. However, it is better > than what we had before, and good enough to step further into > 'define-record-type': > > --8<---------------cut here---------------start------------->8--- > scheme@(guile-user)> ,pp (expandN 0 (define-record-type <box> > (box value) > box? > (value unbox set-box!))) > $2 = (define-record-type > <box> > (box value) > box? > (value unbox set-box!)) > scheme@(guile-user)> ,pp (expandN 1 (define-record-type <box> > (box value) > box? > (value unbox set-box!))) > $3 = (%define-record-type > #f > (define-record-type > <box> > (box value) > box? > (value unbox set-box!)) > <box> > (box value) > box? > (value unbox set-box!)) > scheme@(guile-user)> ,pp (expandN 2 (define-record-type <box> > (box value) > box? > (value unbox set-box!))) > $4 = (begin > (define-inlinable > (box value) > (let ((s (allocate-struct <box> 1))) > (struct-set! s 0 value) > s)) > (define <box> > (let ((rtd (make-struct/no-tail > record-type-vtable > 'pw > default-record-printer > '<box> > '(value)))) > (set-struct-vtable-name! rtd '<box>) > (struct-set! rtd (+ 2 vtable-offset-user) box) > rtd)) > (define-inlinable > (box? obj) > (and (struct? obj) > (eq? (struct-vtable obj) <box>))) > (define-tagged-inlinable > ((%%type <box>) > (%%index 0) > (%%copier %%<box>-set-fields)) > (unbox s) > (if (eq? (struct-vtable s) <box>) > (struct-ref s 0) > (throw-bad-struct s 'unbox))) > (define-syntax-rule > (%%<box>-set-fields check? s (getter expr) ...) > (%%set-fields > <box> > (unbox) > check? > s > (getter expr) > ...)) > (define-inlinable > (set-box! s val) > (if (eq? (struct-vtable s) <box>) > (struct-set! s 0 val) > (throw-bad-struct s 'set-box!)))) > scheme@(guile-user)> > --8<---------------cut here---------------end--------------->8--- > > Unfortunately this is as far as we can go with 'expandN', because it > only expands macros at the top-level of the expression. In this case, > the top-level expression is a 'begin' form, which is a core form. At > this point, a real macro expander descends into the core form and > expands subexpressions, but in order to do this properly, it needs to > understand the meanings of the core forms that it's descending into. > > For example, when descending into a 'let' form, it needs to take note of > the variables that are bound by the 'let'. For example: > > scheme@(guile-user)> (expandN 0 (or 1 2 3)) > $2 = (or 1 2 3) > scheme@(guile-user)> (expandN 1 (or 1 2 3)) > $3 = (let ((t 1)) (if t t (or 2 3))) > scheme@(guile-user)> (expandN 2 (or 1 2 3)) > $4 = (let ((t 1)) (if t t (or 2 3))) > > The last two outputs are the same because I made 'expandN' just smart > enough to notice that 'let' is not a macro, in which case it stops > gracefully without triggering an exception. > > Hopefully this illustrates why the old 'macroexpand-1' procedure from > Guile 1.x, which works on plain S-expressions without extra binding > information, and which only expands macros at the top level of the > expression, cannot be usefully implemented on a modern hygienic macro > expander. > > However, what certainly *could* be done is some kind of interactive tool > to incrementally step macro expansions, while keeping track of the > syntax objects behind the scenes. To be useful in realistic cases, it > would need to understand most if not all of the core forms in Guile. > Those core forms are the ones defined using 'global-extend' in > psyntax.scm. > > Regards, > Mark > I had read about those bindings and marks in the paper but I had no clear idea about what they were Now I have a way better idea Also, now I understand what would be required in order to implement some macro stepping and inspecting facilities similar to those available in Racket I don' t know if I will actually try to implement anything, but I will certainly use these notions to my advantage in working with Guile/Guix So thanks
