Re: [ANN] nyacc 1.01.2 released
On Fri, Mar 13, 2020 at 03:29:29PM -0700, Matt Wette wrote: [...] > NEWS for V1.01.2 Thanks! Compiling now... Cheers -- t signature.asc Description: Digital signature
Re: Nyacc question: [found] where are the actions bound?
On Sun, Mar 08, 2020 at 08:10:50AM -0700, Matt Wette wrote: > > > On 3/8/20 3:14 AM, to...@tuxteam.de wrote: > >Hi, [...] > >My question is: where is the stuff resolved which is mentioned > >in grammar actions? Hah. Managed to answer my own question by reading the source. For the benefits of others who might have a similar question (without knowing how to articulate it, like it happened to me), here's the answer: Yes, the actions are "resolved" wrt the (calling) module's top level environment. The magic happens here (that's wrt the all-fresh V1.01.2), around lines 47 ff, in module/nyacc/parse.scm: (define (make-xct av) (if (procedure? (vector-ref av 0)) av (vector-map (lambda (ix f) (eval f (current-module))) (vector-map (lambda (ix actn) (wrap-action actn)) av It's the (eval f (current-module)) which does the trick. The trick happens in make-lalr-parser. Background: I envisioned something like (let ( ...some environment for the actions...) ...make-lalr-parser...) and have make-lalr-parser pick up the bindings in the lexical environment. but had to realize that make-lalr-parser ignores the lexical environment. That's now clear to me, because (eval ... (current-module)) looks at the caller's module's top-level bindings. One would have to call local-eval (from (ice-9 local-eval)) and explicitly pass (the-environment) (from the same module) to achieve what I had in mind. Not that I'm proposing that, mind you. I still barely know what I'm doing at this point. Probably there are very good reasons to resort to the (top-level) module bindings. I just wanted to understand, and I think I do now :-) As to your other proposal (writing out the pre-compiled parser with write-lalr-*), I think it's orthogonal to my issue. This might come in handy when the parser is heavy enough that it takes a significant time constructing it (which it is not in my little use case for now) Thanks & cheers -- tomás signature.asc Description: Digital signature
Re: [ANN] nyacc 1.01.2 released
On Sat, Mar 14, 2020 at 10:34:28AM +0100, tomas wrote: > On Fri, Mar 13, 2020 at 03:29:29PM -0700, Matt Wette wrote: > > [...] > > > NEWS for V1.01.2 > > Thanks! Compiling now... works, thank you a lot :-) Cheers -- tomás signature.asc Description: Digital signature
Incomplete backtrace
I think there are others here better qualified to answer your question, but maybe one helpful thing: have you read the Guile Implementation section of the Guile Reference Manual? In subsection "A Virtual Machine for Guile" there is this paragraph: ''' Note that once a value in a local variable slot is no longer needed, Guile is free to re-use that slot. This applies to the slots that were initially used for the callee and arguments, too. For this reason, backtraces in Guile aren’t always able to show all of the arguments: it could be that the slot corresponding to that argument was re-used by some other variable. ''' I do not know if there is a way to disable slot reuse, for debugging purposes. Anyone...? -- Christopher Howard p: +1 (907) 374-0257 w: https://librehacker.com social: https://gnusocial.club/librehacker gpg: ADDEAADE5D607C8D (keys.gnupg.net) > > Why do I not see the exact place where the problem is? Why are there > underscores instead? Why do I not even see that the error originated > in > test.scm? I can't find any explanation about this in the manual. > > While this code was extracted from a bigger program and it's obvious > where > the problem is, it was super hard to figure it out in the original > program. > Because the backtrace actually doesn't even point in the right place, > it's > quite useless in my opinion. > > Am I doing anything wrong? I use guile 3.0.1. > > Regards, > -- > Jan Synacek > Software Engineer, Red Hat > >
Re: Nyacc question: [found] where are the actions bound?
On 3/14/20 4:59 AM, to...@tuxteam.de wrote: On Sun, Mar 08, 2020 at 08:10:50AM -0700, Matt Wette wrote: On 3/8/20 3:14 AM, to...@tuxteam.de wrote: Hi, [...] My question is: where is the stuff resolved which is mentioned in grammar actions? Hah. Managed to answer my own question by reading the source. For the benefits of others who might have a similar question (without knowing how to articulate it, like it happened to me), here's the answer: Yes, the actions are "resolved" wrt the (calling) module's top level environment. The magic happens here (that's wrt the all-fresh V1.01.2), around lines 47 ff, in module/nyacc/parse.scm: (define (make-xct av) (if (procedure? (vector-ref av 0)) av (vector-map (lambda (ix f) (eval f (current-module))) (vector-map (lambda (ix actn) (wrap-action actn)) av It's the (eval f (current-module)) which does the trick. The trick happens in make-lalr-parser. Background: I envisioned something like (let ( ...some environment for the actions...) ...make-lalr-parser...) and have make-lalr-parser pick up the bindings in the lexical environment. but had to realize that make-lalr-parser ignores the lexical environment. That's now clear to me, because (eval ... (current-module)) looks at the caller's module's top-level bindings. One would have to call local-eval (from (ice-9 local-eval)) and explicitly pass (the-environment) (from the same module) to achieve what I had in mind. Not that I'm proposing that, mind you. I still barely know what I'm doing at this point. Probably there are very good reasons to resort to the (top-level) module bindings. I just wanted to understand, and I think I do now :-) As to your other proposal (writing out the pre-compiled parser with write-lalr-*), I think it's orthogonal to my issue. This might come in handy when the parser is heavy enough that it takes a significant time constructing it (which it is not in my little use case for now) Thanks & cheers -- tomás I get it now. What you expect makes much sense. I will think about that. Matt
Re: Incomplete backtrace
On 3/14/20 7:19 AM, Christopher Howard wrote: I think there are others here better qualified to answer your question, but maybe one helpful thing: have you read the Guile Implementation section of the Guile Reference Manual? In subsection "A Virtual Machine for Guile" there is this paragraph: ''' Note that once a value in a local variable slot is no longer needed, Guile is free to re-use that slot. This applies to the slots that were initially used for the callee and arguments, too. For this reason, backtraces in Guile aren’t always able to show all of the arguments: it could be that the slot corresponding to that argument was re-used by some other variable. ''' I do not know if there is a way to disable slot reuse, for debugging purposes. Anyone...? I started looking into this a while back but gave up for now. What I tried was 1) Add a debug flag "-g" to guild compile. In interactive mode I think it's possible to set the optimization flags, this would be '(debug . #t) 2) In the cps conversion add a hook to un-reuse slots. It didn't work. I think the return value(s) need to be in the first slots. Back to the drawing board. Below is the patch. Below is a patch reflecting the changes I made to guile-2.2.4 to try: --- module/scripts/compile.scm-orig 2018-08-07 03:34:55.0 -0700 +++ module/scripts/compile.scm 2019-06-01 16:47:37.586223675 -0700 @@ -84,6 +84,12 @@ (cons (string->symbol arg) warnings) (alist-delete 'warnings result)) + (option '(#\g "debug") #f #f + (lambda (opt name arg result) + (alist-cons 'optimizations + (cons* #:debug #t (optimizations-for-level 0)) + result))) + (option '(#\O "optimize") #t #f (lambda (opt name arg result) (define (return val) --- module/language/cps/compile-bytecode.scm-orig 2018-10-03 13:55:11.0 -0700 +++ module/language/cps/compile-bytecode.scm 2019-06-02 11:49:31.812374598 -0700 @@ -41,6 +41,12 @@ #:use-module (system base types internal) #:export (compile-bytecode)) +(define-public cps-debug-1 #f) +(define-public cps-debug-2 #f) +(define-public cps-debug-3 #f) +(define-public cps-debug-4 #f) +(define-public cps-debug-5 #f) + (define (kw-arg-ref args kw default) (match (memq kw args) ((_ val . _) val) @@ -84,6 +90,9 @@ (define (compile-function cps asm opts) (let* ((allocation (allocate-slots cps #:precolor-calls? (kw-arg-ref opts #:precolor-calls? #t))) + (allocation (if (kw-arg-ref opts #:debug #f) + (expand-slots allocation cps) + allocation)) (forwarding-labels (compute-forwarding-labels cps allocation)) (frame-size (lookup-nlocals allocation))) (define (forward-label k) @@ -655,6 +664,8 @@ (emit-end-arity asm) (emit-end-program asm + (set! cps-debug-1 cps) + (set! cps-debug-2 allocation) (intmap-for-each compile-cont cps))) (define (emit-bytecode exp env opts) --- module/language/cps/slot-allocation.scm-orig 2019-06-01 16:47:37.586223675 -0700 +++ module/language/cps/slot-allocation.scm 2019-06-02 19:46:49.473749251 -0700 @@ -998,3 +998,35 @@ (shuffles (compute-shuffles cps slots calls live-in)) (frame-size (compute-frame-size cps slots calls shuffles))) (make-allocation slots representations calls shuffles frame-size)) + +;;(use-modules (ice-9 pretty-print)) +;;(define (pp exp) (pretty-print exp #:per-line-prefix " ")) +(define (find-return-slots cps) + (let* ((kt (match (intmap-ref cps 0) (($ $kfun _ m s t c) t + (intmap-fold + (lambda (ix iv kv) + (match iv + (($ $kargs _ _ ($ $continue kx _ ($ $values vals))) + (if (= kx kt) vals kv)) + (_ kv))) + cps #f))) + +(define (expand-slots allocation cps) + (display "expanding slots\n") + (let* ((rs (find-return-slots cps)) + (nr (length rs)) + (rm (map cons rs (iota nr + (match allocation + (($ $allocation slots representations call-allocs shuffles frame-size) + (call-with-values + (lambda () + (intmap-fold + (lambda (ix iv im n) + (if (assq-ref rm ix) + (values (intmap-add im ix (assq-ref rm ix)) n) + (values (intmap-add im ix n) (1+ n + slots empty-intmap nr)) + (lambda (xslots xframe-size) + (make-allocation xslots representations call-allocs shuffles + xframe-size))) +(export expand-slots)
Re: Nyacc question: [found] where are the actions bound?
On Sat, Mar 14, 2020 at 07:31:10AM -0700, Matt Wette wrote: > On 3/14/20 4:59 AM, to...@tuxteam.de wrote: [...] > >and have make-lalr-parser pick up the bindings in the lexical > >environment. [...] > I get it now. What you expect makes much sense. > I will think about that. It might come at a price, I don't know. At least, the calling site has to capture the (lexical) environment, so make-lalr-parser has either to get the env as an extra parameter or it has to become a macro. As I wrote in the other mail, a doc fix could be more than enough. I repeat: I barely know what I'm doing, so I might be dangerous ;-) Cheers & thanks -- tomás signature.asc Description: Digital signature
figuring out behaviour of peg
Hi guile-user,
so i currently am trying to get further with the aprsing of Android
blueprints. Arun, gave the extremely helpful tip to use `pre-post-order'
for this, it seems to be a good way.
I just noticed a few things which i find odd about my grammar and i
don't see where it is coming from. (In the following example i'm using
the Android.bp file from libcutils of Android-10.0.0_r25). I'll show it
for the `list' non-terminal, but is is the same for the `map'
non-terminal (i guess because they have the same structure).
(defvar
(var "libcutils_nonwindows_sources")
(value (list (value (string "fs.cpp"))
-->((value (string "hashmap.cpp"))
(value (string "multiuser.cpp"))
(value (string "socket_inaddr_any_server_unix.cpp"))
(value (string "socket_local_client_unix.cpp"))
(value (string "socket_local_server_unix.cpp"))
(value (string "socket_network_client_unix.cpp"))
(value (string "sockets_unix.cpp"))
(value (string "str_parms.cpp")))<-- )))
I highlighted two parens in this snippet, the question is: Why do they
even exist? - this *should* not be.
Multiple consecutive module definitions also are parsed into a
list. This *should* also not be the case. So is there a way to avoid
this? I guess it has something to do with ()* clauses in the grammar
definition?
Thanks in advance for any help!
Best Regards
Malte
Here are my test files, if you want to try it :)
(use-modules (srfi srfi-1)
(ice-9 peg)
(ice-9 pretty-print)
(ice-9 rdelim)
(sxml transform))
(define *bp*
(read-delimited "" (open-input-file (cadr (command-line))) 'concat))
(define-peg-string-patterns
"blueprint <-- (comment* (ws? defvar ws?)* module* comment*)* .*
module <-- rule ws? map ws?
rule <-- 'cc_binary_host' / 'cc_test_library' / 'cc_test_host' /
'cc_binary' / 'cc_test' /'cc_library_host_static' /
'cc_library_static' / 'cc_library_shared' /
'cc_library_headers' / 'cc_library' / 'cc_defaults' /
'cc_benchmark' / 'python_test_host' / 'genrule' / 'filegroup' /
'ndk_headers' / 'ndk_library' / 'llndk_library' /
'python_binary_host' / 'cc_prebuilt_binary' / 'prebuilt_etc' /
'python_defaults' / 'phony'
defvar <-- var ws? eq ws? value
eq < '='
nl < '\n'
ws < (' ' / '\t' / nl)+
comma < ','
colon < ':'
maplb < '{'
maprb < '}'
listlb < '['
listrb < ']'
comment< ws? '//' ( . !nl )* . ws?
strb < '\"'
append <-- '+'
key<-- ws? (comment ws?)* [-a-zA-Z0-9_]+
bool <-- 'true' / 'false'
integer<-- '-'? [0-9]+
string <-- strb (. !strb)* . strb
map<-- maplb ws? maprb / maplb ws? attribute (comma comment* ws?
attribute)* comma? ws? maprb
attribute <-- key ws? colon ws? expr
expr <-- (value ws? append ws? &value)* value
value <-- bool / integer / string / map / list / var
var<-- [-a-zA-Z0-9_]+
list <-- listlb ws? listrb / listlb ws? comment* ws? value (comma
comment* ws? value)* comma? ws? comment* listrb
")
(pretty-print (peg:tree (match-pattern blueprint *bp*)))
//
// Copyright (C) 2008 The Android Open Source Project
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// some files must not be compiled when building against Mingw
// they correspond to features not used by our host development tools
// which are also hard or even impossible to port to native Win32
libcutils_nonwindows_sources = [
"fs.cpp",
"hashmap.cpp",
"multiuser.cpp",
"socket_inaddr_any_server_unix.cpp",
"socket_local_client_unix.cpp",
"socket_local_server_unix.cpp",
"socket_network_client_unix.cpp",
"sockets_unix.cpp",
"str_parms.cpp",
]
cc_library_headers {
name: "libcutils_headers",
vendor_available: true,
recovery_available: true,
host_supported: true,
export_include_dirs: ["include"],
target: {
vendor: {
override_ex
