We're long overdue for a status update on the grammar engine
(aka PGE), so here's a quick summary of recent changes and
improvements. My thanks go out to many people on irc #parrot
who have been helping with testing and feature implementations,
and especially particle for the incredible tests he's been
committing to the suite.
I'll start with just a quick bullet summary of new features
and recent changes, and then send later messages with more
details on how some of the features work and the documentation
for each.
The big news is that since last week, PGE has a built-in
general-purpose operator precedence parser (shift/reduce
parsing), and uses that parser as the primary engine for
decoding perl 6 rule expressions. The rules engine and
operator precedence engine are integrated together, so that
one can call a shift/reduce parser from a rule and vice
versa. More details about the operator precedence parser
will be in an upcoming email message.
As a quick example, one can now use the <p6rule> subrule
to parse a perl 6 rule expression, and the Match object
that is returned contains the parse tree. Other examples
and demonstrations or parsing are in the examples/pge/
directory.
PGE also includes a <PGE::Text::bracketed> subrule, and
that subrule can be called with a parameter indicating
the type of delimiters to honor in the bracketed text:
<PGE::Text::bracketed> # nested { }, [ ], ( ), < > (default)
<PGE::Text::bracketed: "> # quoted strings
<PGE::Text::bracketed: ()> # nested parens only
<PGE::Text::bracketed: ()[]'> # nested parens, brackets, quote escapes
We now support conjunctions with the C<&> operator, so to
match identifiers that do not contain digits:
<ident> & \D+
Lastly, here's a summary of PGE's newest features (at least the
ones that I can remember right now...):
* More metachars -> \e\f\r\t\h\v \x{...} and \E\F\R\T\H\V \X{...}
* Conjunctions -> <ident> & \D+
* Lookaheads -> <before pattern> and <!before pattern>
* Negated subrules -> <!subrule>
* Commits -> <commit>
* Closures -> <p6rule> {{ print "found a rule!" }}
* with other langs -> :lang(TCL) <expr> {{ puts "Got an expr" }}
* Bracketed text -> <PGE::Text::bracketed: ()>
* Parse p6 patterns -> <p6rule>
* Modifiers -> :w :words :i :ignorecase :w(0) :i(0)
* Better support for matching unicode strings
* Custom <?ws> rules
Comments, tests, suggestions welcomed as always!
Pm
