I'm currently looking into the FPC parser, in order to make it usable as
a general parser, apart from the compiler.
One required change is the handling of used units and identifiers. While
a compiler needs all used files, must inline all included files, and
must know about the declaration of every identifier, a simple parser
(e.g. for syntax highlighting) should do nothing special with such
tokens. For a separation of compiler specific actions from other usages
of the parser I've introduced a Semantics class, that has virtual
methods for all parser events. The base class does almost nothing,
another (derived) CompilerSemantics class will be the new home of the
compiler specific code in the parser. An application can provide its own
class, as required. This separation is the really hairy part, when we
ask the FPC team to accept these modifications.
Another change to the scanner is required, so that it does not only skip
over comments. This modification is not critical, since the scanner will
not normally deserve updates when the language is extended. So we can
provide another scanner, that fits our needs.
But moving the compiler specific code out-of-sight is not enough. A
parser for syntax-highlighting must handle all tokens, even those in
conditional branches. A simple solution could be another Semantics
method, that is called when excluded tokens occur, so that the
application knows about the type of every token.
Another issue is a parse tree or AST, that already is created by the FPC
parser. Unfortunately the nodes in that tree are very compiler specific,
so that it may be required to remove that automatic tree construction,
and let it happen in Semantics actions.
And finally we'll have to talk about parsing invalid code, during
editing. This can be handled by the same hack as for conditionally
excluded code, so that after an syntax error the parser stops parsing,
and simply returns the tokens provided by the scanner. The only
question: how to make the parser resume parsing, once enough tokens have
been skipped?
Now the question is: how to proceed?
I can make available an patch for the parser units of the last fpc
release, implementing the Semantics separation. It may be more
comfortable for the users when a SVN subproject would hold the units to
modify, so that these units can be kept in sync with the FPC trunk. Can
somebody do that?
Then I can provide a package containing stubs, for all units used by the
parser units, so that the parser can be compiled independently from FPC.
And we'll need a test application...
DoDi
--
_______________________________________________
Lazarus mailing list
[email protected]
http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
