Le 28/10/2014 11:23, Mark Rizun a écrit :
Thanks. I see that, however RBBlockNode or RBArrayNode doesn't have tokens.
These classes have only methods in accessing-token protocol.
I think it would be better if we have token object for those classes,
because it makes more sense to hold such information in token object.
Well, not really.
Technically, tokens are used to drive a parser from a scanner.
If an AST node knows how to relate itself to its original source code
chunk and is able to print itself correctly, then tokens are redundant.
In short, if you work with parsers, you'd better know what tokens are.
If you're only working with the AST, tokens are redundant and noise
(i.e. they often have a type (or more than one) which is only understood
by the parser).
Example of how it is done:
RBPragmaNode
accessing-tokens gives access to left and right, which are positions,
not tokens.
Thierry
Mark
2014-10-28 11:59 GMT+02:00 Nicolai Hess <nicolaih...@web.de
<mailto:nicolaih...@web.de>>:
2014-10-27 19:36 GMT+01:00 Mark Rizun <mri...@gmail.com
<mailto:mri...@gmail.com>>:
Hi all,
Trying to understand here how tokens are used in AST.
So far I can not see any order in usage of tokens.
For instance, why RBValueNode doesn't have token? Is it haow
it's supposed to be?
RBValueNode is an abstract class.
Cheers,
Mark
