(If you received this twice: I'm sorry, I'm not sure if my message was
accepted by mailman before I subscribed to the list)
Hi all,
I'm using bison with "Complete Symbols" with %define api.token.constructor,
and yylex is defined to return symbol_type with YY_DECL.
I have some testcases that test basic lexing.
Basically, I parse/lex some synthetic example files, and make sure the
right tokens and values are returned.
Here's a snippet:
for (auto id : ids) {
auto symbol = lexer->yylex(*driver);
ASSERT_EQ(symbol.token(), token::T_BASIC_IDENTIFIER);
ASSERT_EQ(symbol.value.as<std::string>(), id);
}
(Where BASIC_IDENTIFIER is one of the tokens defined in my .yy file and
T_BASIC_IDENTIFIER is in the generated token_type / yytokentype enum)
This was working with Bison 3.4.x, but stopped working with Bison 3.5:
error: no member named 'token' in '...::Parser::symbol_type'
ASSERT_EQ(symbol.token(), token::T_BASIC_IDENTIFIER);
token() was removed in commit 8c87a623, so the error is understandable.
Unfortunatly, I don't understand the distinction between internal and
external token numbers, so I'm not sure how to proceed.
I did try using .type / .type_get() instead of .token(), but as I
understand it these return the internal token numbers instead of the
external token numbers.
And since the yytokentype enum (e.g. T_BASIC_IDENTIFIER) uses the
external numbers, they don't match.
Maybe I'm doing something unorthodox, but what's the best way of
accessing/checking which token type is returned by yylex()?
Cheers, Felix
