Iurii Gerzhedovich created CALCITE-7820:
-------------------------------------------
Summary: nonReservedKeywordsToRemove in parser config.fmpp is
documented but never consumed by Parser.jj
Key: CALCITE-7820
URL: https://issues.apache.org/jira/browse/CALCITE-7820
Project: Calcite
Issue Type: Improvement
Reporter: Iurii Gerzhedovich
default_config.fmpp documents nonReservedKeywordsToRemove as:
# List of non-reserved keywords to remove;
# items in this list become reserved.
nonReservedKeywordsToRemove: [
]
This implies a sub-parser can list a keyword here to make it reserved even
though Calcite's own default_config.fmpp treats it as non-reserved. However,
core/src/main/codegen/templates/Parser.jj never reads this field. The only
place the non-reserved keyword set is assembled is:
<#list (parser.nonReservedKeywords!default.parser.nonReservedKeywords) +
(parser.nonReservedKeywordsToAdd!default.parser.nonReservedKeywordsToAdd) as
keyword>
(repeated 3 times, for NonReservedKeyWord0of3/1of3/2of3). There is no reference
to nonReservedKeywordsToRemove anywhere in Parser.jj, so setting it in a
sub-parser's config.fmpp has zero effect on the generated grammar — the keyword
remains non-reserved.
History: the field was introduced (as an empty list, alongside
nonReservedKeywordsToAdd) in commit 05a086bc6 ("Re-format and re-organize
config.fmpp files...", Oct 2019) and carried through CALCITE-4238 (Sep 2020,
the commit that added the !default... fallback for
nonReservedKeywords/nonReservedKeywordsToAdd) without ever being wired in. It
appears to have never been exercised by any sub-parser (core's own
default_config.fmpp keeps it empty too).
Reproduction:
1. In any Calcite-based parser module, add a word that's in
default_config.fmpp's nonReservedKeywords (e.g. "ADD") to your own
config.fmpp's nonReservedKeywordsToRemove.
2. Regenerate the parser (generateFmppSources/generateParser).
3. The word still parses as a non-reserved identifier — isReservedWord("ADD")
on the generated SqlAbstractParserImpl.Metadata still returns false.
Suggested fix: either wire nonReservedKeywordsToRemove into the <#list ...>
expressions in Parser.jj (subtract it from the combined list), or remove the
field and its documentation from default_config.fmpp if it's intentionally
unsupported, to stop misleading downstream parser authors.
Workaround in use downstream: GridGain's Ignite-3-based SQL parser computes the
effective nonReservedKeywords list itself at build time (default minus
nonReservedKeywordas an explicit override, since the field can't
be relied on as documented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
