[
https://issues.apache.org/jira/browse/CALCITE-7820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Iurii Gerzhedovich updated CALCITE-7820:
----------------------------------------
Description:
default_config.fmpp documents nonReservedKeywordsToRemove as:
# List of non-reserved keywords to remove;
# items in this list become reserved.
nonReservedKeywordsToRemove: [
]
This implies a sub-parser can list a keyword here to make it reserved even
though Calcite's own default_config.fmpp treats it as non-reserved. However,
core/src/main/codegen/templates/Parser.jj never reads this field. The only
place the non-reserved keyword set is assembled is:
<#list (parser.nonReservedKeywords!default.parser.nonReservedKeywords) +
(parser.nonReservedKeywordsToAdd!default.parser.nonReservedKeywordsToAdd) as
keyword>
(repeated 3 times, for NonReservedKeyWord0of3/1of3/2of3). There is no reference
to nonReservedKeywordsToRemove anywhere in Parser.jj, so setting it in a
sub-parser's config.fmpp has zero effect on the generated grammar — the keyword
remains non-reserved.
History: the field was introduced (as an empty list, alongside
nonReservedKeywordsToAdd) in commit 05a086bc6 ("Re-format and re-organize
config.fmpp files...", Oct 2019) and carried through CALCITE-4238 (Sep 2020,
the commit that added the !default... fallback for
nonReservedKeywords/nonReservedKeywordsToAdd) without ever being wired in. It
appears to have never been exercised by any sub-parser (core's own
default_config.fmpp keeps it empty too).
Reproduction:
1. In any Calcite-based parser module, add a word that's in
default_config.fmpp's nonReservedKeywords (e.g. "ADD") to your own
config.fmpp's nonReservedKeywordsToRemove.
2. Regenerate the parser (generateFmppSources/generateParser).
3. The word still parses as a non-reserved identifier — isReservedWord("ADD")
on the generated SqlAbstractParserImpl.Metadata still returns false.
Suggested fix: either wire nonReservedKeywordsToRemove into the <#list ...>
expressions in Parser.jj (subtract it from the combined list), or remove the
field and its documentation from default_config.fmpp if it's intentionally
unsupported, to stop misleading downstream parser authors.
was:
default_config.fmpp documents nonReservedKeywordsToRemove as:
# List of non-reserved keywords to remove;
# items in this list become reserved.
nonReservedKeywordsToRemove: [
]
This implies a sub-parser can list a keyword here to make it reserved even
though Calcite's own default_config.fmpp treats it as non-reserved. However,
core/src/main/codegen/templates/Parser.jj never reads this field. The only
place the non-reserved keyword set is assembled is:
<#list (parser.nonReservedKeywords!default.parser.nonReservedKeywords) +
(parser.nonReservedKeywordsToAdd!default.parser.nonReservedKeywordsToAdd) as
keyword>
(repeated 3 times, for NonReservedKeyWord0of3/1of3/2of3). There is no reference
to nonReservedKeywordsToRemove anywhere in Parser.jj, so setting it in a
sub-parser's config.fmpp has zero effect on the generated grammar — the keyword
remains non-reserved.
History: the field was introduced (as an empty list, alongside
nonReservedKeywordsToAdd) in commit 05a086bc6 ("Re-format and re-organize
config.fmpp files...", Oct 2019) and carried through CALCITE-4238 (Sep 2020,
the commit that added the !default... fallback for
nonReservedKeywords/nonReservedKeywordsToAdd) without ever being wired in. It
appears to have never been exercised by any sub-parser (core's own
default_config.fmpp keeps it empty too).
Reproduction:
1. In any Calcite-based parser module, add a word that's in
default_config.fmpp's nonReservedKeywords (e.g. "ADD") to your own
config.fmpp's nonReservedKeywordsToRemove.
2. Regenerate the parser (generateFmppSources/generateParser).
3. The word still parses as a non-reserved identifier — isReservedWord("ADD")
on the generated SqlAbstractParserImpl.Metadata still returns false.
Suggested fix: either wire nonReservedKeywordsToRemove into the <#list ...>
expressions in Parser.jj (subtract it from the combined list), or remove the
field and its documentation from default_config.fmpp if it's intentionally
unsupported, to stop misleading downstream parser authors.
Workaround in use downstream: GridGain's Ignite-3-based SQL parser computes the
effective nonReservedKeywords list itself at build time (default minus
nonReservedKeywordas an explicit override, since the field can't
be relied on as documented.
> nonReservedKeywordsToRemove in parser config.fmpp is documented but never
> consumed by Parser.jj
> -----------------------------------------------------------------------------------------------
>
> Key: CALCITE-7820
> URL: https://issues.apache.org/jira/browse/CALCITE-7820
> Project: Calcite
> Issue Type: Improvement
> Reporter: Iurii Gerzhedovich
> Priority: Major
>
> default_config.fmpp documents nonReservedKeywordsToRemove as:
> # List of non-reserved keywords to remove;
> # items in this list become reserved.
> nonReservedKeywordsToRemove: [
> ]
> This implies a sub-parser can list a keyword here to make it reserved even
> though Calcite's own default_config.fmpp treats it as non-reserved. However,
> core/src/main/codegen/templates/Parser.jj never reads this field. The only
> place the non-reserved keyword set is assembled is:
> <#list (parser.nonReservedKeywords!default.parser.nonReservedKeywords) +
> (parser.nonReservedKeywordsToAdd!default.parser.nonReservedKeywordsToAdd) as
> keyword>
> (repeated 3 times, for NonReservedKeyWord0of3/1of3/2of3). There is no
> reference to nonReservedKeywordsToRemove anywhere in Parser.jj, so setting it
> in a sub-parser's config.fmpp has zero effect on the generated grammar — the
> keyword remains non-reserved.
> History: the field was introduced (as an empty list, alongside
> nonReservedKeywordsToAdd) in commit 05a086bc6 ("Re-format and re-organize
> config.fmpp files...", Oct 2019) and carried through CALCITE-4238 (Sep 2020,
> the commit that added the !default... fallback for
> nonReservedKeywords/nonReservedKeywordsToAdd) without ever being wired in. It
> appears to have never been exercised by any sub-parser (core's own
> default_config.fmpp keeps it empty too).
> Reproduction:
> 1. In any Calcite-based parser module, add a word that's in
> default_config.fmpp's nonReservedKeywords (e.g. "ADD") to your own
> config.fmpp's nonReservedKeywordsToRemove.
> 2. Regenerate the parser (generateFmppSources/generateParser).
> 3. The word still parses as a non-reserved identifier — isReservedWord("ADD")
> on the generated SqlAbstractParserImpl.Metadata still returns false.
> Suggested fix: either wire nonReservedKeywordsToRemove into the <#list ...>
> expressions in Parser.jj (subtract it from the combined list), or remove the
> field and its documentation from default_config.fmpp if it's intentionally
> unsupported, to stop misleading downstream parser authors.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
