edmondop opened a new issue, #2604:
URL: https://github.com/apache/datafusion-sqlparser-rs/issues/2604
Problem
`Dialect` can take over parsing at four points: `parse_prefix`,
`parse_infix`, `parse_statement`, `parse_column_option`. Each returns
`Option<Result<_, ParserError>>`, and `None` falls back to the default parser.
Table references have no such hook: the plain-table branch of
`Parser::parse_table_factor` unconditionally calls
`self.parse_object_name(true)?`.
As a result, dialect-specific table-name syntax lives in the core parser
behind hardcoded branches:
- BigQuery's unquoted hyphenated names: `dialect_of!(self is
BigQueryDialect) && in_table_clause` in `parse_object_name_inner` (the
`in_table_clause` parameter exists only for this case).
- Snowflake's `@stage` references: a `supports_stages()` branch in
`parse_table_factor` calling `parse_snowflake_stage_table_factor`.
A downstream dialect with its own table-name syntax can't add a branch like
these. The only option today is to tokenize, rewrite the token stream, and
re-parse via `Parser::with_tokens_with_locations`: a second, token-level parser
that can't see whether the parser is inside a FROM clause.
Motivation
Several tools hit this wall:
- Engines that address tables as `namespace:table`, e.g. `FROM
events:analytics` for table `analytics` in namespace `events`.
- Tools parsing dbt SQL, where FROM clauses can contain `{{ ref('model') }}`
or `{{ source('schema', 'table') }}` macros.
- Any catalog with non-standard table addressing that a custom `Dialect`
should parse natively instead of pre-processing.
Proposal
Add a hook with the same contract as the existing four:
```rust
/// Dialect-specific parser for the name of a table in a table factor.
///
/// If `None` is returned, falls back to the default behavior.
fn parse_table_factor_name(&self, _parser: &mut Parser) ->
Option<Result<ObjectName, ParserError>> {
None
}
```
`parse_table_factor` would call it where it calls `parse_object_name(true)`
today. Everything after the name (JSON path, version, partitions, alias,
sample, hints) stays in the core parser. The hook replaces only the name, so a
dialect can't break alias or join parsing.
Alternatives considered
- Hook all of `parse_table_factor`. More general, but a dialect that wants
only new name syntax would have to reimplement aliases, TABLESAMPLE, and the
rest.
- Allow `:` through `Dialect::is_identifier_part`. Then `events:analytics`
tokenizes as one identifier, but the colon becomes legal in every identifier
position (so `x:y` also lexes as one identifier anywhere), and the result is a
single `Ident` that callers must split after parsing.
- Keep rewriting tokens before parsing. Works, but duplicates parsing logic
at the token level without clause context.
Possible follow-up
BigQuery's hyphenated names could move onto this hook, which would let
`parse_object_name` drop the `in_table_clause` parameter. That would
demonstrate the hook covers syntax already in the tree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
