iffyio commented on code in PR #1803:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1803#discussion_r2036578066
##########
src/tokenizer.rs:
##########
@@ -895,7 +895,7 @@ impl<'a> Tokenizer<'a> {
};
let mut location = state.location();
- while let Some(token) = self.next_token(&mut state)? {
+ while let Some(token) = self.next_token(&mut state, buf.last().map(|t|
&t.token))? {
Review Comment:
Can we implement instead in the parser as a special for
`self.dialect.supports_numeric_prefix()`? somewhat similar to what we do for
[BigQuery
here](https://github.com/apache/datafusion-sqlparser-rs/blob/main/src/parser/mod.rs#L9971).
In that when its time to combine identifiers into a compound identifier we
check if each subsequent identifier part is unquoted and prefixed by `.`, and
if so we drop the prefix. I imagine that would be [done
here](https://github.com/apache/datafusion-sqlparser-rs/blob/main/src/parser/mod.rs#L1859)
Thinking that could be a smaller change since the behavior needs a bit of
extra context around the tokens which the tokenizer isn't so good at handling
cleanly
##########
tests/sqlparser_mysql.rs:
##########
@@ -1926,6 +1926,106 @@ fn parse_select_with_numeric_prefix_column_name() {
}
}
+#[test]
+fn parse_qualified_identifiers_with_numeric_prefix() {
+ // Case 1: Qualified column name that starts with digits.
+ mysql().verified_stmt("SELECT t.15to29 FROM my_table AS t");
+ match mysql()
+ .parse_sql_statements("SELECT t.15to29 FROM my_table AS t")
+ .unwrap()
+ .pop()
+ {
Review Comment:
```suggestion
match mysql().verified_stmt("SELECT t.15to29 FROM my_table AS t") {
```
does this format work the same to remove the second parse statement?
##########
tests/sqlparser_mysql.rs:
##########
@@ -1926,6 +1926,106 @@ fn parse_select_with_numeric_prefix_column_name() {
}
}
+#[test]
+fn parse_qualified_identifiers_with_numeric_prefix() {
+ // Case 1: Qualified column name that starts with digits.
+ mysql().verified_stmt("SELECT t.15to29 FROM my_table AS t");
Review Comment:
could we add a test case for the behavior of multiple accesses e.g.
`t.15to29.16to30`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
