goldmedal commented on code in PR #1621:
URL:
https://github.com/apache/datafusion-sqlparser-rs/pull/1621#discussion_r1898786504
##########
src/tokenizer.rs:
##########
@@ -1141,33 +1141,44 @@ impl<'a> Tokenizer<'a> {
let s2 = peeking_take_while(chars, |ch|
ch.is_ascii_hexdigit());
return Ok(Some(Token::HexStringLiteral(s2)));
}
-
// match one period
- if let Some('.') = chars.peek() {
- // Check if this actually is a float point number
- let mut char_clone = chars.peekable.clone();
- char_clone.next();
- // Next char should be a digit, otherwise, it is not a
float point number
- if char_clone
- .peek()
- .map(|c| c.is_ascii_digit())
- .unwrap_or(false)
- {
+ if self.dialect.support_unquoted_hyphenated_identifiers() {
Review Comment:
> Essentially I'm wondering if we can extend [the
work](https://github.com/apache/datafusion-sqlparser-rs/blob/6daa4b059cde8b77b67a3699b174ef0f8edff350/src/parser/mod.rs#L9015-L9063)
to cover this scenario doublle number scenario. Using the example from #1598
it would essentially mean that e.g. the following token stream
>
> `[Word("foo"), Minus, Number("123."), Word("bar")]`
>
> gets combined into
>
> `ObjectName(vec![Word("foo-123"), Word("bar")])`
>
> would something like that be feasible?
It's a good point. "Keeping tokenizer parse decimals as usual" makes sense
to me.
I'll try it. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
