graup opened a new pull request, #1679:
URL: https://github.com/apache/datafusion-sqlparser-rs/pull/1679
Fixes #1673. This is a breaking change.
TypedString contained a `String` without any knowledge of the used quote
style. The parser used `parse_literal_string` to construct this, which doesn't
support any quote styles other than single or double quotes. Namely, it doesn't
support triple quotes from BigQuery, causing the issue reported in #1673.
Additionally, it doesn't round-trip properly, always formatting its string
using single quotes.
I think the most proper fix is to have `TypedString` contain a `Value`
instead, similar to IntroducedString and others. This gives us immediate
support for other quote styles and fixes the formatting to make it
roundtrippable.
This is a breaking change but should be an easy fix in users' codebases,
just (un)wrapping the value. Migration path:
1. When constructing an AST node
```diff
Expr::TypedString {
data_type: DataType::JSON,
-- value: r#"{"class" : {"students" : [{"name" : "Jane"}]}}"#.to_string()
++ value: Value::SingleQuotedString(
++ r#"{"class" : {"students" : [{"name" : "Jane"}]}}"#.to_string()
++ )
},],
```
2. When using AST parser results
```diff
if let Expr::TypedString { data_type, value } = expr {
-- let string_value = value;
++ let string_value = match value {
++ Value::SingleQuotedString(s)
++ | Value::DollarQuotedString(s)
++ | Value::TripleSingleQuotedString(s)
++ | Value::TripleDoubleQuotedString(s)
++ | Value::EscapedStringLiteral(s)
++ | Value::UnicodeStringLiteral(s)
++ | Value::SingleQuotedByteStringLiteral(s)
++ | Value::DoubleQuotedByteStringLiteral(s)
++ | Value::TripleSingleQuotedByteStringLiteral(s)
++ | Value::TripleDoubleQuotedByteStringLiteral(s)
++ | Value::SingleQuotedRawStringLiteral(s)
++ | Value::DoubleQuotedRawStringLiteral(s)
++ | Value::TripleSingleQuotedRawStringLiteral(s)
++ | Value::TripleDoubleQuotedRawStringLiteral(s)
++ | Value::NationalStringLiteral(s)
++ | Value::HexStringLiteral(s)
++ | Value::DoubleQuotedString(s)
++ | Value::TripleDoubleQuotedString(s) => s,
++ _ => "".to_string(),
};
}
```
(Actually, seeing this, maybe it would be nice to have a `Value Into String`
impl? Or is there another Rust idiomatic way?)
While fixing this I noticed another issue: Value uses `escape_quoted_string`
to format single and double quoted strings. This doesn't take into account the
dialect, producing invalid output for BigQuery like `'foo''bar'` (should be
`'foo\'bar'`). I think that's a separate issue better addressed by a separate
PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
