andygrove opened a new pull request, #2989: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2989
Dialects: Spark `SparkSqlDialect` does not process backslash escapes in string literals, so `SELECT '\thello'` parses to a backslash followed by `thello`, and `SELECT 'I\'m'` fails to parse. Spark unescapes these ([string literals](https://spark.apache.org/docs/latest/sql-ref-literals.html#string-literal), [`unescapeSQLString`](https://github.com/apache/spark/blob/v4.0.0/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkParserUtils.scala)), as `DatabricksDialect` already does here. This enables `supports_string_literal_backslash_escape` for Spark. It also enables `ignores_wildcard_escapes`, because Spark keeps the backslash in `\%` and `\_`, as MySQL does. Not covered: `\uXXXX`, `\UXXXXXXXX` and octal escapes, and `\a` / `\f`, which Spark reads as `a` / `f`. The tokenizer handles those the same way for every dialect. Found through apache/datafusion#21516. This PR was written with AI assistance (Claude Code). I reviewed the change and the tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
