[
https://issues.apache.org/jira/browse/SPARK-61177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-61177:
-----------------------------------
Labels: pull-request-available (was: )
> Quote the field names of DataType::to_ddl, and keep the case of the collation
> of DataType::from_ddl, in the Rust client
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61177
> URL: https://issues.apache.org/jira/browse/SPARK-61177
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> Two defects of the DDL of the Rust client's DataType:
> 1. DataType::to_ddl writes the name of each field as it is, with format!("{}
> {}", name, type): a field "my col" gives "my col int", which Spark cannot
> parse, and a backtick in a name is not escaped. Spark's StructType.toDDL
> quotes the names that need it ("`my col` INT"), and so do nested struct
> fields.
> 2. DataType::from_ddl lowercases its input before it reads the collation of a
> string, so "a STRING COLLATE UTF8_LCASE" gives the collation "utf8_lcase",
> where Spark gives UTF8_LCASE. The json() of the type and its equality with
> the types of the server then differ.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
