adriangb opened a new pull request, #27524:
URL: https://github.com/apache/datafusion/pull/27524

   ## Which issue does this PR close?
   
   - None. I can file one if reviewers want it.
   
   ## Rationale for this change
   
   The unparser fails on any expression that casts to a list type:
   
   ```rust
   let expr = cast(col("a"), 
DataType::List(Arc::new(Field::new_list_field(DataType::Utf8, true))));
   Unparser::default().expr_to_sql(&expr)
   // Error: This feature is not implemented: Unsupported DataType: conversion: 
List(Utf8)
   ```
   
   DataFusion's SQL planner already reads `VARCHAR[]` as `List`, `VARCHAR[3]` 
as `FixedSizeList`, and `ARRAY<VARCHAR>` as `List`. The unparser cannot write 
any of them. So a plan that contains such a cast does not round-trip. Type 
coercion can add such a cast to a plan even when the user's SQL has none, for 
example to match a UDF signature.
   
   ## What changes are included in this PR?
   
   - A new `Dialect::array_type_style()` returns a new `ArrayTypeStyle` enum. 
The dialect picks the syntax because array support differs between databases:
   
     | Dialect | Style | `List(Utf8)` | `FixedSizeList(Utf8, 3)` |
     | --- | --- | --- | --- |
     | Default, PostgreSQL, DuckDB | `SquareBracket` | `VARCHAR[]` | 
`VARCHAR[3]` |
     | BigQuery | `AngleBracket` | `ARRAY<STRING>` | error |
     | MySQL, SQLite, Snowflake, `CustomDialect`, and any dialect that does not 
override it | `Unsupported` (trait default) | error | error |
   
     The trait default is `Unsupported`, so every dialect that does not opt in 
keeps the existing error. MySQL and SQLite have no array type. Snowflake's 
`ARRAY` is semi-structured and has no typed element syntax.
   - `CustomDialectBuilder::with_array_type_style` sets the style on a 
`CustomDialect`.
   - `arrow_dtype_to_ast_dtype` handles `List`, `LargeList` and 
`FixedSizeList`. It converts the element type recursively, so nested lists work 
too (`VARCHAR[][]`).
   
   `LargeList` unparses to the same syntax as `List`, so it parses back as 
`List`. This is like `Utf8View`, which already unparses to `VARCHAR`. The 
element field's name and nullability are also not part of the SQL syntax. 
`ListView` and `LargeListView` still return the error.
   
   ## What is the testing strategy for this PR?
   
   - `test_list_to_ast_dtype` in `unparser/expr.rs` checks the output for each 
dialect and list type, and the error for the unsupported cases.
   - `test_cast_to_array` and `test_cast_to_fixed_size_array_default_dialect` 
in `tests/cases/plan_to_sql.rs` round-trip `CAST(NULL AS text[])` and 
`CAST(NULL AS int[2])` through the planner and the unparser.
   
   ## Are there any user-facing changes?
   
   Yes. The unparser can now write casts to list types. The trait method has a 
default that keeps the existing error, so existing `Dialect` implementations 
behave as before. A custom dialect opts in by overriding `array_type_style`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to