alamb commented on code in PR #16290:
URL: https://github.com/apache/datafusion/pull/16290#discussion_r2150955809
##########
datafusion/common/src/config.rs:
##########
@@ -259,10 +259,10 @@ config_namespace! {
/// string length and thus DataFusion can not enforce such limits.
pub support_varchar_with_length: bool, default = true
- /// If true, `VARCHAR` is mapped to `Utf8View` during SQL planning.
- /// If false, `VARCHAR` is mapped to `Utf8` during SQL planning.
- /// Default is false.
- pub map_varchar_to_utf8view: bool, default = true
+ /// If true, string types (VARCHAR, CHAR, Text, and String) are mapped
to `Utf8View` during SQL planning.
+ /// If false, they are mapped to `Utf8`.
+ /// Default is true.
+ pub map_string_types_to_utf8view: bool, default = true
Review Comment:
I think this is an API change, so perhaps we can add a note about this in
the upgrade guide (that we changed the name of the config setting)
##########
datafusion/sqllogictest/test_files/array.slt:
##########
@@ -6082,7 +6082,7 @@ physical_plan
04)------AggregateExec: mode=Partial, gby=[], aggr=[count(Int64(1))]
05)--------ProjectionExec: expr=[]
06)----------CoalesceBatchesExec: target_batch_size=8192
-07)------------FilterExec: substr(md5(CAST(value@0 AS Utf8)), 1, 32) IN
([Literal { value: Utf8View("7f4b18de3cfeb9b4ac78c381ee2ad278"), field: Field {
name: "7f4b18de3cfeb9b4ac78c381ee2ad278", data_type: Utf8View, nullable: false,
dict_id: 0, dict_is_ordered: false, metadata: {} } }, Literal { value:
Utf8View("a"), field: Field { name: "a", data_type: Utf8View, nullable: false,
dict_id: 0, dict_is_ordered: false, metadata: {} } }, Literal { value:
Utf8View("b"), field: Field { name: "b", data_type: Utf8View, nullable: false,
dict_id: 0, dict_is_ordered: false, metadata: {} } }, Literal { value:
Utf8View("c"), field: Field { name: "c", data_type: Utf8View, nullable: false,
dict_id: 0, dict_is_ordered: false, metadata: {} } }])
+07)------------FilterExec: substr(CAST(md5(CAST(value@0 AS Utf8View)) AS
Utf8View), 1, 32) IN ([Literal { value:
Utf8View("7f4b18de3cfeb9b4ac78c381ee2ad278"), field: Field { name:
"7f4b18de3cfeb9b4ac78c381ee2ad278", data_type: Utf8View, nullable: false,
dict_id: 0, dict_is_ordered: false, metadata: {} } }, Literal { value:
Utf8View("a"), field: Field { name: "a", data_type: Utf8View, nullable: false,
dict_id: 0, dict_is_ordered: false, metadata: {} } }, Literal { value:
Utf8View("b"), field: Field { name: "b", data_type: Utf8View, nullable: false,
dict_id: 0, dict_is_ordered: false, metadata: {} } }, Literal { value:
Utf8View("c"), field: Field { name: "c", data_type: Utf8View, nullable: false,
dict_id: 0, dict_is_ordered: false, metadata: {} } }])
Review Comment:
🤔 this looks like it may have added extra casts I wonder if it because
`md5` doesn't support `StringView` natively 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
