founderinaction commented on issue #26336:
URL: https://github.com/apache/datafusion/issues/26336#issuecomment-6710668167
Draft comment for apache/datafusion#26336. DataFusion asks for replies in
your own words, so reword it before posting.
---
I have the three fixes working locally against the Spark 4.2 source. Field
names are now a closed list matching `DatePart.parseExtractField`, so
`QUARTERS`, `DAYOFYEAR` and other non-Spark spellings get a plan error. A NULL
field returns NULL (Spark replaces it with a `DOUBLE` null). SECOND returns
`Decimal(8, 6)`.
One design question before I open the PR. The return type now depends on the
field literal, but since #23232 an explicit cast isn't passed to
`return_field_from_args` as a scalar argument. So for
`date_part('SECOND'::string, ts)`, the style `date_part.slt` uses, planning
declares `Int32`. After the cast is folded, the type becomes `Decimal(8, 6)`
and the optimizer rejects the schema change. A plain literal,
`date_part('SECOND', ts)`, works. Integer fields work either way, because their
type doesn't change.
The options I can see:
1. Require a plain literal for SECOND and a NULL field (the cast form
errors), and switch those slt cases to plain literals. This matches how core
`date_part` already requires a literal field.
2. Keep `Int32` for SECOND, which leaves the fractional-seconds part of this
issue unfixed.
3. Expose string-to-string literal casts as scalar arguments, which would
revisit the explicit-cast rule from #23232.
I'd go with 1, unless you'd prefer a different approach. @andygrove does
that work for you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
