Rich-T-kid commented on code in PR #25873:
URL: https://github.com/apache/datafusion/pull/25873#discussion_r4146805093
##########
datafusion/functions/src/utils.rs:
##########
@@ -120,6 +120,78 @@ where
move |args: &[ColumnarValue]| {
// first, identify if any of the arguments is an Array. If yes, store
its `len`,
// as any scalar will need to be converted to an array of len `len`.
+ //
+ // When exactly one array arg is dict-encoded and no plain-array args
+ // exist alongside it, and NDV < batch size, evaluate `inner` on the
+ // values array (O(NDV)) and scatter the result back with `take`
(O(N)).
+ // Callers opt in by setting EncodingPreservation::dictionary() on the
+ // coercion; their return_type must also strip the dict wrapper so the
+ // declared output type matches the plain-array result of `take`.
+ // When NDV >= batch size the optimization pays no dividend, so we fall
+ // through and cast the dict to its value type before calling `inner`.
Review Comment:
may not be worth it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
