freakyzoidberg commented on code in PR #23337:
URL: https://github.com/apache/datafusion/pull/23337#discussion_r3581022550
##########
datafusion/functions-nested/src/array_has.rs:
##########
@@ -344,6 +361,229 @@ fn array_has_dispatch_for_array<'a>(
Ok(Arc::new(BooleanArray::new(result.finish(), combined_nulls)))
}
+/// Average list length past which the element-null fast path (case 2 of
+/// [`array_has_array_primitive`]) stops beating the per-row `eq` kernel and is
+/// bailed out of -- an empirically measured crossover.
+const NULL_FAST_PATH_MAX_LEN: usize = 512;
+
+/// Per-element fast path for primitive and string element types. Returns
`None`
+/// for any other type (and on a needle/element type mismatch), so the caller
+/// falls back to the per-row `eq` kernel.
+fn array_has_array_fast_path(
+ haystack: &ArrayWrapper<'_>,
+ needle: &ArrayRef,
+ combined_nulls: Option<&NullBuffer>,
+) -> Option<BooleanBuffer> {
+ let needle = needle.as_ref();
+
+ // Normalize for sliced arrays (like `array_has_dispatch_for_scalar`):
slice
+ // to the visible region so `offsets[i] - first_offset` indexes the values.
+ let offsets: Vec<usize> = haystack.offsets().collect();
+ let first_offset = offsets[0];
+ let visible_values = haystack
+ .values()
+ .slice(first_offset, offsets[offsets.len() - 1] - first_offset);
+ let visible_values = visible_values.as_ref();
Review Comment:
Correct, sorry, my branch was stale compared to main - and sill on 0.59.0.
I pushed the change in the last version.
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
