jayshrivastava commented on issue #21207: URL: https://github.com/apache/datafusion/issues/21207#issuecomment-5027120629
Took a brief look at https://github.com/apache/datafusion/pull/21931. The implementation in my PR happens to be - `InList`: `concat` the expressions - `Map`: `OR` the maps Not sure if the `OR` is better or worse than `MultiMapLookupExpr`. In the `MultiMapLookupExpr` we process all rows in all the maps. I think `OR` also ends up vectorizing the hash lookup for each row for each map. I wonder if it's worth exploring a way to prune the record batch as soon as one row evaluates `true` in the `OR`, skipping subsequent maps. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
