comphead commented on issue #12359: URL: https://github.com/apache/datafusion/issues/12359#issuecomment-2350278684
For LeftOuter/RightOuter and FullJoin the rootcause is the same. SMJ emits a null row if the filtered match is not found. But if the filtered match row is found it should emit the joined row instead of null row. And here we come to the same issue as for LeftAnti, so we need to know when all the right rows processed for the given left row. Given that SMJ emits rows in `freeze_streamed` which can be called anytime once the output size is equal to batch size there is 2 possible options: - Try to calculate if the SMJ is about to switch to ordering == Less which moves left pointer down, meaning everything is processed for the row - Move emit to some other place, for example we can fill the buffer with rows and filtered flags until the ordering switch, and only then make a final emit based on the data in the buffer + filtered -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
