daphnenhuch-at commented on issue #15833: URL: https://github.com/apache/datafusion/issues/15833#issuecomment-2842967139
That still doesn't solve my problem here. I added the extra sort on both columns as suggested. Now it seems the userPrimaryKey is sorted and the fileRowNumbers start at 1808 and go until 9999 and then the last stretch is 0 through 1809 which is not sorted. Definitely still seems like the record batch size is playing a funky role here. Are `target_partitions` or `enable_round_robin_partitioning` simply not supported in cases where we need the final result ot be sorted?  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
