andygrove opened a new pull request, #6527: URL: https://github.com/apache/datafusion-comet/pull/6527
## Which issue does this PR close? Backport of #6312, which fixed #6301. ## Rationale for this change The mixed field-ID directory test is flaky on Spark 3.4 and 3.5 because its two writes can leave empty Parquet files. Depending on which task fails first, Spark adds a different exception wrapper and the one-level cause assertion intermittently fails. PR #6511 hit this exact failure in its Spark 3.5 scans job. The source PR explicitly identified `branch-1.1` as needing the same change. ## What changes are included in this PR? This cherry-picks #6312 onto `branch-1.1`. Each side of the test is repartitioned to one output file, eliminating the empty files and leaving one deterministic failing task. The backport has the same stable patch ID as the source commit. ## How are these changes tested? The source PR passed the Comet suites across Spark 3.4, 3.5, 4.0, and 4.2. For this backport, `git diff --check` passes and `git patch-id --stable` confirms that the patch is identical to #6312. The release-branch PR CI will rerun the full required test tier. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
