edmondop commented on PR #25798: URL: https://github.com/apache/datafusion/pull/25798#issuecomment-6171219035
@alamb, I put together a small consumer-side experiment in [apache/datafusion-ballista#2592](https://github.com/apache/datafusion-ballista/pull/2592). The test places a test-only `StageBoundary` below Ballista’s real `ShuffleWriterExec` and runs the plan through the default query-stage executor. A test-side driver waits until both boundary streams reach the release gate, primes both input partitions, waits for both to become ready, and confirms that no batches have reached the shuffle writer. It then releases the boundary and checks that the shuffle files contain the original batches. This shows the API can be used in a Ballista worker-stage plan. The test—not Ballista’s scheduler—calls `prime`, `is_ready`, and `release`, so it doesn’t claim scheduler integration. The PR is marked **DO NOT MERGE** and is only an API-consumer experiment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
