aglinxinyuan commented on PR #6126: URL: https://github.com/apache/texera/pull/6126#issuecomment-4884617099
Added `MultiRegionWorkflowIntegrationSpec` (9a31f76) to cover the multi-region path end-to-end, since removing the cache sources touches cross-region delivery and we only had scheduling unit tests there before. It's the X-shaped workflow you described: two CSV sources fan into a hash join (build/probe) → a Python sort UDF, and also into a union — the join and union are the two terminal outputs. The join's build→probe dependency forces the build side to be materialized, so the plan spans ≥2 regions and the data crosses a region boundary via the input-port materialization reader threads (the mechanism that replaced cache source in #3425). It asserts both structure and correctness: ≥2 regions (via the same `CostBasedScheduleGenerator` the controller uses), the self-join on the unique `Order ID` key yields 100 rows sorted by the Python UDF, and the union concatenates both sources into 200 rows. Tagged `@IntegrationTest` so it runs in the `amber-integration` job with real Python workers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
