aglinxinyuan commented on PR #6132:
URL: https://github.com/apache/texera/pull/6132#issuecomment-4885226216
Thanks @Yicong-Huang — addressed all three:
1. **Native Sort** — swapped the hand-rolled Python sort UDF for the native
`SortOpDesc` (pandas `sort_values`).
2. **Operator coverage** — reworked the X into one representative per
family: CSV scan, hash join, aggregate, native Sort, union, projection, Python
UDF. Join-build / aggregate / sort are all blocking, so the plan still spans
several regions crossed by the reader-thread path.
3. **PR description** — rewritten to match the code (the old diagram still
described the pre-rework Python-source version).
Assertions now: terminal A (join→aggregate→sort) → 7 per-region COUNTs,
sorted by `Region`, summing to 100; terminal B (union→projection→UDF) → 200
rows narrowed to `{Region, Order ID}`. Compiles + scalafmt + scalafix clean
locally; the run itself is covered by the `amber-integration` job.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
