Hi Spark devs, I've recently opened SPARK-62006 <https://issues.apache.org/jira/browse/SPARK-62006>, which preserves output ordering for full outer joins using sort-merge join.
The motivation is to retain useful ordering guarantees from the sort-merge join, allowing unnecessary downstream sorting to be avoided. I'd appreciate feedback on the approach, particularly from anyone familiar with Spark's sort-merge join implementation and physical operator ordering guarantees. I also have another smaller, unrelated open PR which improves allocs on writing to Parquet: - PR #59566 <https://github.com/apache/spark/pull/59566>: [SPARK-60321][SQL] Avoid intermediate copies of byte-array-backed strings in Parquet writes. Thanks a lot! Best, Saath
