pingzh opened a new pull request, #6373: URL: https://github.com/apache/datafusion-comet/pull/6373
## Which issue does this PR close? Closes #6118. ## Rationale for this change Celeborn setup and compatibility information is buried in the tuning guide. In particular, requesting native shuffle with the tested Celeborn 0.6.0 and 0.7.0 clients still delegates exchanges to ordinary Spark/Celeborn shuffle. Users need that support status before following a setup recipe. ## What changes are included in this PR? - Add a Celeborn guide under Integrations, following the existing integration-guide style, with support status, client artifact coordinates, driver/executor startup classpaths, configuration, supported behavior, and troubleshooting. - Explain how to distinguish delegated from native exchanges, and the current observability limit: neither `CometExchange` nor remote-read byte counters identify Celeborn versus a local replacement. - Document existing shuffle metrics and output accounting after oversized-row fallback. - Keep frame/admission budgets and compression advice in `tuning/celeborn.md`, using the current tuning-page layout, and replace duplicated configuration prose with links. The guide was checked against #5352, its linked upstream PRs, the subsequent compatibility and oversized-row fixes, and the current implementation and tests. It does not claim a released native-compatible Celeborn client or live-cluster qualification. ## How are these changes tested? - Prettier check on all changed Markdown files and `git diff --check` passed. - Sphinx HTML build from `docs/source` passed. Its 67 warnings exactly match an unchanged-source baseline (including missing generated release/compatibility pages, existing cross-references, and the unavailable Mermaid CLI); this change adds no warnings. - Checked 26 rendered internal links/anchors and the new integration navigation entry. - Independently reviewed setup, compatibility, fallback, and metrics statements against source and tests. Documentation only; no Spark/Celeborn cluster tests or full release-site generation were run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
