mrhhsg opened a new pull request, #69635: URL: https://github.com/apache/doris/pull/69635
## Summary Backport cloud-mode S3-vault-backed query spill from #68032 to branch-4.1. Local spill remains the default; S3 spill is opt-in with `spill_storage_type=s3`. - Add remote spill directory, file I/O, bounded multipart uploads, read coalescing, cleanup/retry, and per-file filesystem pinning. - Report retained remote-spill bytes from BEs to the master FE and display them as a separate `__remote_spill__` row in cloud-mode `SHOW DATA` warehouse views. The value is best-effort, not a billing metric. - Include remote spill read/write traffic in query profiles and audit statistics. - Preserve branch-4.1 object-storage interfaces, `SHOW DATA` result layouts, and Java 8 test compatibility. Add only the optional Thrift fields needed for reporting and audit dispatch. ## Validation - FE build and ASAN BE build with Java extensions: passed. - Focused FE unit tests: 80 passed. - Focused ASAN BE unit tests: 155 passed (82 spill/runtime-state, 73 affected-operator tests). - Local `test_show_data_warehouse` and `test_audit_log_behavior` regressions: passed. - `clang-format`, `check-format`, and committed diff whitespace check: passed. - A live cloud S3-vault end-to-end test was not run locally. ## Release note Cloud-mode query spill can use an S3 storage vault when configured with `spill_storage_type=s3`. `SHOW DATA` reports a best-effort view of currently retained remote spill bytes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
