andygrove commented on code in PR #2293: URL: https://github.com/apache/datafusion-ballista/pull/2293#discussion_r3777047968
########## docs/source/contributors-guide/development.md: ########## @@ -182,6 +182,29 @@ This includes instructions for: - Comparing performance with Apache Spark - Running load tests +## DataFusion dependency + +Between releases, `main` depends on a pinned git revision of +[`apache/datafusion`](https://github.com/apache/datafusion) `main` rather than a crates.io release, so +that breaking changes arrive in small increments instead of piling up at release time. + +The `datafusion*` entries in the workspace `Cargo.toml` move through one cycle per release: + +| Phase | `datafusion*` dependency | +| ---------------------------- | ------------------------------------------- | +| Between releases | pinned git `rev` of DataFusion `main` | +| DataFusion release candidate | the RC tag, to validate early | +| DataFusion released | the crates.io version | +| Ballista release branch cut | (branch keeps the crates.io version) | +| Back on `main` | pinned git `rev` of DataFusion `main` again | + +A release branch therefore only ever depends on published crates. Each `rev` bump on `main` is a PR +that ports any DataFusion API changes, refreshes affected golden files, and keeps `cargo test` and +`cargo clippy --all-targets --workspace -- -D warnings` green. + +The remaining release steps are described in the +[release process](https://github.com/apache/datafusion-ballista/blob/main/dev/release/README.md). + Review Comment: Actually, we can add a skill to the repo for doing the upgrade pr. We can run manually to start with and then try GitHub agent feature to automate it perhaps -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
