andygrove opened a new pull request, #1991: URL: https://github.com/apache/datafusion-ballista/pull/1991
# Which issue does this PR close? N/A — no tracking issue. Happy to file one if preferred. # Rationale for this change `ballista/core/proto/datafusion.proto` and `datafusion_common.proto` are vendored copies of DataFusion's protobuf definitions. They are **not** compiled into Rust here: `ballista/core/build.rs` maps their packages to the real `datafusion-proto` / `datafusion-proto-common` crates via `extern_path`, so the vendored files exist only so that `protoc` can resolve `ballista.proto`'s `import` statements at build time. Because of that they must stay compatible with the DataFusion version Ballista depends on. Today they are re-synced by hand whenever DataFusion is upgraded (and occasionally in standalone catch-up commits). That drifts silently: nothing fails the build when the vendored stubs lag the pinned crate, because a stub only needs the definitions `ballista.proto` actually references. # What changes are included in this PR? - `dev/update_datafusion_proto.py` — copies the two `.proto` files out of the exact crate source resolved by `cargo metadata --locked` and rewrites the one import path that differs from our flat layout (`datafusion/proto-common/proto/datafusion_common.proto` → `datafusion_common.proto`). A `--check` mode prints a diff and exits non-zero on drift. - `ci/scripts/rust_proto_check.sh` + a new `datafusion-proto-sync-check` job in `rust.yml` that runs `--check` on every PR. - `dev/update_datafusion_versions.py` now prints a reminder to re-run the proto sync after a DataFusion bump. Notes: - No proto content changes here — on the current pin (DataFusion 53) the vendored `datafusion.proto` already matches the crate, so the check passes as-is. - `datafusion-proto-common` did not ship its `.proto` file until v54, so on 53 that file cannot be sourced from the crate. The script skips any proto a release does not ship (rather than failing) and will automatically cover `datafusion_common.proto` once the workspace moves to DataFusion 54. # Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
