2010YOUY01 opened a new issue, #24487: URL: https://github.com/apache/datafusion/issues/24487
### Is your feature request related to a problem or challenge? Part of https://github.com/apache/datafusion/issues/21048 Follows-up: https://github.com/apache/datafusion/pull/24083 ### Describe the solution you'd like #24083 introduces `xtask` to implement individual test commands, including setup steps such as environment variables. This allows both local and CI runs to use the same commands, making local reproduction easier and ensuring it is equivalent to CI. See the issue write-up for how it works. It is essentially just a simple binary within the project; `xtask` is the conventional name for this pattern. For example, the default test run command is: https://github.com/apache/datafusion/blob/26fbf4d015d7071115c5fdadf1156291d8a2afb1/.github/workflows/rust.yml#L324-L335 Now both ci and local run can be executed with ``` # Use `cargo xtask help` for the full usage # syntax: cargo xtask ci step [step-name] [extra-args] > cargo xtask ci step test workspace (...run tests) # Use 'explain' to show the underlying command > cargo xtask ci step test workspace --explain cd /Users/yongting/Code/datafusion2/datafusion-ci-steps-xtask && \ cargo llvm-cov \ --profile ci \ --exclude datafusion-examples \ --exclude ffi_example_table_provider \ --exclude datafusion-cli \ --workspace \ --lib \ --tests \ --bins \ --features serde,avro,json,backtrace,integration-tests,parquet_encryption,substrait \ --codecov \ --output-path target/codecov.json ``` Next steps: Using this `rust` workflow as reference, it is organized as: `workflow(entire file) -> job (several commands) -> step (command)` Link: https://github.com/apache/datafusion/blob/main/.github/workflows/rust.yml - [ ] Introduce `xtask` orchestration for CI jobs/workflows, such as `cargo xtask ci job rust`, to simplify reproducing the full CI job/workflows locally. - [ ] Fully migrate the `rust.yml` workflow so it is easy to reproduce locally. - [ ] ...and then roll this out to other CI workflows. ### Describe alternatives you've considered _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
