potiuk commented on a change in pull request #17964:
URL: https://github.com/apache/airflow/pull/17964#discussion_r700906397
##########
File path: scripts/ci/selective_ci_checks.sh
##########
@@ -353,8 +351,9 @@ function check_if_setup_files_changed() {
if [[ $(count_changed_files) != "0" ]]; then
# In case the setup files changed, we automatically force upgrading to
newer dependencies
- # no matter what was set before
- upgrade_to_newer_dependencies="${INCOMING_COMMIT_SHA}"
+ # no matter what was set before. We set it to random number to make
sure that it will be
+ # always invalidating the layer in Docker that triggers installing the
dependencies
+ upgrade_to_newer_dependencies="${RANDOM}"
Review comment:
Asking for RANDOM ?
The reason is that we need to make sure that the right layer is invalidated
when the Dockerfile is rebuilt in "eager upgrade" even if the `setup.py` does
not change (when we build it in `main`).
I just updated the description (see the latest version) where it was
described in detail (see the changes in CI.rst).
I used `commit SHA` before, but the problem is that we have different
commits in `main` and `pr` kind of builds (in Github actions they are passed in
different variables in this case). But RANDOM will do the same thing. What we
want here we want to invalidate everything in the image from that place where
ARG UPGRADE_TO_NEWER_DEPENDENCIES is used.
Everything below that line should be invalidated - regardless from
`setup.py`.
https://github.com/apache/airflow/blob/7c4d3173f2c73ceeadb7155cf367983a90c6dca8/Dockerfile.ci#L282
It could `technically` be just `true` (and it was initially actually), but
having RANDOM here makes it more "certain" that it will be invalidated in case
we build the image with a cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
