Thanks for looking and analysis :). Very correct. However this is a special case actually. It only builds it that long because we have a change in this branch that requires rebuilding more layers of the image. In this case we add mysql dependencies in a different way and we rebuild the whole stage of the docker image.
Once it gets merged to master and DockerHub image is built, this step will be skipped as we will pull the pre-built image from dockerhub (via Docker caching) and Docker rebuilding only source layers will be way faster. Typical case is more like this one: https://travis-ci.org/apache/airflow/jobs/561963482 where the docker pull&build step takes 1:40 seconds rather than 5 minutes and the steps takes less than 3 minutes rather than 5-8 minutes. But indeed one of changes that we are going to implement soon is going to pre-commit hooks and moving (most) of the static checks to single job: POC here: https://travis-ci.org/PolideaInternal/airflow/builds/555443375?utm_source=github_status&utm_medium=notification. In this case most of the static checks take all of less than 5 minutes in one job. In the POC we have separate pylint job and check licence job but we will be able to put it in two separate jobs after some recent changes. So that optimisation is also in progress (but it needs a bit more work :D ). J. On Mon, Jul 22, 2019 at 11:45 AM Vladimir Sitnikov < sitnikov.vladi...@gmail.com> wrote: > Jarek>Decreasing it to 5 (which I believe is the case) is terrible for us > now > Jarek>We are nearing 1.10.4 release with Airflow and if we have more than > one PR > Jarek>in the queue it waits for several hours to run! > > I see 5 "pre-test" Travis jobs take 5-8 minutes each, and all the time is > basically spent in "building airflow image". > Have you considered merging all the jobs to one? > > For instance: > > https://travis-ci.org/apache/airflow/jobs/561927828 > The job took 2m20sec, and the payload was as follows: > > Finished the script ci_check_license.sh > > It took 2 seconds > > It is not like it would dramatically reduce Airflow's impact on Travis, > however it would free up executors with little to no drawbacks. > > Vladimir > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
