Yep having pre-commits is cool and we extensively use it as part of our setup in Airflow. Since we are heavily Pythonic project we are using the fantastic https://pre-commit.com/ framework. We have ~ 60 fast pre-commit checks that are run both as pre-commit hooks for developers and as CI verification steps: https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#pre-commit-hooks
Speaking of the speed and job starvation. We continued to experience it today and we decided to combine back our jobs to get much smaller number of them. We've done it smartly = reusing the build machines for several different tests and cleaning the state in between and running several "smaller" chunks of tests sequentially. Since we are using docker and docker-compose, this is all easily doable. PR here for those interested: https://github.com/apache/airflow/pull/11504 On Tue, Oct 13, 2020 at 5:23 PM Allen Wittenauer <a...@effectivemachines.com.invalid> wrote: > > > On Oct 13, 2020, at 5:00 AM, Jarek Potiuk <jarek.pot...@polidea.com> > wrote: > > Who else is using GitHub Actions extensively? > > > It's funny you mention that. A lot of the stress on the ASF > Jenkins instance was helped tremendously by deploying Apache Yetus. It > might be useful for some users on GitHub Actions to take a look at using > our current top of tree to see if it would be helpful for them as well as > to help us test it out before we publish to the GitHub Marketplace as part > of our next release. (Yetus becoming the first ... and maybe only? ... > Apache project to publish there.) > > Web pages of interest: > > * Base page of Apache Yetus' testing facilities: > http://yetus.apache.org/documentation/in-progress/precommit/ > > * The specific page about GitHub Actions: > http://yetus.apache.org/documentation/in-progress/precommit/robots/githubactions/ > > Note that the _default_ is pure static linting to make the curve a > bit easier. To get actual builds, ASF license checks, and a few other > things turned on, you'll need to set the build tool and likely provide the > list of plug-ins one wants to turn on. The list of plug-ins available is > on the bottom of that first page. > > If you have any questions, let us know! Thanks. -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
