This is a great post and great approach from the Superset team. FYI. In Airflow we are working on enabling the very same approach that we did for AWS in GCP, with a slightly different approach when it comes to auto-scaling (a bit simpler I think). We will share it when we finish.
J. On Mon, May 3, 2021 at 10:05 PM Rob DiCiuccio <rob.diciuc...@gmail.com> wrote: > Hi all, > > I was asked to share details of our setup for ephemeral builds and how > we're leveraging GitHub Actions and AWS Fargate to create manual test > environments on pull requests in Apache Superset. I wrote this up in a blog > post last week and I'm happy to answer any questions about the approach: > > https://preset.io/blog/2021-4-28-ephemeral-test-environments-in-superset/ > > One thing not mentioned in the blog post is the current pain around GitHub > Actions delays making this feature unusable at times. I'm aware of multiple > discussions already going on around this issue. We are looking into setting > up self-hosted runners, similar to Airflow's approach, but we're still > investigating our options and usage patterns. > > Cheers, > Rob > -- +48 660 796 129
