Does the scope of this PR warrant an AIP? On Tue, Apr 16, 2024 at 6:40 AM Jarek Potiuk <[email protected]> wrote:
> Hello here, > > I have a kind request for help from maintainers (and other contributors who > are not maintainers) - on the Connexion 3 migration for Airflow. PR here > (unfortunately - it's one big PR and cannot be split): > https://github.com/apache/airflow/pull/39055. > > I would love some general comments on this - especially from those who are > more experts than me on those web frameworks - is it safe and ok to > migrate, do we need to do some more testing on that? What do other > maintainers think? > > This is not a "simple" change - it introduces a pretty fundamental change > in how our web app is handled - It changes from WSGI to ASGI interface > (though we use gunicorn as WSGI). But it's also absolutely needed - > because we already had some security issues connected with old > dependencies (Werkzeug) - raised - and Connexion 3 migration seems to be > the easiest way to get to the latest, maintained versions of the > dependencies. > > That's why I'd really like a few more maintainers - and people from the > Astronomer, Google and AWS to take it for a spin and help to test that > change and say "yep. It looks good, we can merge it". I would especially > appreciate some more "scale" testing on it. It seems that performance and > resource usage is not affected and ASGI interface and uvicorn should nicely > replace all the different worker types we could have for gunicorn - but I > would love to have confirmation for that. > > The PR has been started by Vlada and Maks from the Google team - and > with the help of Sudipto and Satoshi - two interns from Major League > Hacking - supported by Royal Bank of Canada - finally we have a stable, > working version and green PR. Airflow webserver + API seems to work well, > stable (and generally back-compatible) on both - development (local + > breeze) and PROD image. > > I took a mentorship and leading role on it - but personally I have been > learning on the go about WSGI/ASGI and all changes needed - I am not an > expert at all in those. We followed the directions from Connexion'\s > maintainer Robb Snyders - and I asked him to help and comment on the PR in > a number of places - but that's why I need more help and experts' eyes and > hands to be quite sure it can be safely nerged. > > I extracted it and squashed more than 100 commits on it into a single one > to make it easier to start new conversations. > > Once again PR here: https://github.com/apache/airflow/pull/39055 > > Also - we need to decide when is the best time to merge the PR - it does > not introduce a lot of changes in the code of the app, but it changes a lot > of test code to make it compatible with Startlette test client - we can > continue rebasing it and fixing new changes for a short while - but I think > the sooner we migrate it - the better - it will give more time for testing > in the future MINOR airflow version. > > J. >
