Nice, I know there are plenty of folks looking for SQLA 2 support with Airflow :)
On Tue, 12 Aug 2025 at 07:03, Amogh Desai <amoghde...@apache.org> wrote: > Good collaboration Iliya and Jarek! > > Thanks & Regards, > Amogh Desai > > > On Tue, Aug 12, 2025 at 12:26 AM Shahar Epstein <sha...@apache.org> wrote: > > > Well done Jarek and Iliya :) > > > > > > Shahar > > > > On Mon, Aug 11, 2025 at 5:42 PM Pavankumar Gopidesu < > > gopidesupa...@gmail.com> > > wrote: > > > > > Woohoo great work both. That's indeed a lot of effort :) > > > > > > Pavan > > > > > > > > > On Mon, Aug 11, 2025 at 6:03 AM Dev iL <gid....@gmail.com> wrote: > > > > > > > Thanks for the shout-out, and your patient guidance throughout this > > > > process, Jarek :) > > > > > > > > At multiple steps along the way this PR grew so large that parts of > it > > > had > > > > to be split out into "sub-PRs" - so in addition to the main one, > there > > > were > > > > also 6 sub-PRs by yours truly and another one by Jarek. > > > > > > > > One of the (currently undocumented) additions in this collection of > PRs > > > is > > > > a new environment variable, SQLALCHEMY_ENGINE_DEBUG, which when set, > > > > enables verbose logging for sqlalchemy, as well as dumping a snapshot > > of > > > > the callstack once every few minutes (the delay is only configurable > > via > > > > hard-coding currently). This was particularly helpful in diagnosing > db > > > > hangs/timeouts. Another PR reconfigured some unit tests, which were > > > > asserting things about text appearing in stdout/stderr, to ignore > > logging > > > > prints that come from the aforementioned verbosity - so that tests > > > doesn't > > > > fail due to the extra logs. As a side note, while this capability was > > > made > > > > with the CI in mind, I didn't get a chance to try it there, so YMMV. > > > > > > > > Iliya > > > > > > > > P.S. > > > > Since we now support SQLA2 in at least one configuration, my next > > project > > > > is adding psycopg v3 support. This is easier than I envisioned > > initially, > > > > since v3 can be installed alongside v2 due to having a different > > package > > > > name. We'll see how it goes... > > > > > > > > > > > > On Mon, 11 Aug 2025, 0:16 Jarek Potiuk, <ja...@potiuk.com> wrote: > > > > > > > > > Thanks to Herculean work of Ilya - and with a little bit of my help > > we > > > > now > > > > > have SQLALchemy 2 tests running in our "canary builds" now. > > > > > > > > > > Currently (until FAB 5) they only run in "canary" builds and only > in > > > > Python > > > > > 3,13 (because that's where we have no FAB) but it will allow us to > > see > > > if > > > > > we do not add anything "not working" for sqlalchemy 2. > > > > > > > > > > That was a bit of a difficult one - because SQLA2 hanged with MySQL > > for > > > > our > > > > > migrations, in a strange way that did not allow us to easily > inspect > > > the > > > > > problem - we've worked together with Ilya and I added wya better > > > > > diagnostics for this case - yet eventually Ilya solved it by adding > > > > > autocommit while running the migration - which seemed to help :). > > > > > > > > > > More to come soon when we get to FAB 5 upgrade - we might be able > to > > > > > migrate to sqlalchemy 2 for 3.1 (keep fingers crossed). FAB 5 > alpha 9 > > > has > > > > > been recently released by daniel and we are about to take it for a > > > spin. > > > > > > > > > > J. > > > > > > > > > > > > > > >
