FWIW, I'm liking the Breeze idea. I hadn't thought of it at first, but it seems to make the most sense.
- ferruzzi ________________________________ From: Beck, Vincent <vincb...@amazon.com.INVALID> Sent: Wednesday, June 7, 2023 7:21 AM To: dev@airflow.apache.org Subject: RE: [EXTERNAL]Deferrable operators in system tests, how? CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Same here, I like the breeze option On 2023-06-07, 3:32 AM, "Pankaj Koti" <pankaj.k...@astronomer.io.inva <mailto:pankaj.k...@astronomer.io.inva>LID> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. +1 to both ideas. I have a stronger +1 to include the flag with breeze since triggerer is an Airflow component and breeze commands having support for it would be nice. Regards, Pankaj Koti *Senior Software Engineer, *OSS Engineering Team. Location: Pune, India Timezone: Indian Standard Time (IST) Email: pankaj.k...@astronomer.io <mailto:pankaj.k...@astronomer.io> Mobile: +91 9730079985 On Wed, Jun 7, 2023 at 5:07 AM Daniel Standish <daniel.stand...@astronomer.io.inva <mailto:daniel.stand...@astronomer.io.inva>lid> wrote: > Don't think fixture would break that. It would just be test code not in the > dag. It would just ensure that the triggerer is running before the tests > that use the triggerer need it. But doing it in breeze makes more sense for > sure. Although I suppose a combination approach could be considered EG, if > not in the breeze environment, and no triggerer running in any case, then > spin one up. But that might be overkill and over complicating it. > > On Tue, Jun 6, 2023, 4:24 PM Vandon, Raphael <vand...@amazon.com.inva > <mailto:vand...@amazon.com.inva>lid> > wrote: > > > Thanks for pointing out solutions I hadn't considered. > > > > > we could simply have a pytest fixture that will do the job (based on > > what Daniel proposed). > > > It will not support running tests via Airflow UI or regular python > > execution of the test files, > > > but maybe it's a good idea that does not rely on docker/breeze. > > > > I like the idea that example dags are actual dags, that you can drop in > > your dags folder and run (provided you have the right config). > > As you mention, adding a pytest fixture to run the triggerer will > probably > > break that ? > > > > It'd also break the setup Phani described __, if I understood correctly ? > > > > >> Our approach is to spin up Airflow , install the providers that > > currently have deferrable functionality, > > >> and run system tests on those example DAGs > > > > So I'd be in favor of adding a flag to breeze to start a triggerer. It's > > the most flexible option, lets people run tests in full airflow, in > breeze, > > or outside of breeze by running the triggerer themselves. > > > > It also sounds like it wouldn't be too hard to implement, even though I'm > > not familiar with breeze code. I can try to give it a shot later this > week > > or the next. > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > <mailto:dev-unsubscr...@airflow.apache.org> > > For additional commands, e-mail: dev-h...@airflow.apache.org > > <mailto:dev-h...@airflow.apache.org> > > > Т���������������������������������������������������������������������ХF�V�7V'67&�&R�R���âFWb�V�7V'67&�&T�&f��r�6�R��&pФf�"FF�F����6����G2�R���âFWbֆV��&f��r�6�R��&p
