Example dags are one place we have historically put dags / tasks that go in docs but are not run as system tests. Is that not still the case?
On Mon, Aug 19, 2024 at 9:01 AM Ferruzzi, Dennis <ferru...@amazon.com.invalid> wrote: > Another part I had not considered is that since it's part of the hot-loop, > then a user-defined callable could very possibly bring the whole thing to a > grinding stop, either intentionally or not. So yeah, I guess without > massive work to somehow decouple that so the whole scheduler loop didn't > freeze waiting for a callable trigger rule, that could be a very bad idea. > > > - ferruzzi > > > ________________________________ > From: Jarek Potiuk <ja...@potiuk.com> > Sent: Sunday, August 18, 2024 2:46 PM > To: dev@airflow.apache.org > Subject: RE: [EXT] [Discussion] Add a new TriggerRule: Never > > CAUTION: This email originated from outside of the organization. Do not > click links or open attachments unless you can confirm the sender and know > the content is safe. > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. > Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez > pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que > le contenu ne présente aucun risque. > > > > While I think "system" tests use case is indeed - as Daniel wrote - not a > good reason, I find it intriguing as a "commenting out" things from DAG. > Sounds like a useful authoring feature to have a "never" trigger rule. > > The "callable" use case is likely a bit too much - we've been discussing > "flexible" triggering rule, but since they are part of the scheduler > hot-loop, it's likely not a good idea to make them "editable" by the users, > and certainly not a good idea to have the code on trigger rules controlled > by DAG author (it would mean that code submitted by DAG author would be > executable in scheduler context) - we could make it possible via plugins, > but IMHO having freely configurable triggering rules is quite a bit too > much of a freedom.. > > J. > > > On Fri, Aug 16, 2024 at 11:44 PM Daniel Standish > <daniel.stand...@astronomer.io.invalid> wrote: > > > It doesn't really sound right to include the task in the system test dag, > > purely for the purpose of getting it into the docs. Why just put it in > the > > docs as an inline example? > > > > If you want to conditionally run certain tasks, one option is to raise > > AirflowSkipException > > >
