>From Guido's post: "Around this time the renaming seems to have been renamed".
Naming is hard. J. On Wed, Oct 23, 2024 at 7:11 AM Omkar P <droiddev5...@gmail.com> wrote: > Yes, completely agree with above comments, dag is an Airflow term now > rather > than just a "directed acyclic graph". Believe it or not, I've worked with > work > folks who've been using dags in Airflow for years now and are pro devs but > have trouble remembering the full form of a DAG! For them, dag = Airflow. > > workflow, pipeline, flow are surely better terms but will be a major > behavior > change for the developer community, so we'll need a solid plan on how to > introduce it, when we do. > > While we discuss this, I'd like to share about the Great Renaming in core > Python started by Guido back in 2009. We could probably get some learnings > from there? Who knows! > > Guido's blog (2009): > https://python-history.blogspot.com/2009/03/great-or-grand-renaming.html > Follow-up discussion (2024): > https://discuss.python.org/t/finishing-the-great-renaming/54082 > > Cheers, > Omkar > > On Wed, Oct 23, 2024 at 7:10 AM Wei Lee <weilee...@gmail.com> wrote: > > > I think we should probably just accept it as an airflow term. At least, > > that’s how I understood it when I first used Airflow. I feel renaming it > at > > this stage would require considerable effort from maintainers and > existing > > users without providing equivalent benefits. > > > > Best, > > Wei > > > > > On Oct 23, 2024, at 8:01 AM, Kaxil Naik <kaxiln...@gmail.com> wrote: > > > > > > Same agreed with Brent & Daniel -- maybe we re-kindle this discussion > for > > > Airflow 4 :) -- but right now it will cause too much disruption > > > > > > On Tue, 22 Oct 2024 at 21:27, Constance Martineau > > > <consta...@astronomer.io.invalid> wrote: > > > > > >> In my experience, when you ask those with Airflow experience what a > dag > > is, > > >> they'll start talking about workflow attributes - stuff like dags > being > > a > > >> series of steps or tasks with owners. The structure doesn't come up. > > >> > > >> Echo-ing others, at this point, my vote is to embrace the name and > > >> de-emphasize the mathematical structure aspect. > > >> > > >> On Tue, Oct 22, 2024 at 3:47 PM Vikram Koka > > <vik...@astronomer.io.invalid> > > >> wrote: > > >> > > >>> It's an interesting discussion and I remember struggling with this > > when I > > >>> started working with Airflow. > > >>> But, I also agree with the viewpoint of it being an established > concept > > >> now > > >>> regardless of the origin. > > >>> > > >>> I am personally leaning towards the perspective best expressed by > > Daniel > > >>> Standish and Brent of using Dag as a word, rather than the computer > > >> science > > >>> concept. > > >>> > > >>> Best regards, > > >>> Vikram > > >>> > > >>> > > >>> On Tue, Oct 22, 2024 at 9:46 AM Oliveira, Niko > > >> <oniko...@amazon.com.invalid > > >>>> > > >>> wrote: > > >>> > > >>>> I agree with the general sentiment of: You're right Ryan, DAG isn't > > >> great > > >>>> and I'd rather workflow, but changing it will cause much more > wreckage > > >>> than > > >>>> it solves. > > >>>> > > >>>> Also agree with the idea to just move away from defining DAG. I > think > > >>>> we've been naturally doing that as a community for a while now > anyway, > > >> so > > >>>> that feels like a natural step. > > >>>> > > >>>> Cheers, > > >>>> Niko > > >>>> > > >>>> ________________________________ > > >>>> From: Ash Berlin-Taylor <a...@apache.org> > > >>>> Sent: Tuesday, October 22, 2024 9:06:39 AM > > >>>> To: dev@airflow.apache.org > > >>>> Subject: RE: [EXT] Airflow should deprecate the term "DAG" for end > > >> users > > >>>> > > >>>> CAUTION: This email originated from outside of the organization. Do > > not > > >>>> click links or open attachments unless you can confirm the sender > and > > >>> know > > >>>> the content is safe. > > >>>> > > >>>> > > >>>> > > >>>> AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur > > >> externe. > > >>>> Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne > > >>> pouvez > > >>>> pas confirmer l’identité de l’expéditeur et si vous n’êtes pas > certain > > >>> que > > >>>> le contenu ne présente aucun risque. > > >>>> > > >>>> > > >>>> > > >>>> Best argument in favour of keeping “dags” as a term — getting to > > re-use > > >>>> puns like https://i.imgflip.com/1xhtwh.jpg > > >>>> > > >>>> In all seriousness: I don’t mind either way, both sides have good > > >> reasons > > >>>> presented. > > >>>> > > >>>> -a > > >>>> > > >>>>> On 22 Oct 2024, at 17:03, Daniel Standish > > >>>> <daniel.stand...@astronomer.io.INVALID> wrote: > > >>>>> > > >>>>> Yeah just say, when asked where the name comes from, "well, no one > > >>>> actually > > >>>>> knows but..." and then make something up. > > >>>>> > > >>>>> On Tue, Oct 22, 2024 at 8:31 AM Jarek Potiuk <ja...@potiuk.com> > > >> wrote: > > >>>>> > > >>>>>> Just to clarify - "directed acyclic graph" is the tongue-twister, > > >>>>>> > > >>>>>> On Tue, Oct 22, 2024 at 5:29 PM Jarek Potiuk <ja...@potiuk.com> > > >>> wrote: > > >>>>>> > > >>>>>>> I like what both Daniel and Brent wrote. I would very much want > to > > >> be > > >>>>>> able > > >>>>>>> to say just "dag" without explaining it further. > > >>>>>>> > > >>>>>>> For me every time I explain "DAG" at a talk it's a > tongue-twister, > > >>> and > > >>>> I > > >>>>>>> almost stutter on trying to recall how to pronounce it properly. > > >>>>>>> > > >>>>>>> J. > > >>>>>>> > > >>>>>>> > > >>>>>>> On Tue, Oct 22, 2024 at 5:27 PM Brent Bovenzi > > >>>>>> <br...@astronomer.io.invalid> > > >>>>>>> wrote: > > >>>>>>> > > >>>>>>>> I remember we explored renaming "DAG" when starting on AIP-38 to > > >>>>>> modernize > > >>>>>>>> the UI. Both "pipeline" or "workflow" are more descriptive of > what > > >>> one > > >>>>>> is > > >>>>>>>> actually doing while Directed Acyclic Graph is an implementation > > >>>> detail. > > >>>>>>>> But I agree with Daniel Standish, at this point "DAG" has become > > >>> "dag" > > >>>>>> , a > > >>>>>>>> word in its own right. > > >>>>>>>> > > >>>>>>>> Examples for "dag" are abound in community discussion, Airflow > > >>> Summit > > >>>>>>>> talks, documentation and even in the UI. Let's embrace "dag". A > > >> user > > >>>>>> just > > >>>>>>>> needs to learn one new word vs the technical concept behind that > > >>>> word. I > > >>>>>>>> think that is much less effort than refactoring so much code, > > >>>>>>>> documentation, blog posts, stack overflow questions, etc. > > >>>>>>>> > > >>>>>>>> On Tue, Oct 22, 2024 at 10:51 AM Daniel Standish > > >>>>>>>> <daniel.stand...@astronomer.io.invalid> wrote: > > >>>>>>>> > > >>>>>>>>> I am skeptical. Seems like introducing a lot of pain for > > >>>> questionable > > >>>>>>>>> benefit. But, I am def sympathetic to the idea. I agree the > > >>>>>>>> association > > >>>>>>>>> with "directed acyclic graph" is not helpful. > > >>>>>>>>> > > >>>>>>>>> And along those lines, I offer here some less invasive > > >> mitigations. > > >>>>>>>>> > > >>>>>>>>> One thing we can do no matter what is to de-emphasize the math > > >> nerd > > >>>>>>>> origins > > >>>>>>>>> of the name. That is to say, in docs / website / etc, *never > > >>> define* > > >>>>>>>>> airflow's "dag" concept as a directed acyclic graph. Always > > >> define > > >>>> it > > >>>>>>>> as a > > >>>>>>>>> pipeline, collection of tasks, workflow etc. > > >>>>>>>>> > > >>>>>>>>> The "directed acyclic graph" part of it is like a historical > > >>>> footnote, > > >>>>>>>> and > > >>>>>>>>> we could make one mention of it somewhere hidden. > > >>>>>>>>> > > >>>>>>>>> We could also start using lowercase in the docs in general e.g. > > >>>>>> writing > > >>>>>>>>> "dag" / "dags" instead of writing "DAG" / "DAGs" etc. The > upper > > >>> case > > >>>>>>>> part > > >>>>>>>>> of it makes it look like an acronym; but "dag" in airlfow is > just > > >>> an > > >>>>>>>>> airflow concept and the association with "DAGs" is not really > > >>>>>> unhelpful. > > >>>>>>>>> > > >>>>>>>>> In other words embrace that "dag" in airflow is its own thing, > is > > >>>>>>>>> *not* strictly > > >>>>>>>>> speaking a directed acyclic graph (which nobody knows about > > >>> anyway), > > >>>>>> and > > >>>>>>>>> tell them what it is in simple terms that normal people > > >> understand. > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> On Tue, Oct 22, 2024 at 7:27 AM Jarek Potiuk <ja...@potiuk.com > > > > >>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> DAG is so embedded into what we do that it will be extremely > > >>>>>>>> difficult to > > >>>>>>>>>> get rid of it completely. Also I think it will make a lot of > > >>>>>> "google" > > >>>>>>>>>> searches and "stack overflow" searches not finding the right > > >>>>>> answers. > > >>>>>>>>> This > > >>>>>>>>>> is one of the strengths of Airflow - besides the community and > > >>> ideas > > >>>>>>>> that > > >>>>>>>>>> Bernd mentioned - is the vast number of examples, problems and > > >>>>>>>> solutions > > >>>>>>>>>> you can so easily find (and we have to remember that all the > AI > > >>>>>>>> trained > > >>>>>>>>> on > > >>>>>>>>>> past data will be also rather poorly matching queries of > people. > > >>>>>>>>>> > > >>>>>>>>>> I am not too attached to DAG. I could easily switch. And if we > > >> do > > >>> - > > >>>>>> I > > >>>>>>>>>> would be for using workflow or pipeline instead of `dag` if > not > > >>> the > > >>>>>>>> above > > >>>>>>>>>> reason, but I think I am here with Igor that it might cause > more > > >>>>>>>> problems > > >>>>>>>>>> than it solves. > > >>>>>>>>>> > > >>>>>>>>>> But I am not 100% against - if others will think it's a good > > >>> idea, I > > >>>>>>>> am > > >>>>>>>>> ok > > >>>>>>>>>> with it. > > >>>>>>>>>> > > >>>>>>>>>> J, > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> On Tue, Oct 22, 2024 at 3:12 PM Abhishek Bhakat > > >>>>>>>>>> <abhishek.bha...@astronomer.io.invalid> wrote: > > >>>>>>>>>> > > >>>>>>>>>>> Agreed that the word DAG makes very less sense to someone new > > >> to > > >>>>>>>>> workflow > > >>>>>>>>>>> orchestration. But it does also show the nature of being > > >> acyclic. > > >>>>>>>> Sure, > > >>>>>>>>>> as > > >>>>>>>>>>> Bas mentioned, there are ways to workaround it. Still, in my > > >>>>>>>> opinion, > > >>>>>>>>>> there > > >>>>>>>>>>> is generally no need for cyclic behavior in workflow > > >>>>>> orchestration. > > >>>>>>>>> Most > > >>>>>>>>>>> (*if > > >>>>>>>>>>> not all*) cases can be in some way can be covered using an > > >>> acyclic > > >>>>>>>>> manner > > >>>>>>>>>>> with multiple runs. Hence, the idempotency. So I would want > the > > >>>>>>>>> "acyclic" > > >>>>>>>>>>> word to stick. > > >>>>>>>>>>> > > >>>>>>>>>>> Regards, > > >>>>>>>>>>> Avi > > >>>>>>>>>>> > > >>>>>>>>>>> On Tue, Oct 22, 2024 at 12:41 PM <bernd.stroe...@kosakya.de> > > >>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>>> Brilliant, I am on the way to become an Airflow Fan; so many > > >> new > > >>>>>>>>> ideas. > > >>>>>>>>>>>> > > >>>>>>>>>>>> The Term DAG is misleading; it should be replaced by the > more > > >>>>>>>> general > > >>>>>>>>>>> Term > > >>>>>>>>>>>> Airflow (Workflow) Graph (AFG) or Airflow (Petri) Net (AFN) > > >>>>>> (maybe > > >>>>>>>>>>> without > > >>>>>>>>>>>> a direction); > > >>>>>>>>>>>> and ... these Graphs should be stored in a Graph Database. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Every Node or Sup-Graph of an Airflow Graph (AFG) might be > > >>>>>>>> assigned > > >>>>>>>>> to > > >>>>>>>>>> an > > >>>>>>>>>>>> executable (Python-, Rust-, ... ) member of a library. > > >>>>>>>>>>>> > > >>>>>>>>>>>> A running Graph might have a different structure than a > > >>>>>>>> configuration > > >>>>>>>>>>>> Graph. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Forget that if you think it's bullshit. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Best Regards > > >>>>>>>>>>>> > > >>>>>>>>>>>> Bernd Ströhle > > >>>>>>>>>>>> M: +49 171 5357916 > > >>>>>>>>>>>> E: bernd.stroe...@gmail.com > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> -----Original Message----- > > >>>>>>>>>>>> From: Igor Kholopov <ikholo...@google.com.INVALID> > > >>>>>>>>>>>> Sent: Tuesday, October 22, 2024 12:02 PM > > >>>>>>>>>>>> To: dev@airflow.apache.org > > >>>>>>>>>>>> Subject: Re: Airflow should deprecate the term "DAG" for end > > >>>>>> users > > >>>>>>>>>>>> > > >>>>>>>>>>>> Even though the term "DAG" is clearly suboptimal, it is part > > >> of > > >>>>>>>>> Airflow > > >>>>>>>>>>>> DAG definition interface at so many levels, that any attempt > > >> to > > >>>>>>>>> change > > >>>>>>>>>> it > > >>>>>>>>>>>> will only introduce more chaos, not reduce it. The only > thing > > >>>>>>>> that is > > >>>>>>>>>>> worse > > >>>>>>>>>>>> than a poorly chosen name in the code is when there are two > > >> ways > > >>>>>>>> to > > >>>>>>>>>>> define > > >>>>>>>>>>>> the same thing. Countless articles and tutorials will > suddenly > > >>>>>>>> become > > >>>>>>>>>>>> confusing as they all refer to workflows as "DAG"s. > > >>>>>>>>>>>> > > >>>>>>>>>>>> We are already at risk of scaring the users away with a > number > > >>>>>> of > > >>>>>>>>>>> breaking > > >>>>>>>>>>>> changes in Airflow 3, promising even more breaking changes > for > > >>>>>> the > > >>>>>>>>> most > > >>>>>>>>>>>> basic things is not something that people are looking for. > > >>>>>>>> Attempting > > >>>>>>>>>> to > > >>>>>>>>>>>> change the fundamental terms will be interpreted as an even > > >>>>>>>> stronger > > >>>>>>>>>>> signal > > >>>>>>>>>>>> of project immaturity. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Given that, I oppose the idea of changing the term in the > long > > >>>>>>>> run. I > > >>>>>>>>>>> even > > >>>>>>>>>>>> stricter oppose the idea of deprecating it in the DAG > > >> definition > > >>>>>>>>>>> interface. > > >>>>>>>>>>>> We better put our time and efforts in other places in > Airflow, > > >>>>>> of > > >>>>>>>>> which > > >>>>>>>>>>>> there are plenty. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Kind regards, > > >>>>>>>>>>>> Igor > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Tue, Oct 22, 2024 at 10:36 AM Bas Harenslak > > >>>>>>>>>> <b...@astronomer.io.invalid > > >>>>>>>>>>>> > > >>>>>>>>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>>> Couple of thoughts: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> 1. The boundaries/properties of “DAG” have already faded > over > > >>>>>>>> time, > > >>>>>>>>>>>>> for example there are now several ways to create cyclic > > >>>>>> graphs, > > >>>>>>>>> e.g. > > >>>>>>>>>>>>> using the @continuous schedule. I imagine these properties > > >>>>>>>>> vanishing > > >>>>>>>>>>>>> even more in the future, so from that perspective I support > > >>>>>>>>> changing > > >>>>>>>>>>>>> “DAG" to a more generic name. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> 2. How other orchestration frameworks do naming: > > >>>>>>>>>>>>> Dagster: pipeline > > >>>>>>>>>>>>> Prefect: flow > > >>>>>>>>>>>>> Flyte: workflow > > >>>>>>>>>>>>> Temporal: workflow > > >>>>>>>>>>>>> Kestra: flow > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> I think “workflow” is the most fitting name. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> 3. Given the large impact of this change, I suggest > defining > > >> a > > >>>>>>>>> clear > > >>>>>>>>>>>>> path forward. Would we first introduce the deprecation in > > >>>>>>>> Airflow > > >>>>>>>>> 3, > > >>>>>>>>>>>>> and remove “DAG” in Airflow 4? > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Bas > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> On 22 Oct 2024, at 09:22, Neil <neil4r...@gmail.com> > wrote: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> I don't see a problem with the term DAG, especially when > > >>>>>> most > > >>>>>>>>> other > > >>>>>>>>>>>>>> platforms embrace the term wholeheartedly. > > >>>>>>>>>>>>>> I don't see anything intimidating or confusing about it at > > >>>>>>>> all, > > >>>>>>>>>>>>>> changing the term though would be fairly confusing to most > > >>>>>>>> users > > >>>>>>>>>> who > > >>>>>>>>>>>>>> have been > > >>>>>>>>>>>>> using > > >>>>>>>>>>>>>> the term for years. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> On Tue, Oct 22, 2024 at 1:18 AM Tzu-ping Chung > > >>>>>>>>>>>>>> <t...@astronomer.io.invalid > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> I totally agree with doing away with the term DAG. The > only > > >>>>>>>>>> problem > > >>>>>>>>>>>>> (aside > > >>>>>>>>>>>>>>> from actually telling people—including myself—to stop > using > > >>>>>>>> the > > >>>>>>>>>>>>>>> term) > > >>>>>>>>>>>>> is to > > >>>>>>>>>>>>>>> come up with a reasonable alternative. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> I can’t recall who, but someone mentioned “workflow” is > not > > >>>>>>>> very > > >>>>>>>>>>>>> accurate > > >>>>>>>>>>>>>>> for Airflow. The term “definition” was proposed, but > it’s a > > >>>>>>>> bit > > >>>>>>>>>>>>>>> broad; I tried to use it in a few places and kept finding > > >>>>>>>> myself > > >>>>>>>>>>>>>>> doubting “what definition?” and wanting to clarify “DAG > > >>>>>>>>>> definition” > > >>>>>>>>>>>>>>> (defeating the purpose). > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> TP > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> On 22 Oct 2024, at 13:07, Jens Scheffler > > >>>>>>>>>>>>>>>> <j_scheff...@gmx.de.INVALID> > > >>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Hi Ryan, > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Thanks for posting. I share the exactly same > observation, > > >>>>>>>> had a > > >>>>>>>>>>>>>>>> short > > >>>>>>>>>>>>>>> laight because the DAG question is always an introduction > > >>>>>> if > > >>>>>>>>>>>>>>> someone > > >>>>>>>>>>>>> joins > > >>>>>>>>>>>>>>> the party. I think a global renaming makes sense. > > >>>>>> Especially > > >>>>>>>>> when > > >>>>>>>>>>>>>>> we > > >>>>>>>>>>>>> also > > >>>>>>>>>>>>>>> rename Dataset to Asset this is also a reasonable step. > > >>>>>>>> Concepts > > >>>>>>>>>>>>>>> still > > >>>>>>>>>>>>> can > > >>>>>>>>>>>>>>> stay the same. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> So I hope I don‘t need to join hiding below the desk > with > > >>>>>>>> you > > >>>>>>>>> and > > >>>>>>>>>>>>>>>> +1 > > >>>>>>>>>>>>> for > > >>>>>>>>>>>>>>> raising the discussion. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Technically we can still think if we keep details of > > >>>>>> python > > >>>>>>>>> names > > >>>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>> same because the execution is still a DAG… but user > facing > > >>>>>> it > > >>>>>>>>> is a > > >>>>>>>>>>>>> workflow. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Jens > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Sent from my Smartphone > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> On 21. Oct 2024, at 23:56, Ryan Hatter < > > >>>>>>>>>> ryan.hat...@astronomer.io > > >>>>>>>>>>>>> .invalid> > > >>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> Everyone please sheathe your swords... at least for > now. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> The term "DAG" has very little meaning to Airflow > users. > > >>>>>>>>> Indeed, > > >>>>>>>>>>>>>>>>> it > > >>>>>>>>>>>>> has > > >>>>>>>>>>>>>>>>> little meaning outside of some mathematicians and > > >>>>>> software > > >>>>>>>>>>>>>>>>> engineers > > >>>>>>>>>>>>> for > > >>>>>>>>>>>>>>>>> whom the properties of a DAG actually matter. For > someone > > >>>>>>>> new > > >>>>>>>>> to > > >>>>>>>>>>>>>>>>> data engineering or workflow orchestration, one of the > > >>>>>>>> first > > >>>>>>>>>>>>>>>>> questions they > > >>>>>>>>>>>>>>> will > > >>>>>>>>>>>>>>>>> likely have is, "what on earth is a DAG?" The answer is > > >>>>>>>> almost > > >>>>>>>>>>>>>>>>> always, "It's a directed acyclic graph. You don't need > to > > >>>>>>>>> worry > > >>>>>>>>>>>>>>>>> about what > > >>>>>>>>>>>>> that > > >>>>>>>>>>>>>>>>> means; it's just a term for your workflow." > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> The term "DAG" is problematic for at least a couple > > >>>>>>>> important > > >>>>>>>>>>>> reasons: > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> *Complexity for New Users*: As mentioned above, "DAG" > is > > >>>>>>>>>>>>>>>>> unnecessarily intimidating and confusing. We want > Airflow > > >>>>>>>> to > > >>>>>>>>> be > > >>>>>>>>>>>>>>>>> approachable, and > > >>>>>>>>>>>>>>> using > > >>>>>>>>>>>>>>>>> technical jargon like "DAG" right off the bat creates > an > > >>>>>>>>> initial > > >>>>>>>>>>>>>>> barrier to > > >>>>>>>>>>>>>>>>> understanding. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> *Disconnect Between DAG and Workflow Concepts*: The DAG > > >>>>>> is > > >>>>>>>>> just > > >>>>>>>>>>>>>>>>> one component of an Airflow workflow. The workflow > > >>>>>> includes > > >>>>>>>>> its > > >>>>>>>>>>>>>>>>> schedule, retries, timeouts, a dozen other parameters, > > >>>>>> and > > >>>>>>>>> other > > >>>>>>>>>>>>>>>>> metadata that > > >>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>> DAG component doesn’t account for. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> Consider the following from the Airflow homepage > > >>>>>>>>>>>>>>>>> <https://airflow.apache.org/>. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> Apache Airflow® is a platform created by the community > to > > >>>>>>>>>>>>>>> programmatically > > >>>>>>>>>>>>>>>>> author, schedule and monitor workflows. > > >>>>>>>>>>>>>>>>> Then, if we look at the "What is Airflow?" docs page > > >>>>>>>>>>>>>>>>> < > > >>>>>>>>>> > > >> https://airflow.apache.org/docs/apache-airflow/stable/index.html > > >>>>>>>>>>>>>>>>>> , > > >>>>>>>>>>>>> we > > >>>>>>>>>>>>>>> can > > >>>>>>>>>>>>>>>>> see that the docs explain what Airflow is without using > > >>>>>>>> "DAG." > > >>>>>>>>>>>>>>>>> It's > > >>>>>>>>>>>>>>> only in > > >>>>>>>>>>>>>>>>> the *workflow* Python code that the term is introduced > > >>>>>> out > > >>>>>>>> of > > >>>>>>>>>>>>>>>>> nowhere > > >>>>>>>>>>>>>>> as a > > >>>>>>>>>>>>>>>>> comment that awkwardly tries to explain it: > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> # A DAG represents a workflow, a collection of tasks > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> It makes sense to not refer to DAGs in these > > >>>>>> introductions > > >>>>>>>> to > > >>>>>>>>>>>>>>>>> Airflow, because *Airflow doesn't orchestrate DAGs; it > > >>>>>>>>>>> orchestrates > > >>>>>>>>>>>> workflows*. > > >>>>>>>>>>>>>>> The > > >>>>>>>>>>>>>>>>> DAG is the model that, for reasons irrelevant to almost > > >>>>>>>> every > > >>>>>>>>>>>>>>>>> user, workflows must adhere to. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> So, I propose at least adding an alias for the term > "DAG" > > >>>>>>>> and > > >>>>>>>>>>>>>>>>> updating documentation to replace "DAG" with > "workflow". > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> For example, instead of... > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> @dag( > > >>>>>>>>>>>>>>>>> schedule="@daily", > > >>>>>>>>>>>>>>>>> ... > > >>>>>>>>>>>>>>>>> dagrun_timeout=timedelta(hours=1) > > >>>>>>>>>>>>>>>>> ) > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> Users could do... > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> @workflow( > > >>>>>>>>>>>>>>>>> schedule="@daily", > > >>>>>>>>>>>>>>>>> ... > > >>>>>>>>>>>>>>>>> run_timeout=timedelta(hours=1) > > >>>>>>>>>>>>>>>>> ) > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> And with that... I will start running away. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>> > > >> ------------------------------------------------------------------ > > >>>>>>>>>>>>>>>> --- To unsubscribe, e-mail: > > >>>>>>>> dev-unsubscr...@airflow.apache.org > > >>>>>>>>>>>>>>>> For additional commands, e-mail: > > >>>>>>>> dev-h...@airflow.apache.org > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>> > > >>> ------------------------------------------------------------------- > > >>>>>>>>>>>>>>> -- To unsubscribe, e-mail: > > >>>>>>>> dev-unsubscr...@airflow.apache.org > > >>>>>>>>>>>>>>> For additional commands, e-mail: > > >>>>>> dev-h...@airflow.apache.org > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>> > > >>> --------------------------------------------------------------------- > > >>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > >>>>>>>>>>>> For additional commands, e-mail: > dev-h...@airflow.apache.org > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>> > > >>>> > > >>> > > >> > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > >
