Hah, the naming problem! First, regarding the term "Agent" - while it's a common and fitting term in monitoring solutions (e.g., OpenTelemetry's agent collector), I think it's more suitable as a replacement for "worker" rather than for naming the executor.
Re: Agent - I like this term and it is very common in monitoring solutions, even OTel uses it for agent collector. That said, it is a right term to replace "worker", not for naming the executor. I hear the arguments against "remote" and "distributed", but they both are quite applicable here, so still keeping them in my options. Here are my options for the Executor-Worker pair in order of preference: 1. External Executor - Remote Worker (yeaa, "external" is too broad as well. However, to me, it conveys the meaning and doesn't conflict with existing concepts in Airflow) 2. Edge Executor - Edge/Remote Worker (I do understand that "edge execution" may not be the only use case, but it effectively conveys the meaning) 3. Distributed Executor - Proxy/Remote Worker Thanks Shubham On 2024-08-27, 1:22 PM, "Ash Berlin-Taylor" <a...@apache.org <mailto:a...@apache.org>> wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain que le contenu ne présente aucun risque. Control-M at least uses this term <https://documents.bmc.com/supportu/controlm-saas/en-US/Documentation/Agent_Installation.htm> <https://documents.bmc.com/supportu/controlm-saas/en-US/Documentation/Agent_Installation.htm>> > The Agent installation installs Control-M/Agent. Additional Agents enable you > to run jobs on multiple computers. This enhances performance and creates > greater load balancing control. Agent is also used by things like DataDog <https://docs.datadoghq.com/agent/?tab=Linux> <https://docs.datadoghq.com/agent/?tab=Linux>> > The Datadog Agent is software that runs on your hosts. I'm sure they're are more in similar vein. The common thread between them is that it's something you install onto existing node to add capabilities to, which to my mind fits with AIP-69 GitHub (Actions) calls them runners <https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners> <https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners>> (which already means something in Airflow but I'm not opposed to retiring/replacing that use as it's not very commonly changed from the default) I don't much mind one way or another, I'm equally okay with Remote Worker too -ash On 27 August 2024 20:55:10 BST, "Scheffler Jens (XC-AS/EAE-ADA-T)" <jens.scheff...@de.bosch.com.inva <mailto:jens.scheff...@de.bosch.com.inva>LID> wrote: >I dislike agent. -0.8 binding from me 😀 > >Agent sounds like another function, as a term does not have anything in >relation to „worker/executor“. Agent as a term tells me the thing is very >independent and runs on higher level targets. But the remote/distributed >worker will just do what the scheduler instructs to do. Controlled from >„remote“/central site… > >Sent from Outlook for iOS<https://aka.ms/o0ukef> <https://aka.ms/o0ukef>> >________________________________ >From: Jarek Potiuk <ja...@potiuk.com <mailto:ja...@potiuk.com>> >Sent: Tuesday, August 27, 2024 7:05:29 PM >To: dev@airflow.apache.org <mailto:dev@airflow.apache.org> ><dev@airflow.apache.org <mailto:dev@airflow.apache.org>> >Subject: Re: [DISCUSS] Name for the Executor of AIP-69 > >agent is fine. > >On Tue, Aug 27, 2024 at 6:27 PM Ash Berlin-Taylor <a...@apache.org ><mailto:a...@apache.org>> wrote: > >> I like agent too >> >> On 27 August 2024 16:44:40 BST, Daniel Standish >> <daniel.stand...@astronomer.io.inva >> <mailto:daniel.stand...@astronomer.io.inva>LID> wrote: >> >If we're looking for alternatives, does the word "agent" work? >> > >> >On Tue, Aug 27, 2024 at 8:41 AM Daniel Standish < >> >daniel.stand...@astronomer.io <mailto:daniel.stand...@astronomer.io>> wrote: >> > >> >> Personally I don't mind remote executor. We need to keep in mind that >> >> this is not just naming the executor but the provider and all the other >> >> related objects. >> >> >> >> E.g. in #41729 you have these objects >> >> >> >> from airflow.providers.remote.models.remote_job import RemoteJob >> >>> from airflow.providers.remote.models.remote_logs import RemoteLogs >> >>> from airflow.providers.remote.models.remote_worker import >> RemoteWorker >> >> >> >> >> >> I sorta feel the reverse that distributed is better way to describe in >> >> docs and remote is maybe better for this executor-worker system. >> >> >> >> Distributed is a mode of execution that would apply to both remote >> >> executor and celery and k8s executor and ecs. Remote tells you >> something >> >> specific about the way it is distributed. E.g. it's in a far off >> corner of >> >> your corporate vpn :) >> >> >> >> I also don't feel super persuaded by the notion that remote is too >> broadly >> >> used and understood by the community. I personally did not know that >> our >> >> distributed executor systems were described this way in the docs until >> it >> >> was brought to the attention of the list. >> >> >> >> >> >> On Tue, Aug 27, 2024 at 8:29 AM Oliveira, Niko >> <oniko...@amazon.com.inva <mailto:oniko...@amazon.com.inva>lid> >> >> wrote: >> >> >> >>> As you'd expect Jens I also vote +1 for distributed :) >> >>> >> >>> Remote as it means today is not only in our docs but in our previous >> >>> summit talks, town halls, Github Issues/Discussions, Slack threads, >> etc. >> >>> It's a term we've used as a community for years. So we should not >> change >> >>> all that under the feet of our users just because it is difficult to >> find a >> >>> new name for AIP-69. I think distributed is good for this new feature, >> but >> >>> I am also curious to see if anyone has other proposals. >> >>> >> >>> ________________________________ >> >>> From: Vincent Beck <vincb...@apache.org <mailto:vincb...@apache.org>> >> >>> Sent: Tuesday, August 27, 2024 7:53:27 AM >> >>> To: dev@airflow.apache.org <mailto:dev@airflow.apache.org> >> >>> Subject: RE: [EXT] [DISCUSS] Name for the Executor of AIP-69 >> >>> >> >>> CAUTION: This email originated from outside of the organization. Do not >> >>> click links or open attachments unless you can confirm the sender and >> know >> >>> the content is safe. >> >>> >> >>> >> >>> >> >>> AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur >> externe. >> >>> Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si vous ne >> pouvez >> >>> pas confirmer l’identité de l’expéditeur et si vous n’êtes pas certain >> que >> >>> le contenu ne présente aucun risque. >> >>> >> >>> >> >>> >> >>> Distributed makes more sense to me. +1 binding on this one. Remote >> >>> executor is too broad to me >> >>> >> >>> On 2024/08/27 14:43:33 Jarek Potiuk wrote: >> >>> > I prefer distributed - as Remote executor is already a >> well-established >> >>> > term. But this is -0.5 on "remote" - if others think it's ok, I am >> fine. >> >>> > >> >>> > On Tue, Aug 27, 2024 at 3:33 PM Scheffler Jens (XC-AS/EAE-ADA-T) >> >>> > <jens.scheff...@de.bosch.com.inva >> >>> > <mailto:jens.scheff...@de.bosch.com.inva>lid> wrote: >> >>> > >> >>> > > Hi Airflow-Devs, >> >>> > > >> >>> > > AIP-69 has come as MVP to the front-door of the repo in form of 3 >> >>> PRs. But >> >>> > > as in the Voting there has been a bit of discussion about how we >> call >> >>> that >> >>> > > "baby" I'd like to have a wrap-up about the name and see what the >> >>> majority >> >>> > > is for. >> >>> > > >> >>> > > The AIP-69 defined the implementation as "Remote Executor" but the >> >>> exact >> >>> > > term is used in the Airflow docs today for all non-local >> executors. It >> >>> > > might be short but could lead to confusion. >> >>> > > >> >>> > > I'd like to ask for a 48h collection of opinions about the options: >> >>> > > >> >>> > > >> >>> > > 1. keep it as "Remote Executor" (and adjust the docs to name all >> >>> > > non-local executors to be "distributed") >> >>> > > 2. use "Distributed Executor" as provider and tool name, docs >> cli >> >>> etc >> >>> > > for AIP-69 >> >>> > > 3. Throw in other ideas of names >> >>> > > >> >>> > > This is not a formal vote but please respond with +1/0/-1 >> >>> > > binding/non-binding until Aug 29th 2024 4PM CEST. >> >>> > > >> >>> > > My opinion is: >> >>> > > A: +1 binding as it is short and was the original name with the AIP >> >>> posted. >> >>> > > B: 0 binding - can live with this if majority likes it >> >>> > > C: no better ideas that I can bring up. HTTP is too technical. A >> funny >> >>> > > name could be "AnyWhere" but nobody will understand. >> >>> > > >> >>> > > Jens >> >>> > > >> >>> > > PS: besides the naming looking forward for CODE reviews/approvers >> in >> >>> PRs >> >>> > > #41729,41730,41731 😀 >> >>> > > >> >>> > > Sent from Outlook for >> >>> > > iOS<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=05%7C02%7CJens.Scheffler%40de.bosch.com%7C934089bb04f84c4cd0f608dcc6ba8279%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638603751568086771%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=q9oy95Fq%2FqydrBkniY9PD9oLfXbfkAZUyRrVVXIXOtU%3D&reserved=0<https://aka.ms/o0ukef>> >> >>> > > >> >>> > > <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=05%7C02%7CJens.Scheffler%40de.bosch.com%7C934089bb04f84c4cd0f608dcc6ba8279%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638603751568086771%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=q9oy95Fq%2FqydrBkniY9PD9oLfXbfkAZUyRrVVXIXOtU%3D&reserved=0<https://aka.ms/o0ukef>>> >> >>> > > >> >>> > >> >>> >> >>> --------------------------------------------------------------------- >> >>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >> >>> <mailto:dev-unsubscr...@airflow.apache.org> >> >>> For additional commands, e-mail: dev-h...@airflow.apache.org >> >>> <mailto:dev-h...@airflow.apache.org> >> >>> >> >>> >>
