In case anyone's referring to the old PR, here is the link to the new PR https://github.com/apache/airflow/pull/52868
Best, Wei On 2025/06/26 12:10:24 Wei Lee wrote: > I agree with Ash's point. Given how other pipeline tools function, this > appears to be a standard feature. Additionally, it seems that this tool can > be utilized in AI, but it doesn't directly relate to it. Therefore, I would > suggest we avoid using "common.ai." > > Based on my observation of this thread, we're more inclined to add it to > the standard provider, so I'll continue working in this direction. > > https://github.com/apache/airflow/pull/52053 > This PR is semi-ready. The major functionality is present, but the test and > docs still need more cases. I'll try to wrap it up these days, but I would > appreciate it if we could get some early reviews. > > I also tried to create some follow-up tasks related to AIP-90 on this > board. https://github.com/orgs/apache/projects/508/views/1 > > Thanks a lot! > > Best, > Wei > > Jarek Potiuk <ja...@potiuk.com> 於 2025年6月25日 週三 下午7:26寫道： > > > > You can think of this very similar to the “Approval” step in CircleCI > > pipelines > > > > https://mahira-technology.medium.com/streamlining-your-workflow-implementing-a-manual-approval-step-in-circleci-pipelines-1e6a44905a92 > > and > > that’s why I think it should be part of the standard provider. (I know some > > people have run Terraform via Airflow pipelines and this sort of approval > > step would be useful in those workflows as an example.) > > > > Fair point. > > > > On Wed, Jun 25, 2025 at 1:23 PM Ash Berlin-Taylor <a...@apache.org> wrote: > > > > > I think that this _should_ be in the standard provider — as Kaxil > > > mentioned many legacy schedulers (Control-M, Informatica etc) have the > > > ability to put a workflow on hold until someone approves it, so I think > > > this deserves to be a part of standard Airflow available to “everyone”, > > > even if they aren’t doing anything to do with AI, so I think putting it > > in > > > common.ai isn’t right. > > > > > > You can think of this very similar to the “Approval” step in CircleCI > > > pipelines > > > > > https://mahira-technology.medium.com/streamlining-your-workflow-implementing-a-manual-approval-step-in-circleci-pipelines-1e6a44905a92 > > > and that’s why I think it should be part of the standard provider. (I > > know > > > some people have run Terraform via Airflow pipelines and this sort of > > > approval step would be useful in those workflows as an example.) > > > > > > -ash > > > > > > > On 25 Jun 2025, at 11:07, Bas Harenslak <b...@astronomer.io.INVALID> > > > wrote: > > > > > > > > I suggest adding it in the standard providers package. The PR ( > > > https://github.com/apache/airflow/pull/52053) adds several core Airflow > > > changes such as DB schema changes, REST API changes, and UI changes. This > > > makes it such a native feature that IMO it belongs in the standard > > package. > > > > > > > > Bas > > > > > > > >> On 25 Jun 2025, at 11:08, Jarek Potiuk <ja...@potiuk.com> wrote: > > > >> > > > >> I am also fine with common.ai . That fits better than standard. > > > >> > > > >> > > > >> On Wed, Jun 25, 2025 at 11:02 AM Amogh Desai < > > amoghdesai....@gmail.com> > > > >> wrote: > > > >> > > > >>> I am convinced about using HITL too. > > > >>> > > > >>> I agree that this should probably be a separate package and should > > not > > > be > > > >>> part of the standard > > > >>> one if possible due to plenty of reasons mentioned by Jens and > > others. > > > >>> > > > >>> "common.io" sounds to be a very interesting place to start, as this > > > >>> HITL operator might not be the only > > > >>> one we will implement in the long run. "human" operator sounds weird > > > to me. > > > >>> > > > >>> Thanks & Regards, > > > >>> Amogh Desai > > > >>> > > > >>> > > > >>> On Tue, Jun 24, 2025 at 1:43 PM Kaxil Naik <kaxiln...@gmail.com> > > > wrote: > > > >>> > > > >>>> Thanks Wei. > > > >>>> > > > >>>> 1) "Human in the Loop": +1 on the naming. Standard names. HITL > > > acronym is > > > >>>> also pretty standard ( > > https://en.wikipedia.org/wiki/Human-in-the-loop > > > | > > > >>>> https://cloud.google.com/discover/human-in-the-loop). "interactive" > > > is a > > > >>>> loaded term and be pretty vague. > > > >>>> 2) re: Standard vs Separate Provider: Fine with either. But if it is > > > in a > > > >>>> separate - the name "human" provider seems odd :) HITL as a > > > functionality > > > >>>> makes sense but a "human" provider seems odd to me. If it is > > separate > > > and > > > >>>> becomes part of "common.ai" - I am fine with that. I am equally > > happy > > > >>> with > > > >>>> keeping it in the Standard provider. Seems like a "core" > > functionality > > > >>>> compared to Control+M, and other legacy tools as well as new AI > > tools. > > > >>>> > > > >>>> On Tue, 24 Jun 2025 at 10:57, Jarek Potiuk <ja...@potiuk.com> > > wrote: > > > >>>> > > > >>>>> I like HITL as an acronym as well - it's well recognized. > > > >>>>> > > > >>>>> Just to add a bit of stir this is an interesting article when > > someone > > > >>>> tried > > > >>>>> to also distinct: > > > >>>>> * HITL (Human In the Loop) > > > >>>>> * with HOTL (Human On the Loop) > > > >>>>> * and HATL (Human Above the Loop) > > > >>>>> * and HBTL (Human Behind the Loop) > > > >>>>> > > > >>>>> Interesting concepts worth understanding the different roles humans > > > can > > > >>>>> play here - But that's more of an interesting side/related read :) > > . > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>> > > > >>> > > > > > https://medium.com/@pawel.rzeszucinski_55101/ai-humans-and-loops-04ee67ac820b > > > >>>>> > > > >>>>> :) > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> On Tue, Jun 24, 2025 at 4:35 AM Wei Lee <weilee...@gmail.com> > > wrote: > > > >>>>> > > > >>>>>> Hi everyone, > > > >>>>>> > > > >>>>>> Thank you for joining this discussion! At this point, it seems we > > > >>> have > > > >>>>>> reached some consensus. > > > >>>>>> > > > >>>>>> For naming, we now agree to use human centric term. To embed the > > > >>> whole > > > >>>>>> idea into operators, I will use HITL (apologies for the previous > > > >>> typo) > > > >>>>> and > > > >>>>>> mention "Human in the Loop" in the documentation and docstrings. > > > >>>>>> > > > >>>>>> Regarding whether it should be a standalone feature or not, it > > seems > > > >>>> more > > > >>>>>> like while it wouldn’t hurt to add it to the standard, it might be > > > >>>> better > > > >>>>>> to keep it separate. I’d like to gather more opinions on this. If > > we > > > >>>>> don’t > > > >>>>>> have a strong opinion about adding it to the standard, I think we > > > >>>> should > > > >>>>>> consider separating it. In the meantime, I will use the same PR to > > > >>>>> develop > > > >>>>>> the major functionality in the standard provider for easier > > > >>> development > > > >>>>> and > > > >>>>>> move it to a separate one if we reach a clear consensus. > > > >>>>>> > > > >>>>>> Thanks! > > > >>>>>> > > > >>>>>> Best, > > > >>>>>> Wei > > > >>>>>> > > > >>>>>>> On Jun 24, 2025, at 4:58 AM, Jens Scheffler > > > >>>> <j_scheff...@gmx.de.INVALID > > > >>>>>> > > > >>>>>> wrote: > > > >>>>>>> > > > >>>>>>> Hi, > > > >>>>>>> > > > >>>>>>> Thanks Wei for taking the lead in starting to implement! Hope I > > can > > > >>>>>> review the next days. > > > >>>>>>> > > > >>>>>>> as I was writing the AIP together with Vikram I was and still am > > > >>> for > > > >>>>>> (=+1) to keep it "human" centric. Also adding an API such that > > > >>> somebody > > > >>>>>> else is able to roll their whatever UI and not being locked into > > > >>>> Airflow > > > >>>>> UI > > > >>>>>> but still with the aim to loop-in humans. > > > >>>>>>> > > > >>>>>>> For the provider question I am for a separate provider because > > (1) > > > >>> it > > > >>>>>> was as such in the AIP, (2) I see it is optional and we should not > > > >>>> force > > > >>>>>> every install to have this (as it has not been there the last 10 > > > >>> years > > > >>>> I > > > >>>>>> assume there are many installs not needing it actually and some > > > >>>>> objections > > > >>>>>> were raised in the discussion that it is accepted if it is an > > > >>> optional > > > >>>>>> feature which it would not be if in standard provider) as well as > > > (3) > > > >>>> we > > > >>>>>> need to adjust the DB and slightly extend this for the human > > > response > > > >>>>> data > > > >>>>>> storage - and I would feel uncomfortable to force this DB > > extension > > > >>>> with > > > >>>>>> every install... then we could also directly package this into > > core > > > - > > > >>>> but > > > >>>>>> as (2) it should be an optional feature. > > > >>>>>>> > > > >>>>>>> As well as (4) other common things like http, ftp, git, sftp, > > smtp > > > >>>> are > > > >>>>>> also pretty like stdnard but are also separate providers. From > > point > > > >>> of > > > >>>>>> security (5) every additional thing adds a bit of complexity and > > if > > > >>> you > > > >>>>>> want to make your setup secure you want to slim it down to the > > > >>>> functions > > > >>>>>> you need. Even though minimal if no human interaction is needed > > then > > > >>> I > > > >>>>>> think we should not force every install to have this. > > > >>>>>>> > > > >>>>>>> TLDR I'd therefore favor (+1) a separate "human" provider; not > > > >>>> favoring > > > >>>>>> (but +0) adding this to standard. > > > >>>>>>> > > > >>>>>>> Jens > > > >>>>>>> > > > >>>>>>> On 23.06.25 08:02, Amogh Desai wrote: > > > >>>>>>>> I was not strongly against using "human" -- it just felt a > > little > > > >>>> odd > > > >>>>>> and > > > >>>>>>>> confusing to me at first. > > > >>>>>>>> > > > >>>>>>>> Jarek's email has convinced me that having HITH is contextual in > > > >>> the > > > >>>>> AI > > > >>>>>>>> space and it is kinda what > > > >>>>>>>> we are doing with this AIP - 90. In fact, using "interactive" > > now > > > >>>>> seems > > > >>>>>> odd > > > >>>>>>>> that it is not descriptive enough > > > >>>>>>>> or doesn't highlight the intention of the operator enough. > > > >>>>>>>> > > > >>>>>>>> I do not have concerns with whatever we decide to name it :D > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> Thanks & Regards, > > > >>>>>>>> Amogh Desai > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> On Mon, Jun 23, 2025 at 9:42 AM Jarek Potiuk <ja...@potiuk.com> > > > >>>>> wrote: > > > >>>>>>>> > > > >>>>>>>>> In standard Provider, yes > > > >>>>>>>>> > > > >>>>>>>>> Re: name: I changed my opinion. Previously I raised concerns > > > >>> about > > > >>>>> it, > > > >>>>>> but > > > >>>>>>>>> they are gone. The name is IMHO perfect. > > > >>>>>>>>> > > > >>>>>>>>> Why do I think "Human-In-The-Loop" is the **right** name. It's > > a > > > >>>> very > > > >>>>>>>>> popular term in AI workflows, and used to interact with the > > > >>> "real" > > > >>>>>> human, > > > >>>>>>>>> and it has a very concrete meaning. Also I think it's really, > > > >>>> really > > > >>>>>> worth > > > >>>>>>>>> looking at the talk by Andrey Karpathy published a few days ago > > > >>>>>>>>> https://www.youtube.com/watch?v=LCEmiRjPEtQ - I think it is > > > >>> very > > > >>>>>>>>> insightful. Andrey coined the term "Vibe Coding", and I think > > he > > > >>> is > > > >>>>>> one of > > > >>>>>>>>> the smartest people in the AI space who is not hype-driven - > > i.e. > > > >>>> he > > > >>>>>> seems > > > >>>>>>>>> to genuinely think that AI is another technology change that is > > > >>>>>> reinventing > > > >>>>>>>>> how we do software. Unlike many of others he is not "selling" > > > >>> their > > > >>>>>> product > > > >>>>>>>>> in AI, he seems to be focused on one thing that I believe also > > is > > > >>>>>> important > > > >>>>>>>>> i.e. "Keeping Human in the Loop". > > > >>>>>>>>> > > > >>>>>>>>> One of the very interesting things I've learned from that talk > > is > > > >>>> how > > > >>>>>>>>> important it is to provide a good User Interface to AI. I.e. > > that > > > >>>> the > > > >>>>>> chat > > > >>>>>>>>> interface is cool, and everything but the crucial part of the > > AI > > > >>>>>>>>> interaction is to wrap the AI results into actionable, quick > > and > > > >>>>>> "nice" way > > > >>>>>>>>> of interacting with various aspects of AI by Human(s), when the > > > >>>> input > > > >>>>>> is > > > >>>>>>>>> not only important, but crucial to get the real value of AI. > > > >>>>>>>>> > > > >>>>>>>>> In this context I think we should focus to make sure that our > > > >>>> "Human > > > >>>>> In > > > >>>>>>>>> the Loop" is indeed designed for the Human - not for LLM > > > >>> imitating > > > >>>>>> Human, > > > >>>>>>>>> not for Agents. It should have a nice, pleasant and efficient > > > >>>>>>>>> UI, that should allow surfacing all the information that is > > > >>>> necessary > > > >>>>>> for > > > >>>>>>>>> the Human to make the decision. That information should be > > > >>>>>>>>> nicely formattable, and you should be able to use the typical > > way > > > >>>>>>>>> that people interact with it - with controls and everything > > they > > > >>>> are > > > >>>>>>>>> used to. A good interface example of a UI is when you use > > Copilot > > > >>>> in > > > >>>>>> your > > > >>>>>>>>> IDE for the translation. For example, the information you get > > (as > > > >>>>>> human) is > > > >>>>>>>>> targeted for humans and is very actionable. It is put in > > context, > > > >>>> you > > > >>>>>> can > > > >>>>>>>>> interact with it individually by accepting individual > > suggestions > > > >>>> (or > > > >>>>>>>>> rejecting them) or accept/reject things in bulk. > > > >>>>>>>>> > > > >>>>>>>>> Here is an example: (for those who do not see embedded picture > > - > > > >>>> link > > > >>>>>> here > > > >>>>>>>>> https://ibb.co/3Y03xN06) > > > >>>>>>>>> > > > >>>>>>>>> [image: Screenshot 2025-06-23 at 06.05.38.png] > > > >>>>>>>>> > > > >>>>>>>>> We should design "Human In the Loop" of ours in a very similar > > > >>> way > > > >>>> - > > > >>>>>> i.e. > > > >>>>>>>>> give the author of the "HIL" interaction capability of adding > > UI > > > >>>>>>>>> components, surfacing the right information - and having rich > > > >>>>>> interaction. > > > >>>>>>>>> Maybe not all the bells and whistles initially (for example > > it's > > > >>> ok > > > >>>>>> for now > > > >>>>>>>>> to just have bulk decisions on the whole interaction, but I > > think > > > >>>>> this > > > >>>>>>>>> should be our long-term design goal to allow for richer > > > >>>> interactions. > > > >>>>>> And - > > > >>>>>>>>> in this context - "Human in the loop" is a very appropriate > > name. > > > >>>>>>>>> > > > >>>>>>>>> BTW. Slightly related - there is a blog post coming about it > > > >>> from a > > > >>>>>> few of > > > >>>>>>>>> us about AI with internationalisation and how we made it to > > > >>> follow > > > >>>>> that > > > >>>>>>>>> (pretty naturally) with Open-Source spirit - by making sure > > that > > > >>> we > > > >>>>>> keep > > > >>>>>>>>> Human In the Loop and that it is designed to follow the Open > > > >>> Source > > > >>>>>> Spirit, > > > >>>>>>>>> Foster collaboration. > > > >>>>>>>>> > > > >>>>>>>>> > > > >>>>>>>>> On Mon, Jun 23, 2025 at 4:42 AM Wei Lee <weilee...@gmail.com> > > > >>>> wrote: > > > >>>>>>>>> > > > >>>>>>>>>> Got it! Yes, it makes sense to keep the phrase widely used. > > > >>>> Thanks a > > > >>>>>> lot! > > > >>>>>>>>>> As a compromise, I will try something like `HITHOperator`, > > which > > > >>>> may > > > >>>>>>>>>> address some of the concerns. We can always rename it to > > > >>> whatever > > > >>>> we > > > >>>>>> decide > > > >>>>>>>>>> before the release. I will also send a follow-up email to this > > > >>>>> thread > > > >>>>>> once > > > >>>>>>>>>> it's ready for review, so that anyone interested can take a > > > >>> look. > > > >>>>>>>>>> > > > >>>>>>>>>> Best, > > > >>>>>>>>>> Wei > > > >>>>>>>>>> > > > >>>>>>>>>>> On Jun 23, 2025, at 10:27 AM, Vikram Koka > > > >>>>>> <vik...@astronomer.io.INVALID> > > > >>>>>>>>>> wrote: > > > >>>>>>>>>>> I agree with the standard provider approach. > > > >>>>>>>>>>> > > > >>>>>>>>>>> > > > >>>>>>>>>>> On Sun, Jun 22, 2025 at 7:26 PM Vikram Koka < > > > >>>> vik...@astronomer.io> > > > >>>>>>>>>> wrote: > > > >>>>>>>>>>>> Thanks Wei, I really appreciate the work, and will review it > > > >>> as > > > >>>>>> soon as > > > >>>>>>>>>>>> possible. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> With respect to the naming, I believe the Human-in-the-loop > > is > > > >>>> the > > > >>>>>>>>>> right > > > >>>>>>>>>>>> phrase, because that is recognized as such both in older > > > >>>> "legacy" > > > >>>>>>>>>> systems, > > > >>>>>>>>>>>> as well as the new AI solutions. I agree that it may be less > > > >>>> than > > > >>>>>> ideal > > > >>>>>>>>>>>> from a technical perspective, but from a user perspective, I > > > >>>>> believe > > > >>>>>>>>>> it is > > > >>>>>>>>>>>> better to stick with a known term, rather than to invent our > > > >>>> own. > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> > > > >>>>>>>>>>>> On Sun, Jun 22, 2025 at 7:07 PM Wei Lee < > > weilee...@gmail.com> > > > >>>>>> wrote: > > > >>>>>>>>>>>> > > > >>>>>>>>>>>>> Hi fellow Airflower, > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> I am currently working on a PoC for AIP-90. I've > > incorporated > > > >>>>> some > > > >>>>>>>>>>>>> suggestions based on comments in the voting thread and Jira > > > >>>> page. > > > >>>>>>>>>> Since > > > >>>>>>>>>>>>> they have not yet been included in the AIP, I want to > > confirm > > > >>>>> with > > > >>>>>>>>>> everyone > > > >>>>>>>>>>>>> to ensure I'm on the right track. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> 1. Many have expressed concerns about the term “Human,” so > > > >>> I'm > > > >>>>> now > > > >>>>>>>>>> using > > > >>>>>>>>>>>>> the term “Interactive” as suggested by Among. For example, > > I > > > >>>>> change > > > >>>>>>>>>>>>> "HumanOperator" to “InteractiveOperator". > > > >>>>>>>>>>>>> 2. This functionality is now part of the standard provider > > > >>>> rather > > > >>>>>> than > > > >>>>>>>>>>>>> being a separate provider as suggested by Bas and Ash. > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Here is the PoC PR. > > > >>>> https://github.com/apache/airflow/pull/52053 > > > >>>>>>>>>>>>> It's not ready to be reviewed yet, but I'll try to wrap it > > up > > > >>>>> over > > > >>>>>> the > > > >>>>>>>>>>>>> next few days. Several features are still missing and will > > be > > > >>>>>>>>>> implemented > > > >>>>>>>>>>>>> in the following pull requests. Thanks! > > > >>>>>>>>>>>>> > > > >>>>>>>>>>>>> Best regards, > > > >>>>>>>>>>>>> Wei Lee > > > >>>>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>>>> > > --------------------------------------------------------------------- > > > >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > >>>>>>>>>> For additional commands, e-mail: dev-h...@airflow.apache.org > > > >>>>>>>>>> > > > >>>>>>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>> --------------------------------------------------------------------- > > > >>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > >>>>>>> For additional commands, e-mail: dev-h...@airflow.apache.org > > > >>>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > --------------------------------------------------------------------- > > > >>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > >>>>>> For additional commands, e-mail: dev-h...@airflow.apache.org > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>>> > > > >>> > > > > > > > > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
