I think that this _should_ be in the standard provider — as Kaxil mentioned many legacy schedulers (Control-M, Informatica etc) have the ability to put a workflow on hold until someone approves it, so I think this deserves to be a part of standard Airflow available to “everyone”, even if they aren’t doing anything to do with AI, so I think putting it in common.ai isn’t right.
You can think of this very similar to the “Approval” step in CircleCI pipelines https://mahira-technology.medium.com/streamlining-your-workflow-implementing-a-manual-approval-step-in-circleci-pipelines-1e6a44905a92 and that’s why I think it should be part of the standard provider. (I know some people have run Terraform via Airflow pipelines and this sort of approval step would be useful in those workflows as an example.) -ash > On 25 Jun 2025, at 11:07, Bas Harenslak <b...@astronomer.io.INVALID> wrote: > > I suggest adding it in the standard providers package. The PR > (https://github.com/apache/airflow/pull/52053) adds several core Airflow > changes such as DB schema changes, REST API changes, and UI changes. This > makes it such a native feature that IMO it belongs in the standard package. > > Bas > >> On 25 Jun 2025, at 11:08, Jarek Potiuk <ja...@potiuk.com> wrote: >> >> I am also fine with common.ai . That fits better than standard. >> >> >> On Wed, Jun 25, 2025 at 11:02 AM Amogh Desai <amoghdesai....@gmail.com> >> wrote: >> >>> I am convinced about using HITL too. >>> >>> I agree that this should probably be a separate package and should not be >>> part of the standard >>> one if possible due to plenty of reasons mentioned by Jens and others. >>> >>> "common.io" sounds to be a very interesting place to start, as this >>> HITL operator might not be the only >>> one we will implement in the long run. "human" operator sounds weird to me. >>> >>> Thanks & Regards, >>> Amogh Desai >>> >>> >>> On Tue, Jun 24, 2025 at 1:43 PM Kaxil Naik <kaxiln...@gmail.com> wrote: >>> >>>> Thanks Wei. >>>> >>>> 1) "Human in the Loop": +1 on the naming. Standard names. HITL acronym is >>>> also pretty standard (https://en.wikipedia.org/wiki/Human-in-the-loop | >>>> https://cloud.google.com/discover/human-in-the-loop). "interactive" is a >>>> loaded term and be pretty vague. >>>> 2) re: Standard vs Separate Provider: Fine with either. But if it is in a >>>> separate - the name "human" provider seems odd :) HITL as a functionality >>>> makes sense but a "human" provider seems odd to me. If it is separate and >>>> becomes part of "common.ai" - I am fine with that. I am equally happy >>> with >>>> keeping it in the Standard provider. Seems like a "core" functionality >>>> compared to Control+M, and other legacy tools as well as new AI tools. >>>> >>>> On Tue, 24 Jun 2025 at 10:57, Jarek Potiuk <ja...@potiuk.com> wrote: >>>> >>>>> I like HITL as an acronym as well - it's well recognized. >>>>> >>>>> Just to add a bit of stir this is an interesting article when someone >>>> tried >>>>> to also distinct: >>>>> * HITL (Human In the Loop) >>>>> * with HOTL (Human On the Loop) >>>>> * and HATL (Human Above the Loop) >>>>> * and HBTL (Human Behind the Loop) >>>>> >>>>> Interesting concepts worth understanding the different roles humans can >>>>> play here - But that's more of an interesting side/related read :) . >>>>> >>>>> >>>>> >>>> >>> https://medium.com/@pawel.rzeszucinski_55101/ai-humans-and-loops-04ee67ac820b >>>>> >>>>> :) >>>>> >>>>> >>>>> >>>>> On Tue, Jun 24, 2025 at 4:35 AM Wei Lee <weilee...@gmail.com> wrote: >>>>> >>>>>> Hi everyone, >>>>>> >>>>>> Thank you for joining this discussion! At this point, it seems we >>> have >>>>>> reached some consensus. >>>>>> >>>>>> For naming, we now agree to use human centric term. To embed the >>> whole >>>>>> idea into operators, I will use HITL (apologies for the previous >>> typo) >>>>> and >>>>>> mention "Human in the Loop" in the documentation and docstrings. >>>>>> >>>>>> Regarding whether it should be a standalone feature or not, it seems >>>> more >>>>>> like while it wouldn’t hurt to add it to the standard, it might be >>>> better >>>>>> to keep it separate. I’d like to gather more opinions on this. If we >>>>> don’t >>>>>> have a strong opinion about adding it to the standard, I think we >>>> should >>>>>> consider separating it. In the meantime, I will use the same PR to >>>>> develop >>>>>> the major functionality in the standard provider for easier >>> development >>>>> and >>>>>> move it to a separate one if we reach a clear consensus. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> Best, >>>>>> Wei >>>>>> >>>>>>> On Jun 24, 2025, at 4:58 AM, Jens Scheffler >>>> <j_scheff...@gmx.de.INVALID >>>>>> >>>>>> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Thanks Wei for taking the lead in starting to implement! Hope I can >>>>>> review the next days. >>>>>>> >>>>>>> as I was writing the AIP together with Vikram I was and still am >>> for >>>>>> (=+1) to keep it "human" centric. Also adding an API such that >>> somebody >>>>>> else is able to roll their whatever UI and not being locked into >>>> Airflow >>>>> UI >>>>>> but still with the aim to loop-in humans. >>>>>>> >>>>>>> For the provider question I am for a separate provider because (1) >>> it >>>>>> was as such in the AIP, (2) I see it is optional and we should not >>>> force >>>>>> every install to have this (as it has not been there the last 10 >>> years >>>> I >>>>>> assume there are many installs not needing it actually and some >>>>> objections >>>>>> were raised in the discussion that it is accepted if it is an >>> optional >>>>>> feature which it would not be if in standard provider) as well as (3) >>>> we >>>>>> need to adjust the DB and slightly extend this for the human response >>>>> data >>>>>> storage - and I would feel uncomfortable to force this DB extension >>>> with >>>>>> every install... then we could also directly package this into core - >>>> but >>>>>> as (2) it should be an optional feature. >>>>>>> >>>>>>> As well as (4) other common things like http, ftp, git, sftp, smtp >>>> are >>>>>> also pretty like stdnard but are also separate providers. From point >>> of >>>>>> security (5) every additional thing adds a bit of complexity and if >>> you >>>>>> want to make your setup secure you want to slim it down to the >>>> functions >>>>>> you need. Even though minimal if no human interaction is needed then >>> I >>>>>> think we should not force every install to have this. >>>>>>> >>>>>>> TLDR I'd therefore favor (+1) a separate "human" provider; not >>>> favoring >>>>>> (but +0) adding this to standard. >>>>>>> >>>>>>> Jens >>>>>>> >>>>>>> On 23.06.25 08:02, Amogh Desai wrote: >>>>>>>> I was not strongly against using "human" -- it just felt a little >>>> odd >>>>>> and >>>>>>>> confusing to me at first. >>>>>>>> >>>>>>>> Jarek's email has convinced me that having HITH is contextual in >>> the >>>>> AI >>>>>>>> space and it is kinda what >>>>>>>> we are doing with this AIP - 90. In fact, using "interactive" now >>>>> seems >>>>>> odd >>>>>>>> that it is not descriptive enough >>>>>>>> or doesn't highlight the intention of the operator enough. >>>>>>>> >>>>>>>> I do not have concerns with whatever we decide to name it :D >>>>>>>> >>>>>>>> >>>>>>>> Thanks & Regards, >>>>>>>> Amogh Desai >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Jun 23, 2025 at 9:42 AM Jarek Potiuk <ja...@potiuk.com> >>>>> wrote: >>>>>>>> >>>>>>>>> In standard Provider, yes >>>>>>>>> >>>>>>>>> Re: name: I changed my opinion. Previously I raised concerns >>> about >>>>> it, >>>>>> but >>>>>>>>> they are gone. The name is IMHO perfect. >>>>>>>>> >>>>>>>>> Why do I think "Human-In-The-Loop" is the **right** name. It's a >>>> very >>>>>>>>> popular term in AI workflows, and used to interact with the >>> "real" >>>>>> human, >>>>>>>>> and it has a very concrete meaning. Also I think it's really, >>>> really >>>>>> worth >>>>>>>>> looking at the talk by Andrey Karpathy published a few days ago >>>>>>>>> https://www.youtube.com/watch?v=LCEmiRjPEtQ - I think it is >>> very >>>>>>>>> insightful. Andrey coined the term "Vibe Coding", and I think he >>> is >>>>>> one of >>>>>>>>> the smartest people in the AI space who is not hype-driven - i.e. >>>> he >>>>>> seems >>>>>>>>> to genuinely think that AI is another technology change that is >>>>>> reinventing >>>>>>>>> how we do software. Unlike many of others he is not "selling" >>> their >>>>>> product >>>>>>>>> in AI, he seems to be focused on one thing that I believe also is >>>>>> important >>>>>>>>> i.e. "Keeping Human in the Loop". >>>>>>>>> >>>>>>>>> One of the very interesting things I've learned from that talk is >>>> how >>>>>>>>> important it is to provide a good User Interface to AI. I.e. that >>>> the >>>>>> chat >>>>>>>>> interface is cool, and everything but the crucial part of the AI >>>>>>>>> interaction is to wrap the AI results into actionable, quick and >>>>>> "nice" way >>>>>>>>> of interacting with various aspects of AI by Human(s), when the >>>> input >>>>>> is >>>>>>>>> not only important, but crucial to get the real value of AI. >>>>>>>>> >>>>>>>>> In this context I think we should focus to make sure that our >>>> "Human >>>>> In >>>>>>>>> the Loop" is indeed designed for the Human - not for LLM >>> imitating >>>>>> Human, >>>>>>>>> not for Agents. It should have a nice, pleasant and efficient >>>>>>>>> UI, that should allow surfacing all the information that is >>>> necessary >>>>>> for >>>>>>>>> the Human to make the decision. That information should be >>>>>>>>> nicely formattable, and you should be able to use the typical way >>>>>>>>> that people interact with it - with controls and everything they >>>> are >>>>>>>>> used to. A good interface example of a UI is when you use Copilot >>>> in >>>>>> your >>>>>>>>> IDE for the translation. For example, the information you get (as >>>>>> human) is >>>>>>>>> targeted for humans and is very actionable. It is put in context, >>>> you >>>>>> can >>>>>>>>> interact with it individually by accepting individual suggestions >>>> (or >>>>>>>>> rejecting them) or accept/reject things in bulk. >>>>>>>>> >>>>>>>>> Here is an example: (for those who do not see embedded picture - >>>> link >>>>>> here >>>>>>>>> https://ibb.co/3Y03xN06) >>>>>>>>> >>>>>>>>> [image: Screenshot 2025-06-23 at 06.05.38.png] >>>>>>>>> >>>>>>>>> We should design "Human In the Loop" of ours in a very similar >>> way >>>> - >>>>>> i.e. >>>>>>>>> give the author of the "HIL" interaction capability of adding UI >>>>>>>>> components, surfacing the right information - and having rich >>>>>> interaction. >>>>>>>>> Maybe not all the bells and whistles initially (for example it's >>> ok >>>>>> for now >>>>>>>>> to just have bulk decisions on the whole interaction, but I think >>>>> this >>>>>>>>> should be our long-term design goal to allow for richer >>>> interactions. >>>>>> And - >>>>>>>>> in this context - "Human in the loop" is a very appropriate name. >>>>>>>>> >>>>>>>>> BTW. Slightly related - there is a blog post coming about it >>> from a >>>>>> few of >>>>>>>>> us about AI with internationalisation and how we made it to >>> follow >>>>> that >>>>>>>>> (pretty naturally) with Open-Source spirit - by making sure that >>> we >>>>>> keep >>>>>>>>> Human In the Loop and that it is designed to follow the Open >>> Source >>>>>> Spirit, >>>>>>>>> Foster collaboration. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Jun 23, 2025 at 4:42 AM Wei Lee <weilee...@gmail.com> >>>> wrote: >>>>>>>>> >>>>>>>>>> Got it! Yes, it makes sense to keep the phrase widely used. >>>> Thanks a >>>>>> lot! >>>>>>>>>> As a compromise, I will try something like `HITHOperator`, which >>>> may >>>>>>>>>> address some of the concerns. We can always rename it to >>> whatever >>>> we >>>>>> decide >>>>>>>>>> before the release. I will also send a follow-up email to this >>>>> thread >>>>>> once >>>>>>>>>> it's ready for review, so that anyone interested can take a >>> look. >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Wei >>>>>>>>>> >>>>>>>>>>> On Jun 23, 2025, at 10:27 AM, Vikram Koka >>>>>> <vik...@astronomer.io.INVALID> >>>>>>>>>> wrote: >>>>>>>>>>> I agree with the standard provider approach. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sun, Jun 22, 2025 at 7:26 PM Vikram Koka < >>>> vik...@astronomer.io> >>>>>>>>>> wrote: >>>>>>>>>>>> Thanks Wei, I really appreciate the work, and will review it >>> as >>>>>> soon as >>>>>>>>>>>> possible. >>>>>>>>>>>> >>>>>>>>>>>> With respect to the naming, I believe the Human-in-the-loop is >>>> the >>>>>>>>>> right >>>>>>>>>>>> phrase, because that is recognized as such both in older >>>> "legacy" >>>>>>>>>> systems, >>>>>>>>>>>> as well as the new AI solutions. I agree that it may be less >>>> than >>>>>> ideal >>>>>>>>>>>> from a technical perspective, but from a user perspective, I >>>>> believe >>>>>>>>>> it is >>>>>>>>>>>> better to stick with a known term, rather than to invent our >>>> own. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Sun, Jun 22, 2025 at 7:07 PM Wei Lee <weilee...@gmail.com> >>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi fellow Airflower, >>>>>>>>>>>>> >>>>>>>>>>>>> I am currently working on a PoC for AIP-90. I've incorporated >>>>> some >>>>>>>>>>>>> suggestions based on comments in the voting thread and Jira >>>> page. >>>>>>>>>> Since >>>>>>>>>>>>> they have not yet been included in the AIP, I want to confirm >>>>> with >>>>>>>>>> everyone >>>>>>>>>>>>> to ensure I'm on the right track. >>>>>>>>>>>>> >>>>>>>>>>>>> 1. Many have expressed concerns about the term “Human,” so >>> I'm >>>>> now >>>>>>>>>> using >>>>>>>>>>>>> the term “Interactive” as suggested by Among. For example, I >>>>> change >>>>>>>>>>>>> "HumanOperator" to “InteractiveOperator". >>>>>>>>>>>>> 2. This functionality is now part of the standard provider >>>> rather >>>>>> than >>>>>>>>>>>>> being a separate provider as suggested by Bas and Ash. >>>>>>>>>>>>> >>>>>>>>>>>>> Here is the PoC PR. >>>> https://github.com/apache/airflow/pull/52053 >>>>>>>>>>>>> It's not ready to be reviewed yet, but I'll try to wrap it up >>>>> over >>>>>> the >>>>>>>>>>>>> next few days. Several features are still missing and will be >>>>>>>>>> implemented >>>>>>>>>>>>> in the following pull requests. Thanks! >>>>>>>>>>>>> >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Wei Lee >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>> --------------------------------------------------------------------- >>>>>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>>>>>>>> For additional commands, e-mail: dev-h...@airflow.apache.org >>>>>>>>>> >>>>>>>>>> >>>>>>> >>>>>>> >>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>>>>> For additional commands, e-mail: dev-h...@airflow.apache.org >>>>>>> >>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >>>>>> For additional commands, e-mail: dev-h...@airflow.apache.org >>>>>> >>>>>> >>>>> >>>> >>> >
