I like HITL as an acronym as well - it's well recognized. Just to add a bit of stir this is an interesting article when someone tried to also distinct: * HITL (Human In the Loop) * with HOTL (Human On the Loop) * and HATL (Human Above the Loop) * and HBTL (Human Behind the Loop)
Interesting concepts worth understanding the different roles humans can play here - But that's more of an interesting side/related read :) . https://medium.com/@pawel.rzeszucinski_55101/ai-humans-and-loops-04ee67ac820b :) On Tue, Jun 24, 2025 at 4:35 AM Wei Lee <weilee...@gmail.com> wrote: > Hi everyone, > > Thank you for joining this discussion! At this point, it seems we have > reached some consensus. > > For naming, we now agree to use human centric term. To embed the whole > idea into operators, I will use HITL (apologies for the previous typo) and > mention "Human in the Loop" in the documentation and docstrings. > > Regarding whether it should be a standalone feature or not, it seems more > like while it wouldn’t hurt to add it to the standard, it might be better > to keep it separate. I’d like to gather more opinions on this. If we don’t > have a strong opinion about adding it to the standard, I think we should > consider separating it. In the meantime, I will use the same PR to develop > the major functionality in the standard provider for easier development and > move it to a separate one if we reach a clear consensus. > > Thanks! > > Best, > Wei > > > On Jun 24, 2025, at 4:58 AM, Jens Scheffler <j_scheff...@gmx.de.INVALID> > wrote: > > > > Hi, > > > > Thanks Wei for taking the lead in starting to implement! Hope I can > review the next days. > > > > as I was writing the AIP together with Vikram I was and still am for > (=+1) to keep it "human" centric. Also adding an API such that somebody > else is able to roll their whatever UI and not being locked into Airflow UI > but still with the aim to loop-in humans. > > > > For the provider question I am for a separate provider because (1) it > was as such in the AIP, (2) I see it is optional and we should not force > every install to have this (as it has not been there the last 10 years I > assume there are many installs not needing it actually and some objections > were raised in the discussion that it is accepted if it is an optional > feature which it would not be if in standard provider) as well as (3) we > need to adjust the DB and slightly extend this for the human response data > storage - and I would feel uncomfortable to force this DB extension with > every install... then we could also directly package this into core - but > as (2) it should be an optional feature. > > > > As well as (4) other common things like http, ftp, git, sftp, smtp are > also pretty like stdnard but are also separate providers. From point of > security (5) every additional thing adds a bit of complexity and if you > want to make your setup secure you want to slim it down to the functions > you need. Even though minimal if no human interaction is needed then I > think we should not force every install to have this. > > > > TLDR I'd therefore favor (+1) a separate "human" provider; not favoring > (but +0) adding this to standard. > > > > Jens > > > > On 23.06.25 08:02, Amogh Desai wrote: > >> I was not strongly against using "human" -- it just felt a little odd > and > >> confusing to me at first. > >> > >> Jarek's email has convinced me that having HITH is contextual in the AI > >> space and it is kinda what > >> we are doing with this AIP - 90. In fact, using "interactive" now seems > odd > >> that it is not descriptive enough > >> or doesn't highlight the intention of the operator enough. > >> > >> I do not have concerns with whatever we decide to name it :D > >> > >> > >> Thanks & Regards, > >> Amogh Desai > >> > >> > >> On Mon, Jun 23, 2025 at 9:42 AM Jarek Potiuk <ja...@potiuk.com> wrote: > >> > >>> In standard Provider, yes > >>> > >>> Re: name: I changed my opinion. Previously I raised concerns about it, > but > >>> they are gone. The name is IMHO perfect. > >>> > >>> Why do I think "Human-In-The-Loop" is the **right** name. It's a very > >>> popular term in AI workflows, and used to interact with the "real" > human, > >>> and it has a very concrete meaning. Also I think it's really, really > worth > >>> looking at the talk by Andrey Karpathy published a few days ago > >>> https://www.youtube.com/watch?v=LCEmiRjPEtQ - I think it is very > >>> insightful. Andrey coined the term "Vibe Coding", and I think he is > one of > >>> the smartest people in the AI space who is not hype-driven - i.e. he > seems > >>> to genuinely think that AI is another technology change that is > reinventing > >>> how we do software. Unlike many of others he is not "selling" their > product > >>> in AI, he seems to be focused on one thing that I believe also is > important > >>> i.e. "Keeping Human in the Loop". > >>> > >>> One of the very interesting things I've learned from that talk is how > >>> important it is to provide a good User Interface to AI. I.e. that the > chat > >>> interface is cool, and everything but the crucial part of the AI > >>> interaction is to wrap the AI results into actionable, quick and > "nice" way > >>> of interacting with various aspects of AI by Human(s), when the input > is > >>> not only important, but crucial to get the real value of AI. > >>> > >>> In this context I think we should focus to make sure that our "Human In > >>> the Loop" is indeed designed for the Human - not for LLM imitating > Human, > >>> not for Agents. It should have a nice, pleasant and efficient > >>> UI, that should allow surfacing all the information that is necessary > for > >>> the Human to make the decision. That information should be > >>> nicely formattable, and you should be able to use the typical way > >>> that people interact with it - with controls and everything they are > >>> used to. A good interface example of a UI is when you use Copilot in > your > >>> IDE for the translation. For example, the information you get (as > human) is > >>> targeted for humans and is very actionable. It is put in context, you > can > >>> interact with it individually by accepting individual suggestions (or > >>> rejecting them) or accept/reject things in bulk. > >>> > >>> Here is an example: (for those who do not see embedded picture - link > here > >>> https://ibb.co/3Y03xN06) > >>> > >>> [image: Screenshot 2025-06-23 at 06.05.38.png] > >>> > >>> We should design "Human In the Loop" of ours in a very similar way - > i.e. > >>> give the author of the "HIL" interaction capability of adding UI > >>> components, surfacing the right information - and having rich > interaction. > >>> Maybe not all the bells and whistles initially (for example it's ok > for now > >>> to just have bulk decisions on the whole interaction, but I think this > >>> should be our long-term design goal to allow for richer interactions. > And - > >>> in this context - "Human in the loop" is a very appropriate name. > >>> > >>> BTW. Slightly related - there is a blog post coming about it from a > few of > >>> us about AI with internationalisation and how we made it to follow that > >>> (pretty naturally) with Open-Source spirit - by making sure that we > keep > >>> Human In the Loop and that it is designed to follow the Open Source > Spirit, > >>> Foster collaboration. > >>> > >>> > >>> On Mon, Jun 23, 2025 at 4:42 AM Wei Lee <weilee...@gmail.com> wrote: > >>> > >>>> Got it! Yes, it makes sense to keep the phrase widely used. Thanks a > lot! > >>>> As a compromise, I will try something like `HITHOperator`, which may > >>>> address some of the concerns. We can always rename it to whatever we > decide > >>>> before the release. I will also send a follow-up email to this thread > once > >>>> it's ready for review, so that anyone interested can take a look. > >>>> > >>>> Best, > >>>> Wei > >>>> > >>>>> On Jun 23, 2025, at 10:27 AM, Vikram Koka > <vik...@astronomer.io.INVALID> > >>>> wrote: > >>>>> I agree with the standard provider approach. > >>>>> > >>>>> > >>>>> On Sun, Jun 22, 2025 at 7:26 PM Vikram Koka <vik...@astronomer.io> > >>>> wrote: > >>>>>> Thanks Wei, I really appreciate the work, and will review it as > soon as > >>>>>> possible. > >>>>>> > >>>>>> With respect to the naming, I believe the Human-in-the-loop is the > >>>> right > >>>>>> phrase, because that is recognized as such both in older "legacy" > >>>> systems, > >>>>>> as well as the new AI solutions. I agree that it may be less than > ideal > >>>>>> from a technical perspective, but from a user perspective, I believe > >>>> it is > >>>>>> better to stick with a known term, rather than to invent our own. > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Sun, Jun 22, 2025 at 7:07 PM Wei Lee <weilee...@gmail.com> > wrote: > >>>>>> > >>>>>>> Hi fellow Airflower, > >>>>>>> > >>>>>>> I am currently working on a PoC for AIP-90. I've incorporated some > >>>>>>> suggestions based on comments in the voting thread and Jira page. > >>>> Since > >>>>>>> they have not yet been included in the AIP, I want to confirm with > >>>> everyone > >>>>>>> to ensure I'm on the right track. > >>>>>>> > >>>>>>> 1. Many have expressed concerns about the term “Human,” so I'm now > >>>> using > >>>>>>> the term “Interactive” as suggested by Among. For example, I change > >>>>>>> "HumanOperator" to “InteractiveOperator". > >>>>>>> 2. This functionality is now part of the standard provider rather > than > >>>>>>> being a separate provider as suggested by Bas and Ash. > >>>>>>> > >>>>>>> Here is the PoC PR. https://github.com/apache/airflow/pull/52053 > >>>>>>> It's not ready to be reviewed yet, but I'll try to wrap it up over > the > >>>>>>> next few days. Several features are still missing and will be > >>>> implemented > >>>>>>> in the following pull requests. Thanks! > >>>>>>> > >>>>>>> Best regards, > >>>>>>> Wei Lee > >>>>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > >>>> For additional commands, e-mail: dev-h...@airflow.apache.org > >>>> > >>>> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > >
