Actually Vikram, Elad - you are right, I was initially under the impression that -mq is "our message queue" -> but since this is about "IBM MQ" as a separate product/entity in IBM you are completely right it should be a separate provider.
On Fri, Mar 6, 2026 at 7:07 PM Elad Kalif <[email protected]> wrote: > If IBM services are independent from one another we should follow how we > do it on microsoft. > > While there is no apache-airflow-providers-microsoft provider. > We do have: > apache-airflow-providers-microsoft-mssql > apache-airflow-providers-microsoft-azure > and others... azure is not the same division as mssql. > > The google provider is a unique case. We have > apache-airflow-providers-google due to historical reasons (google ads has > nothing to do with google cloud) and splitting the provider is not easy > task https://github.com/apache/airflow/issues/15933 > I would also argue that apache-airflow-providers-amazon is not that good. > If someone would like to add integration of Amazon marketplaces APIs > (retail) that has nothing to do with AWS. > > > On Fri, Mar 6, 2026 at 5:47 PM Vikram Koka via dev <[email protected]> > wrote: > >> I am not sure about that Kaxil and Jarek. >> >> I understand the prior model of Google and Amazon, but those had teams >> responsible for all the integrations with that service. >> In the new governance model, the support burden for an interested >> individual or individuals seems too high when the team is not a service >> provider. >> >> Why not let individuals or SIs (system integrators) take the "individual >> service supported" approach? >> This is similar to the Apache services model. >> >> Vikram >> >> >> On Wed, Mar 4, 2026 at 1:20 PM Jarek Potiuk <[email protected]> wrote: >> >> > +1 >> > >> > On Wed, Mar 4, 2026 at 10:17 PM Kaxil Naik <[email protected]> wrote: >> > >> >> It should just be called apache-airflow-providers-ibm, in future folks >> >> might want to add more IBM hooks that are not just message queues. >> >> >> >> Similar to how Amazon & Google provider treats PubSub or Kinesis >> >> >> >> On Tue, 3 Mar 2026 at 16:57, Vikram Koka via dev < >> [email protected]> >> >> wrote: >> >> >> >> > Thanks David. >> >> > I agree with Jason, that a PR is sufficient for this, and an AIP is >> not >> >> > required. >> >> > >> >> > Looking forward to this, >> >> > Vikram >> >> > >> >> > On Tue, Mar 3, 2026 at 7:28 AM Blain David <[email protected]> >> >> > wrote: >> >> > >> >> > > Hello Jason, >> >> > > >> >> > > Thanks for you reply, I've created a draft PR for this provider: >> >> > > >> >> > > https://github.com/apache/airflow/pull/62790 >> >> > > >> >> > > Kind regards, >> >> > > David >> >> > > >> >> > > -----Original Message----- >> >> > > From: Zhe-You(Jason) Liu <[email protected]> >> >> > > Sent: 03 March 2026 10:37 >> >> > > To: [email protected] >> >> > > Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook >> + >> >> > > MessageQueueProvider) to Airflow >> >> > > >> >> > > EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en >> deze >> >> > > niet vertrouwt, klik niet op een link of open geen bijlages. Bij >> >> twijfel, >> >> > > stuur deze e-mail als bijlage naar [email protected]<mailto: >> >> > > [email protected]>. >> >> > > >> >> > > Hi David, >> >> > > >> >> > > Thank you for your interest in adding a new MessageQueueProvider to >> >> > > Airflow! >> >> > > >> >> > > I previously opened an issue about adding more providers that >> support >> >> > > MessageQueueProvider in the community providers. [1] You’re more >> than >> >> > > welcome to contribute one! >> >> > > >> >> > > The only blocker to adding IBM MQ support to MessageQueueProvider >> is >> >> "the >> >> > > addition of a new IBM provider". We need to follow the adoption >> path >> >> > > (AIP-95), similar to the recent Informatica provider. [2] >> >> > > >> >> > > > Whether there is interest in such a provider If yes, whether it >> >> should >> >> > > > live under apache-airflow-providers-ibm And if we formalize this >> as >> >> an >> >> > > > AIP or draft PR >> >> > > >> >> > > So, from my perspective, the IBM Hook, Trigger, and >> >> MessageQueueProvider >> >> > > would be better placed under the IBM provider, and perhaps you >> could >> >> > start >> >> > > by opening a draft PR and then initiating a voting thread on the >> dev >> >> > > mailing list. >> >> > > >> >> > > Thanks! >> >> > > >> >> > > [1] https://github.com/apache/airflow/issues/52712 >> >> > > [2] >> https://lists.apache.org/thread/wsfgh23jm6hkrly4lx1m21ftllqshpgo >> >> > > >> >> > > Best regards, >> >> > > >> >> > > Jason >> >> > > >> >> > > >> >> > > On Tue, Mar 3, 2026 at 4:32 PM Blain David < >> [email protected]> >> >> > > wrote: >> >> > > >> >> > > > Hi all, >> >> > > > >> >> > > > At our company we recently implemented an IBM MQ integration for >> >> > > > Airflow and I would like to gauge interest in contributing this >> as a >> >> > > > new provider package. >> >> > > > >> >> > > > Motivation >> >> > > > >> >> > > > With the introduction of event-driven scheduling and the >> >> > > > MessageQueueProvider abstraction in Airflow, it has become >> >> > > > significantly easier to trigger DAGs from external message >> brokers >> >> (as >> >> > > > described in Astronomer's guide on event-driven scheduling): >> >> > > > >> >> > > > https://www/. >> >> > > > astronomer.io >> >> %2Fdocs%2Flearn%2Fairflow-event-driven-scheduling&data=05 >> >> > > > %7C02%7Cdavid.blain%40infrabel.be >> >> %7C5d12469000204dfffcaa08de7908795c%7 >> >> > > > >> >> Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639081274492894548%7CUnkno >> >> > > > >> >> wn%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXa >> >> > > > >> >> W4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kxL1VOnepYK >> >> > > > MP8Qjuy9vmkja03KziD8Z5yIo72nuNWE%3D&reserved=0 >> >> > > > >> >> > > > Many enterprises still rely heavily on IBM MQ as their primary >> >> > > > enterprise messaging backbone. However, at the moment there is no >> >> > > > official Airflow provider supporting IBM MQ. >> >> > > > >> >> > > > Our implementation enables: >> >> > > > >> >> > > > * An IBMMQHook >> >> > > > * A MessageQueueProvider implementation for IBM MQ >> >> > > > * The ability to trigger DAGs from IBM MQ events >> >> > > > * Standard producer/consumer patterns from within tasks >> >> > > > >> >> > > > This allows IBM MQ to function similarly to Kafka, SQS, etc., >> within >> >> > > > the Airflow event-driven scheduling framework. >> >> > > > >> >> > > > Technical Details >> >> > > > >> >> > > > The implementation is built on top of the open-source IBM MQ >> Python >> >> > > > wrapper: >> >> > > > >> >> > > > * IBM MQ Python (ibmmq) library: >> >> > > > https://gith/ >> >> > > > ub.com >> >> %2Fibm-messaging%2Fmq-mqi-python&data=05%7C02%7Cdavid.blain%40in >> >> > > > frabel.be >> >> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f >> >> > > > >> >> 02e1f27f2%7C0%7C0%7C639081274492911198%7CUnknown%7CTWFpbGZsb3d8eyJFbXB >> >> > > > >> >> 0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsI >> >> > > > >> >> ldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Wwdlf3w6mSFDca3zF37Coo6qfBSabKfjzI7 >> >> > > > BILZDNmg%3D&reserved=0 >> >> > > > >> >> > > > IBM has recently released and documented their modern Python >> binding >> >> > > here: >> >> > > > >> >> > > > https://comm/ >> >> > > > unity.ibm.com >> >> %2Fcommunity%2Fuser%2Fblogs%2Fdylan-goode%2F2025%2F10%2F1 >> >> > > > >> >> 6%2Fnew-python-binding-for-ibm-mq&data=05%7C02%7Cdavid.blain%40infrabe >> >> > > > l.be >> >> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f02e1f >> >> > > > >> >> 27f2%7C0%7C0%7C639081274492929801%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1h >> >> > > > >> >> cGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIj >> >> > > > >> >> oyfQ%3D%3D%7C0%7C%7C%7C&sdata=8Q%2Bl9qAltYOkFflC3yjxmu4NI630oCe8L9F3sB >> >> > > > abTMg%3D&reserved=0 >> >> > > > >> >> > > > The hook supports: >> >> > > > >> >> > > > * Secure connections (TLS) >> >> > > > * Queue get/put operations >> >> > > > * Configurable polling behavior >> >> > > > * Transaction handling where applicable >> >> > > > >> >> > > > The MessageQueueProvider implementation integrates with Airflow's >> >> > > > event-driven scheduling so that DAGs can be triggered based on >> IBM >> >> MQ >> >> > > > messages. >> >> > > > >> >> > > > Why this might make sense: >> >> > > > >> >> > > > * IBM MQ is still widely used in regulated industries >> (banking, >> >> > > > insurance, government). >> >> > > > * Many enterprises using Airflow also run IBM MQ. >> >> > > > * This would allow IBM MQ to be a first-class citizen in >> >> Airflow's >> >> > > > event-driven ecosystem. >> >> > > > * The dependency is officially maintained by IBM and open >> >> source. >> >> > > > >> >> > > > I am willing to act as initial maintainer and code owner, of >> course >> >> > > > this is purely a proposition. >> >> > > > >> >> > > > I would appreciate feedback on: >> >> > > > >> >> > > > * Whether there is interest in such a provider >> >> > > > * If yes, whether it should live under >> >> apache-airflow-providers-ibm >> >> > > > * And if we formalize this as an AIP or draft PR >> >> > > > >> >> > > > Happy to share a draft implementation through a PR if there is >> >> > interest. >> >> > > > >> >> > > > Thanks! >> >> > > > David >> >> > > > >> >> > > >> >> > >> >> >> > >> >
