Aah, good the dependencies are completely isolated too: https://pypi.org/project/ibmmq/ -- so yeah I am fine with that.
On Fri, 6 Mar 2026 at 22:44, Blain David <[email protected]> wrote: > Indeed, I did the same as with the Microsoft providers, as those are the > one I was familiar with. > > -----Original Message----- > From: Jarek Potiuk <[email protected]> > Sent: 06 March 2026 21:49 > To: Elad Kalif <[email protected]> > Cc: [email protected]; Vikram Koka <[email protected]> > Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider (Hook + > MessageQueueProvider) to Airflow > > EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en deze > niet vertrouwt, klik niet op een link of open geen bijlages. Bij twijfel, > stuur deze e-mail als bijlage naar [email protected]<mailto: > [email protected]>. > > Actually Vikram, Elad - you are right, I was initially under the > impression that -mq is "our message queue" -> but since this is about "IBM > MQ" as a separate product/entity in IBM you are completely right it should > be a separate provider. > > On Fri, Mar 6, 2026 at 7:07 PM Elad Kalif <[email protected]> wrote: > > > If IBM services are independent from one another we should follow how > > we do it on microsoft. > > > > While there is no apache-airflow-providers-microsoft provider. > > We do have: > > apache-airflow-providers-microsoft-mssql > > apache-airflow-providers-microsoft-azure > > and others... azure is not the same division as mssql. > > > > The google provider is a unique case. We have > > apache-airflow-providers-google due to historical reasons (google ads > > has nothing to do with google cloud) and splitting the provider is not > > easy task > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith > > ub.com%2Fapache%2Fairflow%2Fissues%2F15933&data=05%7C02%7Cdavid.blain% > > 40infrabel.be%7C753b70a0859d495498a408de7bc1e34f%7Cb82bc314ab8e4d6fb18 > > 946f02e1f27f2%7C0%7C0%7C639084269845151921%7CUnknown%7CTWFpbGZsb3d8eyJ > > FbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpb > > CIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=I2QJFAYkyMpe8ytFByC3omVDvtz1drY > > rnNL5NTf4SIw%3D&reserved=0 I would also argue that > > apache-airflow-providers-amazon is not that good. > > If someone would like to add integration of Amazon marketplaces APIs > > (retail) that has nothing to do with AWS. > > > > > > On Fri, Mar 6, 2026 at 5:47 PM Vikram Koka via dev > > <[email protected]> > > wrote: > > > >> I am not sure about that Kaxil and Jarek. > >> > >> I understand the prior model of Google and Amazon, but those had > >> teams responsible for all the integrations with that service. > >> In the new governance model, the support burden for an interested > >> individual or individuals seems too high when the team is not a > >> service provider. > >> > >> Why not let individuals or SIs (system integrators) take the > >> "individual service supported" approach? > >> This is similar to the Apache services model. > >> > >> Vikram > >> > >> > >> On Wed, Mar 4, 2026 at 1:20 PM Jarek Potiuk <[email protected]> wrote: > >> > >> > +1 > >> > > >> > On Wed, Mar 4, 2026 at 10:17 PM Kaxil Naik <[email protected]> > wrote: > >> > > >> >> It should just be called apache-airflow-providers-ibm, in future > >> >> folks might want to add more IBM hooks that are not just message > queues. > >> >> > >> >> Similar to how Amazon & Google provider treats PubSub or Kinesis > >> >> > >> >> On Tue, 3 Mar 2026 at 16:57, Vikram Koka via dev < > >> [email protected]> > >> >> wrote: > >> >> > >> >> > Thanks David. > >> >> > I agree with Jason, that a PR is sufficient for this, and an AIP > >> >> > is > >> not > >> >> > required. > >> >> > > >> >> > Looking forward to this, > >> >> > Vikram > >> >> > > >> >> > On Tue, Mar 3, 2026 at 7:28 AM Blain David > >> >> > <[email protected]> > >> >> > wrote: > >> >> > > >> >> > > Hello Jason, > >> >> > > > >> >> > > Thanks for you reply, I've created a draft PR for this provider: > >> >> > > > >> >> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2 > >> >> > > F%2Fgithub.com%2Fapache%2Fairflow%2Fpull%2F62790&data=05%7C02% > >> >> > > 7Cdavid.blain%40infrabel.be%7C753b70a0859d495498a408de7bc1e34f > >> >> > > %7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C6390842698451855 > >> >> > > 21%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjA > >> >> > > uMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0% > >> >> > > 7C%7C%7C&sdata=TcqQfxqzMsZ%2BJME%2BZz1F1J0imMpaU30A5pgvN%2BkhX > >> >> > > KE%3D&reserved=0 > >> >> > > > >> >> > > Kind regards, > >> >> > > David > >> >> > > > >> >> > > -----Original Message----- > >> >> > > From: Zhe-You(Jason) Liu <[email protected]> > >> >> > > Sent: 03 March 2026 10:37 > >> >> > > To: [email protected] > >> >> > > Subject: Re: [DISCUSS] Interest in adding an IBM MQ provider > >> >> > > (Hook > >> + > >> >> > > MessageQueueProvider) to Airflow > >> >> > > > >> >> > > EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent > >> >> > > en > >> deze > >> >> > > niet vertrouwt, klik niet op een link of open geen bijlages. > >> >> > > Bij > >> >> twijfel, > >> >> > > stuur deze e-mail als bijlage naar [email protected]<mailto: > >> >> > > [email protected]>. > >> >> > > > >> >> > > Hi David, > >> >> > > > >> >> > > Thank you for your interest in adding a new > >> >> > > MessageQueueProvider to Airflow! > >> >> > > > >> >> > > I previously opened an issue about adding more providers that > >> support > >> >> > > MessageQueueProvider in the community providers. [1] You’re > >> >> > > more > >> than > >> >> > > welcome to contribute one! > >> >> > > > >> >> > > The only blocker to adding IBM MQ support to > >> >> > > MessageQueueProvider > >> is > >> >> "the > >> >> > > addition of a new IBM provider". We need to follow the > >> >> > > adoption > >> path > >> >> > > (AIP-95), similar to the recent Informatica provider. [2] > >> >> > > > >> >> > > > Whether there is interest in such a provider If yes, whether > >> >> > > > it > >> >> should > >> >> > > > live under apache-airflow-providers-ibm And if we formalize > >> >> > > > this > >> as > >> >> an > >> >> > > > AIP or draft PR > >> >> > > > >> >> > > So, from my perspective, the IBM Hook, Trigger, and > >> >> MessageQueueProvider > >> >> > > would be better placed under the IBM provider, and perhaps you > >> could > >> >> > start > >> >> > > by opening a draft PR and then initiating a voting thread on > >> >> > > the > >> dev > >> >> > > mailing list. > >> >> > > > >> >> > > Thanks! > >> >> > > > >> >> > > [1] > >> >> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2 > >> >> > > F%2Fgithub.com%2Fapache%2Fairflow%2Fissues%2F52712&data=05%7C0 > >> >> > > 2%7Cdavid.blain%40infrabel.be%7C753b70a0859d495498a408de7bc1e3 > >> >> > > 4f%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C63908426984520 > >> >> > > 2381%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwL > >> >> > > jAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C > >> >> > > 0%7C%7C%7C&sdata=zSpuj%2B%2Fx7UAUmDXHM0se6QYtpL6DOHBjGWXmvgAST > >> >> > > lc%3D&reserved=0 > >> >> > > [2] > >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flis > >> ts.apache.org%2Fthread%2Fwsfgh23jm6hkrly4lx1m21ftllqshpgo&data=05%7C0 > >> 2%7Cdavid.blain%40infrabel.be%7C753b70a0859d495498a408de7bc1e34f%7Cb8 > >> 2bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639084269845219729%7CUnknown > >> %7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW > >> 4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Ct8WnY45AQJ > >> fUm8%2BcufmaUsRbYtBAK2tdBCeNanqLK0%3D&reserved=0 > >> >> > > > >> >> > > Best regards, > >> >> > > > >> >> > > Jason > >> >> > > > >> >> > > > >> >> > > On Tue, Mar 3, 2026 at 4:32 PM Blain David < > >> [email protected]> > >> >> > > wrote: > >> >> > > > >> >> > > > Hi all, > >> >> > > > > >> >> > > > At our company we recently implemented an IBM MQ integration > >> >> > > > for Airflow and I would like to gauge interest in > >> >> > > > contributing this > >> as a > >> >> > > > new provider package. > >> >> > > > > >> >> > > > Motivation > >> >> > > > > >> >> > > > With the introduction of event-driven scheduling and the > >> >> > > > MessageQueueProvider abstraction in Airflow, it has become > >> >> > > > significantly easier to trigger DAGs from external message > >> brokers > >> >> (as > >> >> > > > described in Astronomer's guide on event-driven scheduling): > >> >> > > > > >> >> > > > https://www/. > >> >> > > > astronomer.io > >> >> %2Fdocs%2Flearn%2Fairflow-event-driven-scheduling&data=05 > >> >> > > > %7C02%7Cdavid.blain%40infrabel.be > >> >> %7C5d12469000204dfffcaa08de7908795c%7 > >> >> > > > > >> >> Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639081274492894548%7CU > >> >> nkno > >> >> > > > > >> >> wn%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiO > >> >> iJXa > >> >> > > > > >> >> W4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=kxL1VOn > >> >> epYK > >> >> > > > MP8Qjuy9vmkja03KziD8Z5yIo72nuNWE%3D&reserved=0 > >> >> > > > > >> >> > > > Many enterprises still rely heavily on IBM MQ as their > >> >> > > > primary enterprise messaging backbone. However, at the > >> >> > > > moment there is no official Airflow provider supporting IBM MQ. > >> >> > > > > >> >> > > > Our implementation enables: > >> >> > > > > >> >> > > > * An IBMMQHook > >> >> > > > * A MessageQueueProvider implementation for IBM MQ > >> >> > > > * The ability to trigger DAGs from IBM MQ events > >> >> > > > * Standard producer/consumer patterns from within tasks > >> >> > > > > >> >> > > > This allows IBM MQ to function similarly to Kafka, SQS, > >> >> > > > etc., > >> within > >> >> > > > the Airflow event-driven scheduling framework. > >> >> > > > > >> >> > > > Technical Details > >> >> > > > > >> >> > > > The implementation is built on top of the open-source IBM MQ > >> Python > >> >> > > > wrapper: > >> >> > > > > >> >> > > > * IBM MQ Python (ibmmq) library: > >> >> > > > https://gith/ > >> >> > > > ub.com > >> >> %2Fibm-messaging%2Fmq-mqi-python&data=05%7C02%7Cdavid.blain%40in > >> >> > > > frabel.be > >> >> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f > >> >> > > > > >> >> 02e1f27f2%7C0%7C0%7C639081274492911198%7CUnknown%7CTWFpbGZsb3d8eyJ > >> >> FbXB > >> >> > > > > >> >> 0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpb > >> >> CIsI > >> >> > > > > >> >> ldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Wwdlf3w6mSFDca3zF37Coo6qfBSabKf > >> >> jzI7 > >> >> > > > BILZDNmg%3D&reserved=0 > >> >> > > > > >> >> > > > IBM has recently released and documented their modern Python > >> binding > >> >> > > here: > >> >> > > > > >> >> > > > https://comm/ > >> >> > > > unity.ibm.com > >> >> %2Fcommunity%2Fuser%2Fblogs%2Fdylan-goode%2F2025%2F10%2F1 > >> >> > > > > >> >> 6%2Fnew-python-binding-for-ibm-mq&data=05%7C02%7Cdavid.blain%40inf > >> >> rabe > >> >> > > > l.be > >> >> %7C5d12469000204dfffcaa08de7908795c%7Cb82bc314ab8e4d6fb18946f02e1f > >> >> > > > > >> >> 27f2%7C0%7C0%7C639081274492929801%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0 > >> >> eU1h > >> >> > > > > >> >> cGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIl > >> >> dUIj > >> >> > > > > >> >> oyfQ%3D%3D%7C0%7C%7C%7C&sdata=8Q%2Bl9qAltYOkFflC3yjxmu4NI630oCe8L9 > >> >> F3sB > >> >> > > > abTMg%3D&reserved=0 > >> >> > > > > >> >> > > > The hook supports: > >> >> > > > > >> >> > > > * Secure connections (TLS) > >> >> > > > * Queue get/put operations > >> >> > > > * Configurable polling behavior > >> >> > > > * Transaction handling where applicable > >> >> > > > > >> >> > > > The MessageQueueProvider implementation integrates with > >> >> > > > Airflow's event-driven scheduling so that DAGs can be > >> >> > > > triggered based on > >> IBM > >> >> MQ > >> >> > > > messages. > >> >> > > > > >> >> > > > Why this might make sense: > >> >> > > > > >> >> > > > * IBM MQ is still widely used in regulated industries > >> (banking, > >> >> > > > insurance, government). > >> >> > > > * Many enterprises using Airflow also run IBM MQ. > >> >> > > > * This would allow IBM MQ to be a first-class citizen in > >> >> Airflow's > >> >> > > > event-driven ecosystem. > >> >> > > > * The dependency is officially maintained by IBM and open > >> >> source. > >> >> > > > > >> >> > > > I am willing to act as initial maintainer and code owner, of > >> course > >> >> > > > this is purely a proposition. > >> >> > > > > >> >> > > > I would appreciate feedback on: > >> >> > > > > >> >> > > > * Whether there is interest in such a provider > >> >> > > > * If yes, whether it should live under > >> >> apache-airflow-providers-ibm > >> >> > > > * And if we formalize this as an AIP or draft PR > >> >> > > > > >> >> > > > Happy to share a draft implementation through a PR if there > >> >> > > > is > >> >> > interest. > >> >> > > > > >> >> > > > Thanks! > >> >> > > > David > >> >> > > > > >> >> > > > >> >> > > >> >> > >> > > >> > > >
