Hi Sarvesh, That's true that Kafka by itself can't work with MQTT directly. Kafka doesn't support any interfaces (like MQTT) apart from its own general-purpose producer&consumer interface. Also Kafka is quite passive in the terms of getting data in or out, i.e., you need some application to read data from it or write them to it.
On the other hand, there is a component in Kafka ecosystem called Kafka Connect [1]. You can find it in the standard Apache Kafka distribution and run it along with your Kafka cluster. It allows to run specialized pieces of code called connectors that read data from some sources and put them into Kafka or vice versa. There are many of them around and most are open source. Your use case sounds like a perfect fit for using Kafka Connect. You will need Lenses MQTT connectors [2] [3] to run in Kafka Connect, but they are open source and run where you run your Kafka Connect. An alternative would be to just write your own application that reads from Kafka and writes to MQTT or vice versa using Kafka's producers and consumers. I hope this helps! Best, Ivan Yurchenko [1] https://kafka.apache.org/documentation/#connect [2] https://docs.lenses.io/connectors/sink/mqtt.html [3] https://docs.lenses.io/connectors/source/mqtt.html On Tue, 8 Oct 2019 at 19:01, Sarvesh Gupta <sarvesh.gu...@factana.com> wrote: > Hi, > > We are building some IoT driven solution for industries, I am very new to > Kafka and I need some clarifications so please help me out with below given > doubts. > We have use case where we wanted to connect Kafka with MQTT as source of > receiving data, But I am little bit confused on this. I saw lots of blog > and videos where people are connecting Kafka with MQTT using confluent, > lenses and some another third party libraries. So my question is can’t we > directly connect Kafka with MQTT without using any third party dependencies. > And I also wanted to know what are ways to connect Kafka and MQTT. If > there is any way to connect apache Kafka and MQTT directly without using > confluent or other platforms then please give me some idea about that > method. And also we are using python language for out product so if there > is any pythonic way to do this then please let me know. > > > Thanks and Regards, > Sarvesh Gupta > >
