Hiya, this doesn’t help answer your question, but as an FYI, Wikimedia has implemented https://github.com/wikimedia/change-propagation to do what you are trying to do: issue HTTP requests (and other things) triggered by incoming messages in Kafka.
On Thu, Aug 23, 2018 at 9:45 AM Andrea Spina <andrea.sp...@radicalbit.io> wrote: > Dear community, > > I'd figure out a kafka connect HTTP/REST sink connector X capable of > achieving PUT actions on an external web service A. I saw some side > projects already implementing something similar [1]. In addition to this > feature, I'd like to log to kafka the answers returning by A so that I can > use it later somewhere else. > > Since I can't find some examples on that (connectors logging to kafka as > "side effect"), my first question is if the former design is something > doable by Kafka Connect and if it well fits the architecture. > > I need this connector because I'm trying to keep it stateless (avoid > storing any state), because AFAIU Kafka Connect technology is not demanded > to keep states along its implementation. Theoretically, if I could store a > state within my kafka connector instance, it'd be way easier achieving my > task. > > So my last question is: is, the fact that connectors should be stateless, > actually true? And why shouldn't I store states through a connector? (I can > eventually open another thread on this if this one is overloaded). > > Any help is much appreciated. Thank you for your work. > > > [1] - https://github.com/llofberg/kafka-connect-rest > -- > *Andrea Spina* > Software Engineer @ Radicalbit Srl > Via Borsieri 41, 20159, Milano - IT >
