Thanks Jacob. I'm using kafka as a distributed queue and my data pipeline is a several components which connected together via a stream abstraction. each component has a input and output stream. Basically the source of this pipeline can be anything and the output can be anything to.
On Tue, Mar 13, 2018 at 1:27 PM, Jacob Sheck <shec0...@gmail.com> wrote: > Sameer when you say that you need to "consume from and produce to a topic" > to me that seems like a good fit for Kafka Streams. Streaming your data > out of Kafka for a transform and back in has some fundamental costs and > operational challenges involved. Are the events in your stream stateless? > If it isn't stateless streams will ensure a consistent playback of events > if needed. Without knowing more about your pipeline it is hard to make > recommendations. Are you possibly using Kafka as a distributed queue? > > On Tue, Mar 13, 2018 at 6:29 AM Sameer Rahmani <lxsame...@gmail.com> > wrote: > > > Hi folks, > > I need to consume from and produce to a topic. I have my own data > pipeline > > to process the data. > > So I was wondering beside the stores and StreamDSL what does Kafka > Streams > > brings to the table > > that might be useful to me ? > > >
