Re: Kafka without Storm/Spark

2016-07-01 Thread Michael Noll
> Because i know that by using Storm, you can guarantee the messages (depending on the type of the Topology) > such as exactly once, at least once. If i simply use kafka consumer and another producer to forward the > messages, could the data tranfer completely be guaranteed as well? Addendum: If

Re: Kafka without Storm/Spark

2016-07-01 Thread Michael Noll
Numan, you may also want to take a look at Kafka Streams, which is a new stream processing library that's included in Apache Kafka since version 0.10. Kafka Streams is definitely more convenient and quicker to implement than the "normal" Kafka producer/consumer clients. Also, Kafka Streams does n

Re: Kafka without Storm/Spark

2016-07-01 Thread numangoceri
Hi, Thanks for your answer. I meant actually if we can verify the data reliability without using Storm or Spark. Because i know that by using Storm, you can guarantee the messages (depending on the type of the Topology) such as exactly once, at least once. If i simply use kafka consumer and ano

Re: Kafka without Storm/Spark

2016-07-01 Thread Rakesh Vidyadharan
Definitely. You can read off kafka using the samples shown in KafkaConsumer javadoc, transform if necessary and publish to the destination topic. On 01/07/2016 03:24, "numan goceri" wrote: >Hello everyone, >I've a quick question:I'm using Apache Kafka producer to write the messages >into a