On Mon, Apr 25, 2016 at 6:34 AM, Joe San <codeintheo...@gmail.com> wrote:
> I have an application that is currently running and is using Rx Streams to > move data. Now in this application, I have a couple of streams whose > messages I would like to write to a single Kafka topic. Given this, I have > say Streams 1 to 5 as below: > > Stream1 - Takes in DataType A Stream2 - Takes in DataType B and so on > > Where these Streams are Rx Observers. All these data types that I get out > of the stream are converted to a common JSON structure. I want this JSON > structure to be pushed to a single Kafka topic. > > Now the questions are: > > 1. > > Should I create one KafkaProducer for each of those Streams or rather Rx > Observer instances? > A single producer instance is fine. In fact, it may be better since you share TCP connections and requests to produce data can be batched together. > 2. > > What happens if multiple threads using its own instance of a > KafkaProducer to write to the same topic? > They can all write to the same topic, but their data will be arbitrarily interleaved since there's no ordering guarantee across these producers. -- Thanks, Ewen
