Generally Kafka isn't super great with a giant number of topics. I'd
recommend designing your system around a smaller number than 10k. There's
an upper limit enforced on the total number of partitions by zookeeper
anyway, somewhere around 29k.
I'd recommend having just a single producer per JVM, t
I do not mind the ordering as I have a Timestamp in all my messages and all
my messaged land in a Timeseries database. So I understand that it is
better to have just one Producer instance per JVM and use that to write to
n number of topics. I mean even if I have 10,000 topics, I can just get
away w
On Mon, Apr 25, 2016 at 6:34 AM, Joe San wrote:
> I have an application that is currently running and is using Rx Streams to
> move data. Now in this application, I have a couple of streams whose
> messages I would like to write to a single Kafka topic. Given this, I have
> say Streams 1 to 5 as
I have an application that is currently running and is using Rx Streams to
move data. Now in this application, I have a couple of streams whose
messages I would like to write to a single Kafka topic. Given this, I have
say Streams 1 to 5 as below:
Stream1 - Takes in DataType A Stream2 - Takes in D
