Re: Kafka 1.0 upgrade

2018-01-11 Thread Brett Rann
we run a 1.0.1 prerelease in production just fine, but the scale is smaller. 20+ clusters with 3-10 brokers each, each cluster with about 120 topics and about 15k partitions. We have unusual messages sizes, so peaks of around 40k messages, 60MB in, 400MB out, per sec in the largest one. we run a

Kafka 1.0 upgrade

2018-01-11 Thread Tolga Can
Hi We are in the process of upgrading our Kafka cluster from 0.9 to 1.0. We will need a cluster similar to LinkedIn’s busiest clusters as described here , 50K partitions 50 VMs. I could not find enough information about if the latest release