Re: Overheads of a large number of partitions for a topic

2014-12-15 Thread Joel Koshy
(inline) On Mon, Dec 15, 2014 at 11:45:07AM -0800, Rajiv Kurian wrote: > I currently have a topic with 1024 partitions. I know it's kind of going > past the recommended limits, but I kept it like that because I am moving a > legacy system to kafka and it has a 1024 parallel partitions. I wanted to

Overheads of a large number of partitions for a topic

2014-12-15 Thread Rajiv Kurian
I currently have a topic with 1024 partitions. I know it's kind of going past the recommended limits, but I kept it like that because I am moving a legacy system to kafka and it has a 1024 parallel partitions. I wanted to understand the costs of having so many partitions a little bit more though. I