Yes. We need those many max partitions as we have a central messaging service and thousands of topics.
On Friday, December 19, 2014, nitin sharma <kumarsharma.ni...@gmail.com> wrote: > hi, > > Few things you have to plan for: > a. Ensure that from resilience point of view, you are having sufficient > follower brokers for your partitions. > b. In my testing of kafka (50TB/week) so far, haven't seen much issue with > CPU utilization or memory. I had 24 CPU and 32GB RAM. > c. 200,000 partitions means around 1MB/week/partition. are you sure you > need so many partitions? > > Regards, > Nitin Kumar Sharma. > > > On Fri, Dec 19, 2014 at 9:12 AM, Achanta Vamsi Subhash < > achanta.va...@flipkart.com <javascript:;>> wrote: > > > > We definitely need a retention policy of a week. Hence. > > > > On Fri, Dec 19, 2014 at 7:40 PM, Achanta Vamsi Subhash < > > achanta.va...@flipkart.com <javascript:;>> wrote: > > > > > > Hi, > > > > > > We are using Kafka for our messaging system and we have an estimate for > > > 200 TB/week in the coming months. Will it impact any performance for > > Kafka? > > > > > > PS: We will be having greater than 2 lakh partitions. > > > > > > -- > > > Regards > > > Vamsi Subhash > > > > > > > > > -- > > Regards > > Vamsi Subhash > > > -- Regards Vamsi Subhash
