Re: Optimal number of partitions for topic

2015-11-17 Thread Chen Song
factors and provide a few simple > formulas. More... | > > | | > > | View on blog.confluent.io | Preview by Yahoo | > > | | > > | | > > > > Naidu Saladi > > > > From: Daniel Compton > > To: users@kafka.apache.org > > Sent: Wedne

Re: Optimal number of partitions for topic

2015-05-21 Thread Daniel Compton
s to > explain a few important determining factors and provide a few simple > formulas. More... | > > | | > > | View on blog.confluent.io | Preview by Yahoo | > > | | > > | | > > > > Naidu Saladi > > > > From: Daniel Compton > > T

Re: Optimal number of partitions for topic

2015-05-21 Thread Carles Sistare
tant determining factors and provide a few simple formulas. More... | > | | > | View on blog.confluent.io | Preview by Yahoo | > | | > | | > > Naidu Saladi > > From: Daniel Compton > To: users@kafka.apache.org > Sent: Wednesday, May 20, 2015 8:21

Re: Optimal number of partitions for topic

2015-05-20 Thread Saladi Naidu
explain a few important determining factors and provide a few simple formulas. More... | | | | View on blog.confluent.io | Preview by Yahoo | | | |   |   Naidu Saladi From: Daniel Compton To: users@kafka.apache.org Sent: Wednesday, May 20, 2015 8:21 PM Subject: Re: Optimal number of

Re: Optimal number of partitions for topic

2015-05-20 Thread Daniel Compton
One of the beautiful things about Kafka is that it uses the disk and OS disk caching really efficiently. Because Kafka writes messages to a contiguous log, it needs very little seek time to move the write head to the next point. Similarly for reading, if the consumers are mostly up to date with th

Re: Optimal number of partitions for topic

2015-05-20 Thread Manoj Khangaonkar
With knowing the actual implementation details, I would get guess more partitions implies more parallelism, more concurrency, more threads, more files to write to - all of which will contribute to more CPU load. Partitions allow you to scale by partitioning the topic across multiple brokers. Parti