Thanks Dave, Seems good option!
Thanks, Amit On Wed, Jul 20, 2016 at 10:24 PM, Tauzell, Dave < dave.tauz...@surescripts.com> wrote: > For fault -tolerance you want the replicated partitions to be on different > physical servers. I think a better set is to have 3 brokers, 9 partitions > and a replication factor of 2. This will ensure that the replicated data > is always on a different physical hosts. > > -Dave > > Dave Tauzell | Senior Software Engineer | Surescripts > O: 651.855.3042 | www.surescripts.com | dave.tauz...@surescripts.com > Connect with us: Twitter I LinkedIn I Facebook I YouTube > > > -----Original Message----- > From: Amit K [mailto:amitk....@gmail.com] > Sent: Wednesday, July 20, 2016 11:40 AM > To: users@kafka.apache.org > Subject: Re: Regarding kafka partition and replication > > Thanks for reply. > > The infrastructure, which I mentioned, is already in place and is getting > used so can not alter it. > > But given that, if I have 9 partitions with replication factor of 2, will > that help me to have a good fault tolerant and optimal, in regard with > hardware use, system? > > Thanks, > Amit > > On Tue, Jul 19, 2016 at 6:48 PM, Tauzell, Dave < > dave.tauz...@surescripts.com > > wrote: > > > Having multiple brokers on the same node has a couple of problems for > > a production installation: > > > > 1. You'll have multiple brokers contending for disk and memory > > resources 2. You could have your partitions replicated to the same > > node which means if that node fails you would lose data. > > > > I think you are better off having 3 nodes with 3 brokers. You can keep > > with 9 partitions in case you want to add physical nodes in the future > > and have a replication factor of 2 or 3. > > > > -Dave > > > > Dave Tauzell | Senior Software Engineer | Surescripts > > O: 651.855.3042 | www.surescripts.com | dave.tauz...@surescripts.com > > Connect with us: Twitter I LinkedIn I Facebook I YouTube > > > > > > -----Original Message----- > > From: Amit K [mailto:amitk....@gmail.com] > > Sent: Monday, July 18, 2016 8:55 PM > > To: users@kafka.apache.org > > Subject: Regarding kafka partition and replication > > > > Hi, > > > > I have 3 nodes, each with 3 brokers, Kafka cluster along with 3 > > zookeeper cluster. So total 9 brokers spread across 3 different > > machines. I am adhered to Kafka 0.9. > > > > In order to optimally use the infrastructure for 2 topics (as of now, > > is not expected to grow drastically in near future), I am thinking of > > having 9 partitions with 3 (or 6?) replication factor. Will this help > > me having good distribution of partitions and replicas across brokers? > > System does not have hugh load (<50 requests/sec of less than 1 kb > > load each) as of now and is neither expected to get higher load than > this. > > > > If this replication and partition does not help, please suggest a > > better topic partition and replication strategy. > > > > Also please guide me with any articles or documents about setting up > > multi node Kafka cluster in regard to partition, replication, general > > properties to be used (a kind of good practice etc.) > > > > Thanks, > > Amit > > This e-mail and any files transmitted with it are confidential, may > > contain sensitive information, and are intended solely for the use of > > the individual or entity to whom they are addressed. If you have > > received this e-mail in error, please notify the sender by reply > > e-mail immediately and destroy all copies of the e-mail and any > attachments. > > > This e-mail and any files transmitted with it are confidential, may > contain sensitive information, and are intended solely for the use of the > individual or entity to whom they are addressed. If you have received this > e-mail in error, please notify the sender by reply e-mail immediately and > destroy all copies of the e-mail and any attachments. >
