I'm not an expert but I prefer keeping zookeepers on the same hosts as
kafka brokers and mimic each-others topology. The reason is to minimize
the chance of e.g. kafka brokers being able to talk to one another but
zookeepers not, or vice-versa. So, I'd say I /do/ want my kafka broker
and the co-located zookeeper to go down together - for simplicity -
prefer that to some asymmetric failures to debug. This comes from past
experience, albeit using other technologies, when relying on 2 different
clustering mechanism made failures in one but not the other very
difficult to debug.
Also, I think I read this advice somewhere a long time ago (don't recall
where) and it made sense to me (given the prior experience) and we've
never tried a different arrangement.
As to the overheads, I believe it's mostly disk IO and can hopefully be
addressed by separate disks for each but it's never been a bottleneck
for us, so can't really say.
Thanks,
Michał
On 02/06/17 21:47, Mohammed Manna wrote:
Usually, the overhead comes when you have kafka and zookeeper doing the
housekeeping (i.e. Disk IO) on the same server. ZK even suggests that you
should keep their logs on totally different physical machine for better
performance. Furthermore, if a mechanical failure occurs, you might not
want both zookeeper and broker going down together.
Can anyone else chime in for some better points?
On 2 Jun 2017 7:57 pm, "Meghana Narasimhan" <mnarasim...@bandwidth.com>
wrote:
Hi,
What are the pros and cons of setting up Zookeeper on the same server as
the Kafka broker ? Earlier offsets were being written to zookeeper which
was a major overhead but with offsets being written to Kafka now, what
other requirements should be taken into consideration for setting up
Zookeeper on the same server as Kafka vs having a separate zookeeper
cluster ?
Thanks,
Meghana
--
Signature
<http://www.openbet.com/> Michal Borowiecki
Senior Software Engineer L4
T: +44 208 742 1600
+44 203 249 8448
E: michal.borowie...@openbet.com
W: www.openbet.com <http://www.openbet.com/>
OpenBet Ltd
Chiswick Park Building 9
566 Chiswick High Rd
London
W4 5XT
UK
<https://www.openbet.com/email_promo>
This message is confidential and intended only for the addressee. If you
have received this message in error, please immediately notify the
postmas...@openbet.com <mailto:postmas...@openbet.com> and delete it
from your system as well as any copies. The content of e-mails as well
as traffic data may be monitored by OpenBet for employment and security
purposes. To protect the environment please do not print this e-mail
unless necessary. OpenBet Ltd. Registered Office: Chiswick Park Building
9, 566 Chiswick High Road, London, W4 5XT, United Kingdom. A company
registered in England and Wales. Registered no. 3134634. VAT no.
GB927523612
