Re: Is Zookeeper already providing load balancing for Kafka brokers?

2013-07-11 Thread Ryan Chan
Hi, So, is it possible to configure the weighting? I believe this need to be done on the zookeeper side, can you give me some hints? Thanks. On Thu, Jul 11, 2013 at 11:50 PM, Jun Rao wrote: > Yes. > > Thanks, > > Jun > > > On Wed, Jul 10, 2013 at 10:55 PM, Ryan C

Is Zookeeper already providing load balancing for Kafka brokers?

2013-07-10 Thread Ryan Chan
We are already using "zk.connect" to connect zookeeper and registered multiple brokers (same topic/partitions), so when a consumer request ZK, is load balancing already done? Thanks

Re: How to create the initial zookeeper chroot path for zk.connect?

2013-04-22 Thread Ryan Chan
11:10 PM, Scott Clasen wrote: > Since There is only 1 chroot for a zk cluster, if you specified for each > server there would be a potential for error/mismatch > > Things would probably go really bad if you had mismatched chroots :) > > Sent from my iPhone > > On Apr 21

Re: How to create the initial zookeeper chroot path for zk.connect?

2013-04-21 Thread Ryan Chan
gt; > Not > > Zk1:2181/kafka,zk2:2181/Kafka,zk3:2181/Kafka > > > Sent from my iPhone > > On Apr 20, 2013, at 9:03 AM, Neha Narkhede > wrote: > > > Please file a bug and mention the Kafka and zookeeper versions used for > the > > test. > >

Re: How to create the initial zookeeper chroot path for zk.connect?

2013-04-20 Thread Ryan Chan
if you use all 3 zookeeper servers will creating and reading > the node, do you still see the problem ? > > zkCli.sh -server zookeeper1:2181,zookeeper2:2181,zookeeper3:2181 > create /testkafka > ls / > > Thanks > Neha > > On Fri, Apr 19, 2013 at 8:55 PM, Ryan Chan wrote: >

Re: How to create the initial zookeeper chroot path for zk.connect?

2013-04-19 Thread Ryan Chan
will make a > > ticket here to get that done. > > On Apr 18, 2013 10:47 PM, "Ryan Chan" wrote: > > > >> Yes, using the latest Kafka 0.7.2, just tried to reproduce again > >> > >> 1. Install a single node Kafka, three nodes zookeeper instances

Re: How to create the initial zookeeper chroot path for zk.connect?

2013-04-18 Thread Ryan Chan
Utils.java:48) at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:35) .. Any idea? On Fri, Apr 19, 2013 at 5:32 AM, Neha Narkhede wrote: > That is odd. Is it reproducible ? > > On Wed, Apr 17, 2013 at 8:03 PM, Ryan Chan wrote: > > Hi, > > > > Yes, I can see the new path exists by

Re: How to create the initial zookeeper chroot path for zk.connect?

2013-04-17 Thread Ryan Chan
Hi, Yes, I can see the new path exists by using the command "ls /" On Thu, Apr 18, 2013 at 1:29 AM, Neha Narkhede wrote: > After creating the path, did you get a chance to confirm that it got > created correctly ? > > Thanks, > Neha > > On Wed, Apr 17, 20

How to create the initial zookeeper chroot path for zk.connect?

2013-04-17 Thread Ryan Chan
Sorry as I am new to Kafka/ZK, but the instruction seems not clear.. In the config, it was said that we can use a chroot-ed ZK ( http://kafka.apache.org/configuration.html), e.g. localhost:2181/kafka But when I start the Kafka using the setting, it was showing.. java.lang.IllegalArgumentExceptio

Re: Local forwarder for performance & reliability

2013-03-15 Thread Ryan Chan
Hi, On Fri, Mar 15, 2013 at 1:18 PM, Neha Narkhede wrote: > Ryan, > > Have you tried configuring the num.retries option on the log4j producer ? > When there is a temporary network glitch, it will retry sending the > messages instead of losing them. > Memory is limited and this is our concern. (W

Local forwarder for performance & reliability

2013-03-14 Thread Ryan Chan
Hi, We are currently using the Log4j appender to write to the Kafka (on machine on another datacenter), but problem occur when the network is interrupted as the message is stored in memory. Are there any local forwarder/client or similar solutions that can be installed on the localhost and take c