Hi, I kinda doubt whether I make it as an ensemble, since it shows
root@DO-mq-dev:/etc/zookeeper/conf# zkServer.sh status JMX enabled by default Using config: /etc/zookeeper/conf/zoo.cfg Mode: standalone Mode is standalone instead of something else, here is my zoo.cfg, I did follow the instruction to config it # http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. dataDir=/var/lib/zookeeper # Place the dataLogDir to a separate physical disc for better performance # dataLogDir=/disk2/zookeeper # the port at which the clients will connect clientPort=2181 # specify all zookeeper servers # The fist port is used by followers to connect to the leader # The second one is used for leader election DO-mq-dev.1=10.100.70.128:2888:3888 pof-kstorm-dev1.2=10.100.70.28:2888:3888 pof-kstorm-dev2.3=10.100.70.29:2888:3888 # To avoid seeks ZooKeeper allocates space in the transaction log file in # blocks of preAllocSize kilobytes. The default block size is 64M. One reason # for changing the size of the blocks is to reduce the block size if snapshots # are taken more often. (Also, see snapCount). #preAllocSize=65536 # Clients can submit requests faster than ZooKeeper can process them, # especially if there are a lot of clients. To prevent ZooKeeper from running # out of memory due to queued requests, ZooKeeper will throttle clients so that # there is no more than globalOutstandingLimit outstanding requests in the # system. The default limit is 1,000.ZooKeeper logs transactions to a # transaction log. After snapCount transactions are written to a log file a # snapshot is started and a new transaction log file is started. The default # snapCount is 10,000. #snapCount=1000 # If this option is defined, requests will be will logged to a trace file named # traceFile.year.month.day. #traceFile= # Leader accepts client connections. Default value is "yes". The leader machine # coordinates updates. For higher update throughput at thes slight expense of # read throughput the leader can be configured to not accept clients and focus # on coordination. leaderServes=yes # Enable regular purging of old data and transaction logs every 24 hours autopurge.purgeInterval=24 autopurge.snapRetainCount=5 And myid in dataDir is At 10.100.70.128 /var/lib/zookeeper contains 1 At 10.100.70.28 /var/lib/zookeeper contains 2 At 10.100.70.29 /var/lib/zookeeper contains 3 I did make myid as 1, 2, 3 corresponding 3 nodes before, but seeing something make such myid, it might be more accurate. Is there anything wrong or missing to not able to make it an ensemble? thanks Alec On Thu, Oct 9, 2014 at 12:06 PM, Guozhang Wang <wangg...@gmail.com> wrote: > Sa, > > Usually you would not want to set up kafka brokers at the same machines > with zk nodes, as that will add depending failures to the server cluster. > > Back to your original question, it seems your zk nodes do not form an > ensemble, since otherwise their zk data should be the same. > > Guozhang > > On Thu, Oct 9, 2014 at 11:37 AM, Sa Li <sal...@gmail.com> wrote: > > > Hi, All > > > > I setup a 3-node kafka cluster on top of 3-node zk ensemble. Now I > launch 1 > > broker on each node, the brokers will be randomly distributed to zk > > ensemble, see > > > > DO-mq-dev.1 > > [zk: localhost:2181(CONNECTED) 1] ls /brokers/ids > > [0, 1] > > pof-kstorm-dev1.2 > > [zk: localhost:2181(CONNECTED) 1] ls /brokers/ids > > [] > > pof-kstorm-dev2.3 > > [zk: localhost:2181(CONNECTED) 1] ls /brokers/ids > > [2] > > > > which means zk1 hosts 2 brokers, zk3 hosts 1 brokers, that will raise a > > problem, that I am unable to create a topic with replications, say 3, it > > will throw such exceptions > > > > Error while executing topic command replication factor: 3 larger than > > available brokers: 0 > > > > Is there any ways that I can create a topic which can be replicated > > throughout entire zk ensemble as I know we will have to introduce more > than > > 1 broker in single zk Server if we want to be able to create replicated > > topics/ > > > > thanks > > > > -- > > > > Alec Li > > > > > > -- > -- Guozhang > -- Alec Li
