I'm using CDH3 flume, 0.9.4-cdh3u4.
I've configured multiple masters (3 of them), but nodes only appear to
communicate with a single one of them at any given time. Which results in a
very strange split-brain display when I look at the status of nodes on any
given master. One master will show two lost nodes and one idle. The next
will show one idle and one lost. The last will show a single idle node.
Taken together, I see all three of my nodes idle as I'd expect. But each
node only appears on a single master.
This cannot be the normal behavior, as it makes no sense.
Below is my config. What am I missing? Where have I gone wrong?
<flume-site.xml>
<configuration>
<property>
<name>flume.master.servers</name>
<value>flume001,flume002,flume003</value>
<description>This is the address for the config servers status server
(http)
</description>
</property>
<property>
<name>flume.collector.event.host</name>
<value>flume001</value>
<description>This is the host name of the default "remote"
collector.
</description>
</property>
<property>
<name>flume.collector.port</name>
<value>35853</value>
<description>This default tcp port that the collector listens to in
order to receive events it is collecting.
</description>
</property>
<property>
<name>flume.report.server.port</name>
<value>45678</value>
</property>
<property>
<name>flume.master.admin.port</name>
<value>35873</value>
</property>
<property>
<name>flume.master.zk.use.external</name>
<value>true</value>
</property>
<property>
<name>flume.master.zk.servers</name>
<value>zk001:2181,zk002:2181,zk003:2181,zk004:2181,zk005:2181</value>
</property>
</configuration>
<flume-conf.xml>
...
<property>
<name>flume.master.servers</name>
<value>flume001,flume002,flume003</value>
<description>A comma-separated list of hostnames, one for each
machine in the Flume Master.
</description>
</property>
<property>
<name>flume.master.store</name>
<value>zookeeper</value>
<description>How the Flume Master stores node configurations. Must
be either 'zookeeper' or 'memory'.</description>
</property>
<property>
<name>flume.master.serverid</name>
<value>1</value>
<description>The unique identifier for a machine in a
Flume Master ensemble. Must be different on every
master instance.</description>
</property>
...
(all other values default)
---------------------------------------
Cameron Nicholson
Systems Engineer
Topsy Labs
