Each sink needs to have a different priority. If multiple sinks have
same priority, only one of them will be used.
基勇 wrote:
Hello,guys
I tested flume failover feature found failover support only two nodes,
may I ask whether it is so?
config file:
storm@storm01:~/apache-flume-1.5.0-bin/conf$ more flume-sink.properties
#Name the compents on this agent
a1.sources = r1
a1.sinks = k1 k2 k3 k4
a1.channels = c1
#Describe the sinkgroups
a1.sinkgroups = g1 g2
a1.sinkgroups.g1.sinks = k1 k2 k3 k4
a1.sinkgroups.g1.processor.type = failover
a1.sinkgroups.g1.processor.priority.k1 = 10
a1.sinkgroups.g1.processor.priority.k3 = 10
a1.sinkgroups.g1.processor.priority.k4 = 10
a1.sinkgroups.g1.processor.priority.k2 = 5
a1.sinkgroups.g1.processor.maxpenalty = 10000
#a1.sinkgroups.g2.sinks = k3 k4
#a1.sinkgroups.g2.processor.type = load_balance
#a1.sinkgroups.g2.processor.backoff = true
#a1.sinkgroups.g2.processor.selector = round_robin
#Describe/config the source
a1.sources.r1.type = syslogtcp
a1.sources.r1.port = 5140
a1.sources.r1.host = localhost
a1.sources.r1.channels = c1
#Describe the sink
a1.sinks.k1.type = avro
a1.sinks.k1.channel = c1
a1.sinks.k1.hostname = 192.168.220.159
a1.sinks.k1.port = 44411
a1.sinks.k2.type = avro
a1.sinks.k2.channel = c1
a1.sinks.k2.hostname = 192.168.220.159
a1.sinks.k2.port = 44422
a1.sinks.k3.type = avro
a1.sinks.k3.channel = c1
a1.sinks.k3.hostname = 192.168.220.159
a1.sinks.k3.port = 44433
a1.sinks.k4.type = avro
a1.sinks.k4.channel = c1
a1.sinks.k4.hostname = 192.168.220.159
a1.sinks.k4.port = 44444
#Use a channel which butters events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
only k3 and k2 node received the data.
i stop k2 k3. k1 k4 can receive the data;
exception infomation:
2014-07-08 08:10:06,403
(SinkRunner-PollingRunner-FailoverSinkProcessor) [ERROR -
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)]
Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: All sinks failed to process,
nothing left to failover to
at
org.apache.flume.sink.FailoverSinkProcessor.process(FailoverSinkProcessor.java:191)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
2014-07-08 08:10:11,408
(SinkRunner-PollingRunner-FailoverSinkProcessor) [INFO -
org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:206)]
Rpc sink k2: Building RpcClient with hostname: 192.168.220.159, port:
44422
2014-07-08 08:10:11,408
(SinkRunner-PollingRunner-FailoverSinkProcessor) [INFO -
org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:126)]
Attempting to create Avro Rpc client.
2014-07-08 08:10:11,408
(SinkRunner-PollingRunner-FailoverSinkProcessor) [WARN -
org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:620)]
Using default maxIOWorkers
2014-07-08 08:10:11,417
(SinkRunner-PollingRunner-FailoverSinkProcessor) [ERROR -
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)]
Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: All sinks failed to process,
nothing left to failover to
at
org.apache.flume.sink.FailoverSinkProcessor.process(FailoverSinkProcessor.java:191)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
2014-07-08 08:10:16,422
(SinkRunner-PollingRunner-FailoverSinkProcessor) [INFO -
org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:206)]
Rpc sink k3: Building RpcClient with hostname: 192.168.220.159, port:
44433
2014-07-08 08:10:16,423
(SinkRunner-PollingRunner-FailoverSinkProcessor) [INFO -
org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:126)]
Attempting to create Avro Rpc client.
2014-07-08 08:10:16,425
(SinkRunner-PollingRunner-FailoverSinkProcessor) [WARN -
org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:620)]
Using default maxIOWorkers
2014-07-08 08:10:16,431
(SinkRunner-PollingRunner-FailoverSinkProcessor) [ERROR -
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)]
Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: All sinks failed to process,
nothing left to failover to
at
org.apache.flume.sink.FailoverSinkProcessor.process(FailoverSinkProcessor.java:191)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
