Hi,
I'm trying to send events from flume agent to flume central collector using
avro sink/source pair.
This is the avro sink configuration, I have two sinks to drain the channel:
agent.sinks.AvroSink1-1.type = avro
agent.sinks.AvroSink1-1.channel = fileChannel
agent.sinks.AvroSink1-1.hostname = X.X.X.X
agent.sinks.AvroSink1-1.port = 45451
agent.sinks.AvroSink1-1.connect-timeout = 60000
agent.sinks.AvroSink1-1.request-timeout = 60000
agent.sinks.AvroSink1-1.batch-size = 1000
agent.sinks.AvroSink1-1.compression-type=deflate
agent.sinks.AvroSink1-1.compression-level=9
agent.sinks.AvroSink1-2.type = avro
agent.sinks.AvroSink1-2.channel = fileChannel
agent.sinks.AvroSink1-2.hostname = X.X.X.X
agent.sinks.AvroSink1-2.port = 45451
agent.sinks.AvroSink1-2.connect-timeout = 60000
agent.sinks.AvroSink1-2.request-timeout = 60000
agent.sinks.AvroSink1-2.batch-size = 1000
agent.sinks.AvroSink1-2.compression-type=deflate
agent.sinks.AvroSink1-2.compression-level=9
However I see many connectivity timeout exceptions although when I try to
connect directly using telnet there is no problem.
Anything I can change or check to improve this?
30 Sep 2013 01:32:56,600 ERROR
[SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.SinkRunner$PollingRunner.run:160) - Unable to deliver
event. Exception follows.
org.apache.flume.EventDeliveryException: Failed to send events
at
org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:382)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:724)
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient {
host: X.X.X.X, port: 45451 }: Failed to send batch
at
org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:294)
at
org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:366)
... 3 more
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient {
host: X.X.X.X: 45451 }: Handshake timed out after 60000ms
at
org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:338)
at
org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:282)
... 4 more
Caused by: java.util.concurrent.TimeoutException
at
java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258)
at java.util.concurrent.FutureTask.get(FutureTask.java:119)
at
org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:336)
... 5 more
30 Sep 2013 01:32:57,212 INFO
[SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.sink.AbstractRpcSink.createConnection:205) - Rpc sink
AvroSink1-1: Building RpcClient with hostname: X.X.X.X, port: 45451
30 Sep 2013 01:32:57,213 INFO
[SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.sink.AvroSink.initializeRpcClient:126) - Attempting to
create Avro Rpc client.
30 Sep 2013 01:32:58,136 ERROR
[SinkRunner-PollingRunner-DefaultSinkProcessor]
(org.apache.flume.SinkRunner$PollingRunner.run:160) - Unable to deliver
event. Exception follows.
org.apache.flume.EventDeliveryException: Failed to send events
at
org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:382)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:724)
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient {
host: X.X.X.X, port: 45451 }: Failed to send batch
at
org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:294)
at
org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:366)
... 3 more
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient {
host: X.X.X.X, port: 45451 }: Handshake timed out after 60000ms
at
org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:338)
at
org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:282)
... 4 more
Caused by: java.util.concurrent.TimeoutException
at
java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258)
at java.util.concurrent.FutureTask.get(FutureTask.java:119)
at
org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:336)
... 5 more
