A puzzy problem about flume Failover Sink Processor

2017-04-01 Thread hui....@wbkit.com
I have a puzzy problem about flume Failover Sink Processor,Description as below: Enviroment is that we do HA configuration in HDFS,that is we have 2 namenode,1 status for active(host name:n1) and another for standby(hostname:n2). We configure our flume.conf like this: # Flume agent confi

Re: A puzzy problem about flume Failover Sink Processor

2017-04-01 Thread Denes Arvay
Hi, If you are using HDFS HA you don't have to do any special configuration on Flume side. One HDFS Sink is enough, just use the logical nameservice id in the hdfs.path property instead of the host name. See: https://community.cloudera.com/t5/Data-Ingestion-Integration/Flume-HDFS-HA/td-p/29141 R

Re: A puzzy problem about flume Failover Sink Processor

2017-04-01 Thread Chris Horrocks
The following error occurs when your flume agent tries to write to the standby NameNode: "Operation category WRITE is not supported in state standby" What failover mechanism are you using for your NameNodes? -- Chris Horrocks On Sat, Apr 1, 2017 at 11:31 am, hui@wbkit.com wrote: I have a