Hi, I believe, this line: agent1.sinks.HDFS.hdfs.type = hdfs
should be: agent1.sinks.HDFS.type = hdfs Brock On Mon, Sep 17, 2012 at 5:17 AM, prabhu k <prabhu.fl...@gmail.com> wrote: > Hi Users, > > I have followed the below link for sample text file to HDFS sink using tail > source. > > http://cloudfront.blogspot.in/2012/06/how-to-use-host-escape-sequence-in.html#more > > I have executed flume-ng like as below command. it seems got stuck. and > attached flume.conf file. > > #bin/flume-ng agent -n agent1 -c /conf -f conf/flume.conf > > > flume.conf > ========== > agent1.sources = tail > agent1.channels = MemoryChannel-2 > agent1.sinks = HDFS > > agent1.sources.tail.type = exec > agent1.sources.tail.command = tail -F > /usr/local/flume_dir/flume/flume-1.2.0-incubating-SNAPSHOT/flume_test.txt > agent1.sources.tail.channels = MemoryChannel-2 > > agent1.sources.tail.interceptors = hostint > agent1.sources.tail.interceptors.hostint.type = > org.apache.flume.interceptor.HostInterceptor$Builder > agent1.sources.tail.interceptors.hostint.preserveExisting = true > agent1.sources.tail.interceptors.hostint.useIP = false > > agent1.sinks.HDFS.hdfs.channel = MemoryChannel-2 > agent1.sinks.HDFS.hdfs.type = hdfs > agent1.sinks.HDFS.hdfs.path = hdfs://<hostname>:54310/user > > agent1.sinks.HDFS.hdfs.fileType = dataStream > agent1.sinks.HDFS.hdfs.writeFormat = text > agent1.channels.MemoryChannel-2.type = memory > > > > flume.log > ========== > 12/09/17 15:40:05 INFO lifecycle.LifecycleSupervisor: Starting lifecycle > supervisor 1 > 12/09/17 15:40:05 INFO node.FlumeNode: Flume node starting - agent1 > 12/09/17 15:40:05 INFO nodemanager.DefaultLogicalNodeManager: Node manager > starting > 12/09/17 15:40:05 INFO properties.PropertiesFileConfigurationProvider: > Configuration provider starting > 12/09/17 15:40:05 INFO lifecycle.LifecycleSupervisor: Starting lifecycle > supervisor 10 > 12/09/17 15:40:05 INFO properties.PropertiesFileConfigurationProvider: > Reloading configuration file:conf/flume.conf > 12/09/17 15:40:05 INFO conf.FlumeConfiguration: Processing:HDFS > 12/09/17 15:40:05 INFO conf.FlumeConfiguration: Processing:HDFS > 12/09/17 15:40:05 INFO conf.FlumeConfiguration: Processing:HDFS > 12/09/17 15:40:05 INFO conf.FlumeConfiguration: Processing:HDFS > 12/09/17 15:40:05 INFO conf.FlumeConfiguration: Processing:HDFS > 12/09/17 15:40:05 INFO conf.FlumeConfiguration: Added sinks: HDFS Agent: > agent1 > 12/09/17 15:40:05 WARN conf.FlumeConfiguration: Configuration empty for: > HDFS.Removed. > 12/09/17 15:40:05 INFO conf.FlumeConfiguration: Post-validation flume > configuration contains configuration for agents: [agent1] > 12/09/17 15:40:05 INFO properties.PropertiesFileConfigurationProvider: > Creating channels > 12/09/17 15:40:05 INFO properties.PropertiesFileConfigurationProvider: > created channel MemoryChannel-2 > 12/09/17 15:40:05 INFO nodemanager.DefaultLogicalNodeManager: Starting new > configuration:{ sourceRunners:{tail=EventDrivenSourceRunner: { > source:org.apache.flume.source.ExecSource@c24c0 }} sinkRunners:{} > channels:{MemoryChannel-2=org.apache.flume.channel.MemoryChannel@140c281} } > 12/09/17 15:40:05 INFO nodemanager.DefaultLogicalNodeManager: Starting > Channel MemoryChannel-2 > 12/09/17 15:40:05 INFO nodemanager.DefaultLogicalNodeManager: Starting > Source tail > 12/09/17 15:40:05 INFO source.ExecSource: Exec source starting with > command:tail -F > /usr/local/flume_dir/flume/flume-1.2.0-incubating-SNAPSHOT/flume_test.txt > > Please suggest and help me on this issue. -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
