Hi sarvana, My flume agent is surely running, when I see in running processes ps -ef it is showing flume agent process is running.But my custom source is not running. I have implemented logging, if flume custom source is running then it will write to log files. I have written many log statements I know where it is stopping, but not know why it is stopping.
My custom source works fine as when I run from shell directly but not from java program, then custom source is running for many hours. And also flume is working fine I have written other custom sources before and they are running for days continuously when I start from java program. Here I'm not understanding what is happening internally, when I start to run flume through java program then only my custom source stops working where as when I run through shell then it is working fine. Is your case is also similar. How are you starting flume, either by java program or by shell? On Mon, Jul 14, 2014 at 2:07 PM, SaravanaKumar TR <saran0081...@gmail.com> wrote: > Hi, > > My issue looks little similar to your.My exec source stops collecting data > after some time. > > But I like to know how you could say flume is running but ,your custom > source is not running.Let me know how you identify that. > > Because in running process "ps" i could see a single process i.e. flume > agent running.How we identity if source is running or not. > > Thanks, > Saravana > > > On 14 July 2014 13:48, Venkatesh B <venkateshkava...@gmail.com> wrote: > >> Hello, >> >> I'm running flume in Cent os 6.3 version. >> Apache flume version 1.3 >> >> I had written many FLUME CUSTOM SOURCES, >> All sources are working fine, except two sources. >> >> When I start flume i.e., flume-ng ...... command from shell it is >> working fine, flume is collecting data continuously for 6hours I tested it >> for many times about 10 times. >> >> But when I start flume from java program, i.e., In one ".sh" file I will >> write "flume-ng --conf path to conf folder -f flume configuration file path >> -n agent" command, And from java program I will run this .sh file by >> calling Runtime.getRuntime().exec(.sh file path).waitFor(). >> >> >> In flume custom source I have written logic for collecting data from web >> and then I will write that data to channel processor. Here sink is hdfs. >> >> Here the PROBLEM is flume is collecting data about 7 minutes and then the >> not collecting when I start flume from java program. >> >> When I search the running processes it is showing that flume is running >> but my custom sources is not running. >> I implemented logging for the flume custom source by logging what I came >> to know is that flume custom source is not running at the same time no >> error is caught. >> >> Here flume custom source is stopping without any intimation of error. >> >> What is the difference between running flume directly from shell and >> through java program? >> Why it is stopping did anyone face this problem? >> >> What actually my flume custom source code looks is >> in start() method I'm creating a thread and this thread will collect data. >> >> >> >> Thanking you.. >> >> >> >
