Hi, My issue looks little similar to your.My exec source stops collecting data after some time.
But I like to know how you could say flume is running but ,your custom source is not running.Let me know how you identify that. Because in running process "ps" i could see a single process i.e. flume agent running.How we identity if source is running or not. Thanks, Saravana On 14 July 2014 13:48, Venkatesh B <venkateshkava...@gmail.com> wrote: > Hello, > > I'm running flume in Cent os 6.3 version. > Apache flume version 1.3 > > I had written many FLUME CUSTOM SOURCES, > All sources are working fine, except two sources. > > When I start flume i.e., flume-ng ...... command from shell it is working > fine, flume is collecting data continuously for 6hours I tested it for many > times about 10 times. > > But when I start flume from java program, i.e., In one ".sh" file I will > write "flume-ng --conf path to conf folder -f flume configuration file path > -n agent" command, And from java program I will run this .sh file by > calling Runtime.getRuntime().exec(.sh file path).waitFor(). > > > In flume custom source I have written logic for collecting data from web > and then I will write that data to channel processor. Here sink is hdfs. > > Here the PROBLEM is flume is collecting data about 7 minutes and then the > not collecting when I start flume from java program. > > When I search the running processes it is showing that flume is running > but my custom sources is not running. > I implemented logging for the flume custom source by logging what I came > to know is that flume custom source is not running at the same time no > error is caught. > > Here flume custom source is stopping without any intimation of error. > > What is the difference between running flume directly from shell and > through java program? > Why it is stopping did anyone face this problem? > > What actually my flume custom source code looks is > in start() method I'm creating a thread and this thread will collect data. > > > > Thanking you.. > > >
