Thank you Hari. This is very useful information.
Let me try that scenario where I will send more data to create log-3. Thanks Madhu Munagala On Apr 11, 2013, at 2:57 PM, Hari Shreedharan <[email protected]> wrote: > The File Channel deletes the files from which all events have been removed at > the time of a checkpoint. But, the channel will keep 2 log files per data > directory even if all its events are taken out. Once all events from log-1, > log-2 are taken out and then events get written to log-3, log-1 will be > deleted at the next checkpoint. Unless a log-3 is created, both log-1 and > log-2 will not be deleted. > > > > Hari > > -- > Hari Shreedharan > > On Thursday, April 11, 2013 at 12:13 PM, Madhu Gmail wrote: > >> Hello, >> >> I have not heard from anyone. so just want make sure I have explained the >> issue correctly. >> >> I think this is a common problem for everyone who uses it flume. >> >> when flume sink consumes the log event from file channel, what will happen >> to the data that is committed to local disk under data directory. >> >> will it grow indefinitely like log-1, log-2, log-3.....and so on ??? >> >> do I have to write script to remove the data from data directory ?? >> >> >> >> Madhu Munagala >> (214)679-2872 >> >> On Apr 11, 2013, at 11:52 AM, Madhu Gmail <[email protected]> wrote: >> >>> Hello, >>> >>> How to clean up the data in file channel data folder. After the log events >>> are processed by the sink, I still see the log-1 and log-2 shows 1.6GB and >>> 1.2GB. >>> >>> once the log events are processed by the sink, the channel should not have >>> any data in data directory under file-channel ....?? >>> >>> >>> Madhu Munagala >>> (214)679-2872 >
