RE: File channels creating many large files

2014-11-10 Thread Hari Shreedharan
elcome VSRE emails. Learn more at http://vsre.info/ > > From: Hari Shreedharan [mailto:hshreedha...@cloudera.com] > Sent: 10 November 2014 09:15 > To: user@flume.apache.org > Cc: user@flume.apache.org > Subject: RE: File channels creating many large file

RE: File channels creating many large files

2014-11-10 Thread Needham, Guy
3362 I welcome VSRE emails. Learn more at http://vsre.info/ From: Hari Shreedharan [mailto:hshreedha...@cloudera.com] Sent: 10 November 2014 09:15 To: user@flume.apache.org Cc: user@flume.apache.org Subject: RE: File channels creating many large files That value i

RE: File channels creating many large files

2014-11-10 Thread Hari Shreedharan
t; I welcome VSRE emails. Learn more at http://vsre.info/ > > From: Hari Shreedharan [mailto:hshreedha...@cloudera.com] > Sent: 07 November 2014 17:59 > To: user@flume.apache.org > Cc: user@flume.apache.org > Subject: Re: File channels creating many larg

RE: File channels creating many large files

2014-11-10 Thread Needham, Guy
ject: Re: File channels creating many large files Flume will leave at least 2 files per data directory. Once you have enough events to cause 2 files to be created, there will be at least 2 per dir. You can use maxFileSize parameter to control the size of these files. Thanks, Hari On Fri, Nov 7, 20

Re: File channels creating many large files

2014-11-07 Thread Hari Shreedharan
Flume will leave at least 2 files per data directory. Once you have enough events to cause 2 files to be created, there will be at least 2 per dir. You can use maxFileSize parameter to control the size of these files. Thanks, Hari On Fri, Nov 7, 2014 at 10:25 AM, Jeff Lord wrote: > Guy, > W

Re: File channels creating many large files

2014-11-07 Thread Jeff Lord
Guy, What version of flume is this? -Jeff On Fri, Nov 7, 2014 at 1:19 AM, Needham, Guy wrote: > Hi all, > > I have a configuration with a file channel configured such that: > > a1.channels.ch1.type = file > a1.channels.ch1.checkpointDir = /hadoop/user/flume/channels/checkpoint > a1.channels.c

File channels creating many large files

2014-11-07 Thread Needham, Guy
Hi all, I have a configuration with a file channel configured such that: a1.channels.ch1.type = file a1.channels.ch1.checkpointDir = /hadoop/user/flume/channels/checkpoint a1.channels.ch1.dataDirs = /hadoop/user/flume/channels/data a1.channels.ch1.capacity = 10 a1.channels.ch1.transactionCapa