I think the problem is that there is no way to understand the meaning of that config from the docs, so people keep asking over and over again. The docs make iit sound like it is per topic and just says it is the "maximum size before it is deleted" which makes no sense...
-jay On Tuesday, August 27, 2013, Jun Rao wrote: > For the first question, yes. > > For the second one, this is documented in > http://kafka.apache.org/documentation.html#brokerconfigs > > "Note that all per topic configuration properties below have the format of > csv (e.g., "topic1:value1,topic2:value2")." > Thanks, > Jun > > > > On Tue, Aug 27, 2013 at 11:52 AM, Yu, Libo <libo...@citi.com<javascript:;>> > wrote: > > > Hi Jun, > > > > In a previous email thread > > > http://markmail.org/search/?q=kafka+log.retention.bytes#query:kafka%20log.retention.bytes+page:1+mid:qnt4pbq47goii2ui+state:results > > , > > you said log.retention.bytes is for each partition. Could you clarify on > > that? > > > > Say if I have a topic with three partitions. I want to limit the disk > > space to 1Gb for each partition. > > Then log.retention.bytes should be set to 1Gb (not 3Gb). Is that right? > > > > If I want to use log.retention.bytes.per.topic to set the same limit, > > should it be set to 1G or 3G? > > > > Libo > > > > > > -----Original Message----- > > From: Jun Rao [mailto:jun...@gmail.com <javascript:;>] > > Sent: Saturday, August 17, 2013 12:40 AM > > To: users@kafka.apache.org <javascript:;> > > Subject: Re: questtion about log.retention.bytes > > > > log.retention.bytes is for all topics that are not included in > > log.retention.bytes.per.topic (which defines a map of topic -> size). > > > > Currently, we don't have a total size limit across all topics. > > > > Thanks, > > > > Jun > > > > > > On Fri, Aug 16, 2013 at 2:00 PM, Paul Christian > > <pchrist...@salesforce.com <javascript:;>>wrote: > > > > > According to the Kafka 8 documentation under broker configuration. > > > There are these parameters and their definitions. > > > > > > log.retention.bytes -1 The maximum size of the log before deleting it > > > log.retention.bytes.per.topic "" The maximum size of the log for some > > > specific topic before deleting it > > > > > > I'm curious what the first value 'log.retention.bytes' is for if the > > > second one is for per topic logs, because aren't all logs generated > > > per topic? Is this an aggregate value across topics? > > > > > > Related question, is there a parameter for kafka where you can say > > > only hold this much TOTAL data across all topic ( logs/index together > )? > > I.e. > > > our hosts have this much available space and so value > > > log.retention.whatever.aggregate == 75% total disk space. > > > > > >
