Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

http://kafka.apache.org/documentation.html#configuration now has the following entry. log.retention.{minutes,hours} Thanks, Jun On Thu, Aug 21, 2014 at 7:44 AM, François Langelier wrote: > Thanks Jun, > > I don't see changes in the documentation :( > Just to be sure, > > We can either use "*

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

Thanks Jun, I don't see changes in the documentation :( Just to be sure, We can either use "*log.retention.hours*" (giving the number of *hours* you want the retention to last) or "*log.retention.minutes*" (giving the number of *minutes* you want the retention to last) for all the topics (in serv

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

Added to the documentation. Topic level retentions is called retention.ms and can be found in http://kafka.apache.org/documentation.html#configuration Thanks, Jun On Fri, Aug 15, 2014 at 12:32 PM, François Langelier wrote: > UP! Can someone add the log.retention.hours in the documentation? >

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

up? François Langelier Étudiant en génie Logiciel - École de Technologie Supérieure Capitaine Club Capra VP-Communication - CS Games 2014 Jeux de Génie 2011 à 2014 Magistrat Fraternité du Piranha

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

UP! Can someone add the log.retention.hours in the documentation? And is the retention.hours or retention.minutes available for the by-topic configuration? François Langelier Étudiant en génie Logiciel - École de Technologie Supérieure Capitaine Club Capra

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

No worries, its working. My deployment was flawed and it was running 0.8.0, while I was under the impression that its running 0.8.1.1. After going to proper 0.8.1.1, the parameter started working properly. On 7/10/14, 2:18 PM, "Virendra Pratap Singh" wrote: >Has someone really used : log.retenti

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

Has someone really used : log.retention.minutes in kafka 0.8.1.1. I have my full cluster running on 0.8.1.1 and the logs data is just not getting cleaned up. And I see this message in kafka server.log ... [2014-07-10 20:49:43,786] WARN Property log.retention.minutes is not valid (kafka.utils.Verif

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

Thank you for pointing out to JIRA. On 7/9/14, 6:38 PM, "Jun Rao" wrote: >This is being worked on in >https://issues.apache.org/jira/browse/KAFKA-1325 > >Thanks, > >Jun > > >On Wed, Jul 9, 2014 at 11:42 AM, Virendra Pratap Singh < >vpsi...@yahoo-inc.com.invalid> wrote: > >> Well currently the lo

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

This is being worked on in https://issues.apache.org/jira/browse/KAFKA-1325 Thanks, Jun On Wed, Jul 9, 2014 at 11:42 AM, Virendra Pratap Singh < vpsi...@yahoo-inc.com.invalid> wrote: > Well currently the log rollup is controlled via > > log.roll.hours > Or > log.segment.bytes > > Given that we

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

Well currently the log rollup is controlled via log.roll.hours Or log.segment.bytes Given that we now have support to log retention in minutes, I guess it would be apt to have rollup also have capability to be available in minutes. Whom/where should I ask to have that coded in. One a similar not

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

Actually, Kafka only removes old segments. The last (active) segment is never removed. So, f you want to have a 10 min retention, you need to configure log rolling such that log segments are rolled at least every 10 mins. Thanks, Jun On Tue, Jul 8, 2014 at 10:04 PM, Virendra Pratap Singh < vpsi

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

That's correct. The server where in I was running 0.8.1.1 was not honoring this parameter, despite the fact it was set in it server.properties. Not sure if this fact would play any role, the server which was running 0.8.0 was the leader for all the topics and partition in my setup. And the second s

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

Yeah we probably should do that. I am not a committer though so someone else may help fix this issue? On Tue, Jul 8, 2014 at 2:10 PM, Jason Rosenberg wrote: > Ah, ok.it's just no longer documented as such in the config docs? > > > On Tue, Jul 8, 2014 at 4:46 PM, Guozhang Wang wrote: > > >

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

Ah, ok.it's just no longer documented as such in the config docs? On Tue, Jul 8, 2014 at 4:46 PM, Guozhang Wang wrote: > Jason, > > getLogRetentionTimeMillis() take either "log.retention.minutes" or > "log.retention.hours" and transform the value into milis. So you can > specify using eithe

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

Jason, getLogRetentionTimeMillis() take either "log.retention.minutes" or "log.retention.hours" and transform the value into milis. So you can specify using either granularity. Guozhang On Tue, Jul 8, 2014 at 1:11 PM, Jason Rosenberg wrote: > On a related note, in doing the upgrade from 0.8.0

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

On a related note, in doing the upgrade from 0.8.0, I noticed that the config property changed from 'log.retention.hours' to 'log.retention.minutes'. Would it have made more sense to deprecate rather than replace there? Also, I notice that internally, in the KafkaConfig class, it's represented as

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

Server properties should affect on only the local instance separately. Are you saying the property is not honored even on the 0.8.1 machines? Guozhang On Mon, Jul 7, 2014 at 3:55 PM, Virendra Pratap Singh < vpsi...@yahoo-inc.com.invalid> wrote: > By setting this property > log.retention.mins=10

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

By setting this property log.retention.mins=10 in the server.properties file, which is passed as argument when starting the broker. Virendra On 7/7/14, 3:31 PM, "Guozhang Wang" wrote: >How do you set the retention.minutes property? Is it through zk-based >topics tool? > >Guozhang > > >On Mon, J

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

How do you set the retention.minutes property? Is it through zk-based topics tool? Guozhang On Mon, Jul 7, 2014 at 3:07 PM, Virendra Pratap Singh < vpsi...@yahoo-inc.com.invalid> wrote: > I am running a mixed cluster as I mentioned earlier. 1 broker 0.8.0 and > the other 0.8.1.1. Should the ret

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

I am running a mixed cluster as I mentioned earlier. 1 broker 0.8.0 and the other 0.8.1.1. Should the retention of topics for partitions owned/replicated by the broker running 0.8.1.1 not enforce the server properties settings as defined for that server. So this brings an interesting question, in

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

log.retention.minutes is only available in 0.8.1.*. Thanks, Jun On Fri, Jun 27, 2014 at 2:13 PM, Virendra Pratap Singh < vpsi...@yahoo-inc.com.invalid> wrote: > Running a mixed 2 broker cluster. Mixed as in one of the broker1 is > running 0.8.0 and broker2 one 0.8.1.1 (from the apache release

Re: kafka 0.8.1.1 log.retention.minutes NOT being honored

The retention.minute property is only introduced in 0.8.1: https://issues.apache.org/jira/browse/KAFKA-918 if you are running 0.8.0 then it will not be recognized. Guozhang On Fri, Jun 27, 2014 at 2:13 PM, Virendra Pratap Singh < vpsi...@yahoo-inc.com.invalid> wrote: > Running a mixed 2 brok

kafka 0.8.1.1 log.retention.minutes NOT being honored

Running a mixed 2 broker cluster. Mixed as in one of the broker1 is running 0.8.0 and broker2 one 0.8.1.1 (from the apache release link. Directly using the tar ball, no local build used). I have set the log.retention.minutes=10. However the broker is not honoring the setting. I see its not cleanin