Actually, Kafka only removes old segments. The last (active) segment is never removed. So, f you want to have a 10 min retention, you need to configure log rolling such that log segments are rolled at least every 10 mins.
Thanks, Jun On Tue, Jul 8, 2014 at 10:04 PM, Virendra Pratap Singh < vpsi...@yahoo-inc.com.invalid> wrote: > That's correct. The server where in I was running 0.8.1.1 was not honoring > this parameter, despite the fact it was set in it server.properties. > Not sure if this fact would play any role, the server which was running > 0.8.0 was the leader for all the topics and partition in my setup. And the > second server running 0.8.1.1 has all the replicas (follower). > > Virendra > > On 7/8/14, 12:54 PM, "Guozhang Wang" <wangg...@gmail.com> wrote: > > >Server properties should affect on only the local instance separately. Are > >you saying the property is not honored even on the 0.8.1 machines? > > > >Guozhang > > > >On Mon, Jul 7, 2014 at 3:55 PM, Virendra Pratap Singh < > >vpsi...@yahoo-inc.com.invalid> wrote: > > > >> By setting this property > >> log.retention.mins=10 > >> in the server.properties file, which is passed as argument when starting > >> the broker. > >> > >> Virendra > >> > >> On 7/7/14, 3:31 PM, "Guozhang Wang" <wangg...@gmail.com> wrote: > >> > >> >How do you set the retention.minutes property? Is it through zk-based > >> >topics tool? > >> > > >> >Guozhang > >> > > >> > > >> >On Mon, Jul 7, 2014 at 3:07 PM, Virendra Pratap Singh < > >> >vpsi...@yahoo-inc.com.invalid> wrote: > >> > > >> >> I am running a mixed cluster as I mentioned earlier. 1 broker 0.8.0 > >>and > >> >> the other 0.8.1.1. Should the retention of topics for partitions > >> >> owned/replicated by the broker running 0.8.1.1 not enforce the server > >> >> properties settings as defined for that server. > >> >> > >> >> So this brings an interesting question, in case of heterogeneous > >> >> environment (as is in my case, which system parameters will take > >> >> preference/precedence). > >> >> > >> >> Virendra > >> >> > >> >> On 6/30/14, 9:19 AM, "Guozhang Wang" <wangg...@gmail.com> wrote: > >> >> > >> >> >The retention.minute property is only introduced in 0.8.1: > >> >> > > >> >> >https://issues.apache.org/jira/browse/KAFKA-918 > >> >> > > >> >> >if you are running 0.8.0 then it will not be recognized. > >> >> > > >> >> >Guozhang > >> >> > > >> >> > > >> >> > > >> >> >On Fri, Jun 27, 2014 at 2:13 PM, Virendra Pratap Singh < > >> >> >vpsi...@yahoo-inc.com.invalid> wrote: > >> >> > > >> >> >> Running a mixed 2 broker cluster. Mixed as in one of the broker1 > >>is > >> >> >> running 0.8.0 and broker2 one 0.8.1.1 (from the apache release > >>link. > >> >> >> Directly using the tar ball, no local build used). > >> >> >> > >> >> >> I have set the log.retention.minutes=10. However the broker is not > >> >> >> honoring the setting. I see its not cleaning the log.dir at all. > >> >> >> > >> >> >> However when I set the log.retention.hours=1, then it starts > >>cleaning > >> >> >>the > >> >> >> log. > >> >> >> > >> >> >> When I have the log.retention.minutes set in the server.properties > >> >>then > >> >> >>I > >> >> >> see this logged in server.log: > >> >> >> > >> >> >> Š.. > >> >> >> [2014-06-27 19:21:06,633] WARN Property log.retention.minutes is > >>not > >> >> >>valid > >> >> >> (kafka.utils.VerifiableProperties) > >> >> >> [2014-06-27 19:21:06,633] WARN Property log.retention.minutes is > >>not > >> >> >>valid > >> >> >> (kafka.utils.VerifiableProperties) > >> >> >> ŠŠ > >> >> >> > >> >> >> > >> >> >> I have set these properties too: > >> >> >> > >> >> >> log.cleaner.enable=true > >> >> >> log.cleanup.policy=delete > >> >> >> > >> >> >> > >> >> >> But I see similar warning logged for these properties too. > >> >> >> > >> >> >> Regards, > >> >> >> Virendra > >> >> >> > >> >> >> > >> >> > > >> >> > > >> >> >-- > >> >> >-- Guozhang > >> >> > >> >> > >> > > >> > > >> >-- > >> >-- Guozhang > >> > >> > > > > > >-- > >-- Guozhang > >
