+1 for making consistent default log and offsets retention time. I like Stephane's suggestion too, log retention override should override offset retention too if not explicitly configured.
Please consider additionally: - introducing offsets.retention.hours config property - syncing log and offsets retention.check.interval.ms, if there's no real reason for the two to differ -- consider making retention check interval by default (if not explicitly configured) a fraction of retention time - name all "offsets" configs with "offsets" prefix (now it's a mix of singular/"offset" and plural/"offsets") On Fri, Aug 11, 2017 at 2:01 AM, Guozhang Wang <wangg...@gmail.com> wrote: > +1 from me > > On Wed, Aug 9, 2017 at 9:40 AM, Jason Gustafson <ja...@confluent.io> > wrote: > > > +1 on the bump to 7 days. Wanted to mention one minor point. The > > OffsetCommit RPC still provides the ability to set the retention time > from > > the client, but we do not use it in the consumer. Should we consider > adding > > a consumer config to set this? Given the problems people had with the old > > default, such a config would probably have gotten a fair bit of use. > Maybe > > it's less necessary with the new default, but there may be situations > where > > you don't want to keep the offsets for too long. For example, the console > > consumer commits offsets with a generated group id. We might want to set > a > > low retention time to keep it from filling the offset cache with garbage > > from such groups. > > > > I agree with Jason here, but maybe itself deserves a separate KIP > discussion. > > > > > > -Jason > > > > On Wed, Aug 9, 2017 at 5:24 AM, Sönke Liebau < > > soenke.lie...@opencore.com.invalid> wrote: > > > > > Just had this create issues at a customer as well, +1 > > > > > > On Wed, Aug 9, 2017 at 11:46 AM, Mickael Maison < > > mickael.mai...@gmail.com> > > > wrote: > > > > > > > Yes the current default is too short, +1 > > > > > > > > On Wed, Aug 9, 2017 at 8:56 AM, Ismael Juma <ism...@juma.me.uk> > wrote: > > > > > Thanks for the KIP, +1 from me. > > > > > > > > > > Ismael > > > > > > > > > > On Wed, Aug 9, 2017 at 1:24 AM, Ewen Cheslack-Postava < > > > e...@confluent.io > > > > > > > > > > wrote: > > > > > > > > > >> Hi all, > > > > >> > > > > >> I posted a simple new KIP for a problem we see with a lot of > users: > > > > >> KIP-186: Increase offsets retention default to 7 days > > > > >> > > > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > > > >> 186%3A+Increase+offsets+retention+default+to+7+days > > > > >> > > > > >> Note that in addition to the KIP text itself, the linked JIRA > > already > > > > >> existed and has a bunch of discussion on the subject. > > > > >> > > > > >> -Ewen > > > > >> > > > > > > > > > > > > > > > > -- > > > Sönke Liebau > > > Partner > > > Tel. +49 179 7940878 > > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany > > > > > > > > > -- > -- Guozhang >
