> thread?
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Tue, Feb 16, 2016 at 5:06 PM, Jun Rao <
> j...@confluent.io
> > >
> > > > > wrote:
> > > > > > > >
s that this new policy should work for
> CreateTime
> > > as
> > > > > > well.
> > > > > > > > If a
> > > > > > > > > > topic is configured with CreateTime, messages may not be
> > > added
> > > > in
> > > > > > > > s
ll messages in a log
> > > > > segment.
> > > > > > We
> > > > > > > > can
> > > > > > > > > delete a segment if its largest timestamp is less than
> > > > > > > > > log.retention.min.timestamp. This
; > 2. Right now, the user can specify "delete" as the retention
> > > policy
> > > > > > and a
> > > > > > > > log segment will be deleted either when the size of a
> partition
> > > > > > exceeds a
> > > &
> > > > > > threshold or the timestamp of a segment is older than a
> relative
> > > > period
> > > > > > of
> > > > > > > time (say 7 days) from now. What you are proposing is not a new
> > > > > retention
&
e
> > timestamp?
> > > If
> > > > > so,
> > > > > > could you update the wiki accordingly?
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Sat, Feb 13, 2016 at 3:23 PM, Bill Warshaw <
>
e:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > That is a good catch, thanks for pointing it out. If this KIP is
> > > > > accepted,
> > > > > > we'd need to document this and make the log cleaner not run
> > > > > timestamp
6 at 5:38 AM, Andrew Schofield <
> > > > > andrew_schofield_j...@outlook.com> wrote:
> > > > >
> > > > > > This KIP is related to KIP-32, but I strikes me that it only
> makes
> > > > sense
> > > > > > with one
t; > > with one of the two proposed message timestamp types. If I
> understand
> > > > > correctly, message timestamps are only certain to be monotonically
> > > > > increasing in the log if message.timestamp.type=LogAppendTime.
> > > > >
> &g
is and make the log cleaner not run
> > > > timestamp-based
> > > > > deletion unless message.timestamp.type=LogAppendTime.
> > > > >
> > > > > On Sat, Feb 13, 2016 at 5:38 AM, Andrew Schofield <
> > > > > andrew_schof
drew_schofield_j...@outlook.com> wrote:
> > > >
> > > > > This KIP is related to KIP-32, but I strikes me that it only makes
> > > sense
> > > > > with one of the two proposed message timestamp types. If I
> understand
> > > > >
easing in the log if message.timestamp.type=LogAppendTime.
> > > >
> > > >
> > > >
> > > > Does timestamp-based auto-expiration require use of
> > > > message.timestamp.type=LogAppendTime?
> > > >
> > > >
> > > >
> > > &g
the log if message.timestamp.type=LogAppendTime.
> > > >
> > > >
> > > >
> > > > Does timestamp-based auto-expiration require use of
> > > > message.timestamp.type=LogAppendTime?
> > > >
> > > >
> > > >
> > > >
> &g
sage.timestamp.type=LogAppendTime.
> > >
> > >
> > >
> > > Does timestamp-based auto-expiration require use of
> > > message.timestamp.type=LogAppendTime?
> > >
> > >
> > >
> > >
> > > I think this KIP is a go
le.
> >
> >
> >
> > Andrew Schofield
> >
> >
> >
> >
> > > Date: Fri, 12 Feb 2016 10:38:46 -0800
> > > Subject: Re: [DISCUSS] KIP-47 - Add timestamp-based log deletion policy
> > > From: n...@confluent.io
> > > To: dev@kafk
>
> > I think this KIP is a good idea, but I think it relies on strict ordering
> > of timestamps to be workable.
> >
> >
> >
> > Andrew Schofield
> >
> >
> >
> >
> > > Date: Fri, 12 Feb 2016 10:38:46 -0800
> > > Subject: Re: [DISCU
ld
>
>
>
>
> > Date: Fri, 12 Feb 2016 10:38:46 -0800
> > Subject: Re: [DISCUSS] KIP-47 - Add timestamp-based log deletion policy
> > From: n...@confluent.io
> > To: dev@kafka.apache.org
> >
> > Adding a timestamp based auto-expiration is useful and th
s on strict ordering
> > of timestamps to be workable.
> >
> >
> >
> > Andrew Schofield
> >
> >
> >
> >
> > > Date: Fri, 12 Feb 2016 10:38:46 -0800
> > > Subject: Re: [DISCUSS] KIP-47 - Add timestamp-based log deletion policy
>
o be workable.
>
>
>
> Andrew Schofield
>
>
>
>
> > Date: Fri, 12 Feb 2016 10:38:46 -0800
> > Subject: Re: [DISCUSS] KIP-47 - Add timestamp-based log deletion policy
> > From: n...@confluent.io
> > To: dev@kafka.apache.org
> >
> > Adding a t
-based auto-expiration require use of
message.timestamp.type=LogAppendTime?
I think this KIP is a good idea, but I think it relies on strict ordering of
timestamps to be workable.
Andrew Schofield
> Date: Fri, 12 Feb 2016 10:38:46 -0800
> Subject: Re: [DISCUSS] KIP-47 - Add timestamp
Adding a timestamp based auto-expiration is useful and this proposal makes
sense. Thx!
On Wed, Feb 10, 2016 at 3:35 PM, Jay Kreps wrote:
> I think this makes a lot of sense and won't be hard to implement and
> doesn't create too much in the way of new interfaces.
>
> -Jay
>
> On Tue, Feb 9, 2016
I think this makes a lot of sense and won't be hard to implement and
doesn't create too much in the way of new interfaces.
-Jay
On Tue, Feb 9, 2016 at 8:13 AM, Bill Warshaw wrote:
> Hello,
>
> I just submitted KIP-47 for adding a new log deletion policy based on a
> minimum timestamp of message
Hello,
I just submitted KIP-47 for adding a new log deletion policy based on a
minimum timestamp of messages to retain.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-47+-+Add+timestamp-based+log+deletion+policy
I'm open to any comments or suggestions.
Thanks,
Bill Warshaw
23 matches
Mail list logo
