this kip fixes a "bug" (quirk?) that arises when people implement headers
"in V" (in the payload part of a message).
if you have proper headers you obviously dont need to to stick them in V
and so wont run into this, but its still a valid issue.
On Mon, Dec 19, 2016 at 3:06 PM, Jay Kreps <j...@confluent.io> wrote:
> Makes sense!
>
> -Jay
>
> On Mon, Dec 19, 2016 at 2:40 PM, Michael Pearce <michael.pea...@ig.com>
> wrote:
>
> > Wow just read that def over tired. Hopefully it makes sense. Or you get
> > the gist at least.
> >
> > ________________________________________
> > From: Michael Pearce <michael.pea...@ig.com>
> > Sent: Monday, December 19, 2016 9:19:02 PM
> > To: dev@kafka.apache.org
> > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
> >
> > Hi Jay,
> >
> > Agreed this stemmed as offshoot from KIP-82.
> >
> > Which our main driver for was to be able to have some headers for a null
> > value as such for our routing, audit, tracing and a few other bits which
> > currently we are forced to do with a message wrapper, if we all agreed on
> > KIP-82 that we need native headers and look to implement that the push
> for
> > this would dissipate.
> >
> > This KIP would allow for though one use case that comes to mind we could
> > see which is to have business data with a delete. Though as said this
> isn't
> > something we are pushing for think really we would have.
> >
> > As such in summary yes, if you want to fully support KIP-82 and we can
> get
> > that agreed in principle and a target release version, I think quite a
> few
> > guys at LinkedIn are quite pro it too ;) I'm happy to drop this one.
> >
> > Cheers
> > Mike
> >
> > Sent using OWA for iPhone
> > ________________________________________
> > From: Jay Kreps <j...@confluent.io>
> > Sent: Monday, December 19, 2016 8:51:23 PM
> > To: dev@kafka.apache.org
> > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
> >
> > Hey Michael,
> >
> > Here is the compatibility concern I have:
> >
> > 1. You have a consumer app that relies on value == null to indicate a
> > delete (current semantics).
> > 2. You upgrade Kafka and your clients.
> > 3. Some producer starts using the tombstone field in combination with
> > non-null.
> >
> > I share Ismael's dislike of setting tombstones on records with null
> values.
> > This makes sense as a transitional state, but as an end state its a bit
> > weird. You'd expect to be able to mix null values and tombstones, and
> have
> > the null values remain and the tombstones get compacted. However what
> will
> > happen is both will be compacted and upon debugging this you'll learn
> that
> > we sometimes use null in the value to indicate tombstone. Ismael's
> solution
> > is a bigger compatibility break, though, so not sure if that is better.
> >
> > My other question is the relationship to KIP-82. My read is that this KIP
> > solves some but not all of the problems KIP-82 is intended for. KIP-82,
> on
> > the other hand, seems to address most of the motivating uses for this
> KIP.
> > The exception is maybe item (5) on the list where you want to
> simultaneous
> > delete and convey some information to subscribers, but I couldn't
> construct
> > a concrete examples for that one. Do we need to rationalize these two
> KIPs?
> > That is, do you still advocate this proposal if we do KIP-82 and vice
> > versa? As you may have noticed I'm somewhat emotionally invested in the
> > simplicity of the core data model, so my default position is let's try to
> > avoid stuffing more stuff in, but if we have to add stuff I like each of
> > these individually more than doing both. :-)
> >
> > -Jay
> >
> >
> >
> >
> > On Fri, Dec 16, 2016 at 12:16 PM, Michael Pearce <michael.pea...@ig.com>
> > wrote:
> >
> > > Hi Jay
> > >
> > > I disagree here that we are breaking any compatibility, we went through
> > > this on the discussion thread in fact with the help of that thread is
> how
> > > the kip came to the solution.
> > >
> > > Also on the supported combinations front you mention, we are not taking
> > > anything away afaik.
> > >
> > > Currently supported are only are:
> > > Null value = delete
> > > Non-null value = non delete
> > >
> > > With this kip we would support
> > > Null value + tombstone = delete
> > > Non null value + tombstone = delete
> > > Non null value + no tombstone = non delete
> > >
> > > As for the alternative idea, this is simply a new policy, how can there
> > be
> > > confusion here? For this policy it would be explicit that tombstone
> > marker
> > > would need to be set for a delete.
> > >
> > > I'm going to vent a little now as starting to get quite frustrated.
> > >
> > > We are going round in circles on kip-82 as per use cases there is now
> > many
> > > use cases, how many more are needed? just because confluent don't see
> > these
> > > doesn't mean they aren't real use cases other have, this is the point
> of
> > > the Apache foundation, it shouldn't be the view of just one
> organisation.
> > > It really is getting a feeling of the NIH syndrome. Rather than it
> being
> > > constructive on discussion of the implementation detail.
> > >
> > > kip-87 spawned from as on the kip call we all agreed this was needed.
> And
> > > would at least allow a custom wrapper be supported in a compacted
> topic,
> > > allowing meta data. Which again now I feel we are spinning wheels, and
> > > simply finding reasons not support it.
> > >
> > > Cheers
> > > Mike
> > >
> > >
> > >
> > > Sent using OWA for iPad
> > > ________________________________________
> > > From: Jay Kreps <j...@confluent.io>
> > > Sent: Friday, December 16, 2016 7:09:23 PM
> > > To: dev@kafka.apache.org
> > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
> > >
> > > Hey Michael,
> > >
> > > I do think it might have been better had we started with a separate
> > concept
> > > of null vs delete. Given that we are where we are, I'm not sure that
> the
> > > possible cures we explored so far are better than the disease.
> > >
> > > I apologize for coming to this late, but I didn't really understand the
> > > implications of the KIP and that we'd be breaking compatibility with
> > > existing apps until the vote had begun, and in my defense I'm not sure
> > the
> > > other folks voting did either.
> > >
> > > I think we all agree there are many existing apps that are built with
> the
> > > assumption of "null value non-tombstone" and it isn't possible to
> > > disambiguate these from tombstones on the producer. It isn't that
> anyone
> > is
> > > saying we have to support all four possibilities at once, it's that we
> > > simply can't orphan one of the existing combinations or our users will
> > eat
> > > us!
> > >
> > > If I've understood your alternate solution of adding another setting
> for
> > > compaction, I think this does fix the compatibility problem, but it
> adds
> > an
> > > odd mode the user has to add on all their topics. While the current
> state
> > > is easily explainable, the resulting state where the meaning of
> tombstone
> > > and null are overlapping and ambiguous and dependent on a compaction
> > > setting that could change out of band or not be in sync with your code
> in
> > > some environment seems worse to me then where we currently are. I think
> > the
> > > question is how would this combination be explained to users and does
> it
> > > make sense?
> > >
> > > -Jay
> > >
> > >
> > >
> > > On Fri, Dec 16, 2016 at 9:25 AM, Michael Pearce <michael.pea...@ig.com
> >
> > > wrote:
> > >
> > > > Hi Chaps,
> > > >
> > > > Can we either get one more +1 binding (we have 2 already) on the
> > > existing?
> > > >
> > > > Or have some response on the below possible alternative. We are keen
> to
> > > > get working on this, so we make next feature release.
> > > >
> > > > Cheers
> > > > Mike
> > > >
> > > >
> > > > On 13/12/2016, 16:32, "Michael Pearce" <michael.pea...@ig.com>
> wrote:
> > > >
> > > > Hi Ismael
> > > >
> > > > Did you see our email this morning, what's your thoughts on this
> > > > approach to instead we simply have a brand new policy?
> > > >
> > > > Cheers
> > > > Mike
> > > >
> > > >
> > > > Sent using OWA for iPhone
> > > > ________________________________________
> > > > From: isma...@gmail.com <isma...@gmail.com> on behalf of Ismael
> > > Juma <
> > > > ism...@juma.me.uk>
> > > > Sent: Tuesday, December 13, 2016 11:30:05 AM
> > > > To: dev@kafka.apache.org
> > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
> > > >
> > > > Yes, this is actually tricky to do in a way where we both have
> the
> > > > desired
> > > > semantics and maintain compatibility. When someone creates a
> > > > `ProducerRecord` with a `null` value today, the producer doesn't
> > know
> > > > if
> > > > it's meant to be a tombstone or not. For V3 messages, it's easy
> > when
> > > > the
> > > > constructor that takes a tombstone is used. However, if any other
> > > > constructor is used, it's not clear. A couple of options I can
> > think
> > > > of,
> > > > none of them particularly nice:
> > > >
> > > > 1. Have a third state where tombstone = unknown and the broker
> > would
> > > > set
> > > > the tombstone bit if the value was null and the topic was
> > compacted.
> > > > People
> > > > that wanted to pass a non-null value for the tombstone would have
> > to
> > > > use
> > > > the constructor that takes a tombstone. The drawbacks: third
> state
> > > for
> > > > tombstone in message format, message conversion at the broker
> for a
> > > > common
> > > > case.
> > > >
> > > > 2. Extend MetadataResponse to optionally include topic configs,
> > which
> > > > would
> > > > make it possible for the producer to be smarter about setting the
> > > > tombstone. It would only do it if a tombstone was not passed
> > > > explicitly,
> > > > the value was null and the topic was compacted. The main drawback
> > is
> > > > that
> > > > the producer would be getting a bit more data for each topic even
> > > > though it
> > > > probably won't use it most of the time. Extending
> MetadataResponse
> > to
> > > > return topic configs would be useful for other reasons as well,
> so
> > > that
> > > > part seems OK.
> > > >
> > > > In addition, for both proposals, we could consider adding
> warnings
> > to
> > > > the
> > > > documentation that the behaviour of the constructors that don't
> > take
> > > a
> > > > tombstone would change in the next major release so that
> tombstone
> > =
> > > > false.
> > > > Not sure if this would be worth it though.
> > > >
> > > > Ismael
> > > >
> > > > On Sun, Dec 11, 2016 at 11:15 PM, Ewen Cheslack-Postava <
> > > > e...@confluent.io>
> > > > wrote:
> > > >
> > > > > Michael,
> > > > >
> > > > > It kind of depends on how you want to interpret the tombstone
> > flag.
> > > > If it's
> > > > > purely a producer-facing Kafka-level thing that we treat as
> > > internal
> > > > to the
> > > > > broker and log cleaner once the record is sent, then your
> > approach
> > > > makes
> > > > > sense. You're just moving copying the null-indicates-delete
> > > behavior
> > > > of the
> > > > > old constructor into the tombstone flag.
> > > > >
> > > > > However, if you want this change to more generally decouple the
> > > idea
> > > > of
> > > > > deletion and null values, then you are sometimes converting
> what
> > > > might be a
> > > > > completely valid null value that doesn't indicate deletion
> into a
> > > > > tombstone. Downstream applications could potentially handle
> these
> > > > cases
> > > > > differently given the separation of deletion from value.
> > > > >
> > > > > I guess the question is if we want to try to support the latter
> > > even
> > > > for
> > > > > topics where we have older produce requests. An example where
> > this
> > > > could
> > > > > come up is in something like a CDC Connector. If we try to
> > support
> > > > the
> > > > > semantic difference, a connector might write changes to Kafka
> > using
> > > > the
> > > > > tombstone flag to indicate when a row was truly deleted (vs an
> > > > update that
> > > > > sets it to null but still present; this probably makes more
> sense
> > > > for CDC
> > > > > from document stores or extracting single columns). There are
> > > various
> > > > > reasons we might want to maintain the full log and not turn
> > > > compaction on
> > > > > (or just use a time-based retention policy), but downstream
> > > > applications
> > > > > might care to know the difference between a delete and a null
> > > value.
> > > > In
> > > > > fact both versions of the same log (compacted and
> time-retention)
> > > > could be
> > > > > useful and I don't think it'll be uncommon to maintain both or
> > use
> > > > KIP-71
> > > > > to maintain a hybrid compacted/retention topic.
> > > > >
> > > > > -Ewen
> > > > >
> > > > > On Sun, Dec 11, 2016 at 1:18 PM, Michael Pearce <
> > > > michael.pea...@ig.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Jay,
> > > > > >
> > > > > > Why wouldn't that work, the tombstone value is only looked at
> > by
> > > > the
> > > > > > broker, on a topic configured for compaction as such is
> benign
> > on
> > > > non
> > > > > > compacted topics. This is as much as sending a null value
> > > currently
> > > > > >
> > > > > >
> > > > > > Regards
> > > > > > Mike
> > > > > >
> > > > > >
> > > > > >
> > > > > > Sent using OWA for iPhone
> > > > > > ________________________________________
> > > > > > From: Jay Kreps <j...@confluent.io>
> > > > > > Sent: Sunday, December 11, 2016 8:58:53 PM
> > > > > > To: dev@kafka.apache.org
> > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
> > > > > >
> > > > > > Hey Michael,
> > > > > >
> > > > > > I'm not quite sure that works as that would translate ALL
> null
> > > > values to
> > > > > > tombstones, even for non-compacted topics that use null as an
> > > > acceptable
> > > > > > value sent by the producer and expected by the consumer.
> > > > > >
> > > > > > -Jay
> > > > > >
> > > > > > On Sun, Dec 11, 2016 at 3:26 AM, Michael Pearce <
> > > > michael.pea...@ig.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Ewen,
> > > > > > >
> > > > > > > I think the easiest way to show this is with code.
> > > > > > >
> > > > > > > As you can see we keep the existing behaviour for
> > code/binaries
> > > > calling
> > > > > > > the pre-existing constructors, whereby if the value is null
> > the
> > > > > tombstone
> > > > > > > is set to true.
> > > > > > >
> > > > > > > Regards
> > > > > > > Mike
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > /**
> > > > > > > * Creates a record with a specified timestamp to be
> sent
> > > to
> > > > a
> > > > > > > specified topic and partition
> > > > > > > *
> > > > > > > * @param topic The topic the record will be appended
> to
> > > > > > > * @param partition The partition to which the record
> > > should
> > > > be
> > > > > sent
> > > > > > > * @param timestamp The timestamp of the record
> > > > > > > * @param tombstone if the record should be treated as
> a
> > > > tombstone
> > > > > if
> > > > > > > the topic is compacted
> > > > > > > * @param key The key that will be included in the
> record
> > > > > > > * @param value The record contents
> > > > > > > */
> > > > > > > public ProducerRecord(String topic, Integer partition,
> > > > Boolean
> > > > > > > tombstone, Long timestamp, K key, V value) {
> > > > > > > if (topic == null)
> > > > > > > throw new IllegalArgumentException("Topic
> cannot
> > > be
> > > > > null.");
> > > > > > > if (timestamp != null && timestamp < 0)
> > > > > > > throw new IllegalArgumentException(
> > > > > > > String.format("Invalid timestamp: %d.
> > > > Timestamp
> > > > > > should
> > > > > > > always be non-negative or null.", timestamp));
> > > > > > > if (partition != null && partition < 0)
> > > > > > > throw new IllegalArgumentException(
> > > > > > > String.format("Invalid partition: %d.
> > > > Partition
> > > > > > number
> > > > > > > should always be non-negative or null.", partition));
> > > > > > > if (!tombstone && value == null){
> > > > > > > throw new IllegalArgumentException(
> > > > > > > String.format("Tombstone must be true
> if
> > > null
> > > > > > value"));
> > > > > > > }
> > > > > > > this.topic = topic;
> > > > > > > this.partition = partition;
> > > > > > > this.tombstone = tombstone;
> > > > > > > this.key = key;
> > > > > > > this.value = value;
> > > > > > > this.timestamp = timestamp;
> > > > > > > }
> > > > > > >
> > > > > > > public ProducerRecord(String topic, Integer partition,
> > Long
> > > > > > timestamp,
> > > > > > > K key, V value) {
> > > > > > > this(topic, partition, value == null, timestamp,
> key,
> > > > value);
> > > > > > > }
> > > > > > > ________________________________________
> > > > > > > From: Ewen Cheslack-Postava <e...@confluent.io>
> > > > > > > Sent: Sunday, December 11, 2016 5:45 AM
> > > > > > > To: dev@kafka.apache.org
> > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone Flag
> > > > > > >
> > > > > > > It seemed like this was addressed in the migration section,
> > > > wasn't it?
> > > > > > The
> > > > > > > V2 vs V3 requests and the fact that the broker will
> downgrade
> > > the
> > > > > message
> > > > > > > format (losing zero copy) if you issues a V2 request to a
> > > broker
> > > > using
> > > > > V3
> > > > > > > format handles compatibility, does it not? The existing
> > > > consumers won't
> > > > > > see
> > > > > > > the extra metadata in the value, but they will get a null
> > > > instead and
> > > > > > treat
> > > > > > > it as a tombstone. Certainly there is a performance impact,
> > but
> > > > it
> > > > > seemed
> > > > > > > compatible.
> > > > > > >
> > > > > > > I'm worried about this though:
> > > > > > >
> > > > > > >
> > > > > > > - *NOTE *: With the new version of producer client using
> > > > > > ProduceRequest
> > > > > > > V3 (magic byte = 2), a non tombstone (tombstone bit not
> > set)
> > > > and
> > > > > null
> > > > > > > value
> > > > > > > should not be allowed as the older version of consumer
> > using
> > > > > > > FetchRequest
> > > > > > > V2 will think of this as a tombstone when its actually
> > not.
> > > > > > >
> > > > > > >
> > > > > > > Unless I'm misunderstanding, this ends up breaking binary
> > > > compatibility
> > > > > > for
> > > > > > > the Java API. It sounds like this suggests that passing a
> > null
> > > > value to
> > > > > > the
> > > > > > > existing ProducerRecord constructors would generate an
> > > exception
> > > > since
> > > > > > you
> > > > > > > didn't explicitly enable the tombstone (via whatever new
> > > > constructor is
> > > > > > > provided). But that means you can't swap in a newer client
> > jar
> > > > without
> > > > > > > recompiling and get the same behavior if your app deletes
> > keys
> > > > using
> > > > > the
> > > > > > > current approach because your app will start throwing
> > > > exceptions. Maybe
> > > > > > > this is a tradeoff we're ok with, but we've tried pretty
> hard
> > > to
> > > > avoid
> > > > > > > breaking compatibility like this so far -- it makes picking
> > up
> > > > bug
> > > > > fixes
> > > > > > in
> > > > > > > the clients harder for users of frameworks that have to pin
> > to
> > > > earlier
> > > > > > > default versions for compatibility.
> > > > > > >
> > > > > > > But then later the KIP says:
> > > > > > >
> > > > > > >
> > > > > > > - The old constructors for ProducerRecord without this
> > > > parameter
> > > > > will
> > > > > > be
> > > > > > > kept but updated so that their default behaviour if
> > setting
> > > > null
> > > > > value
> > > > > > > will
> > > > > > > be the tombstone will be set to true to keep existing
> > > > behaviour.
> > > > > > >
> > > > > > >
> > > > > > > So maybe I am misinterpreting something.
> > > > > > >
> > > > > > > And just a nit re: motivation section, item 6 would be
> > clearer
> > > > for a
> > > > > > union
> > > > > > > schema with null (or optional schemas in other formats),
> e.g.
> > > > [null,
> > > > > > > string], in which case losing the schema truly is losing
> > > > information
> > > > > > > (whereas null is already the only valid value for a pure
> null
> > > > schema).
> > > > > > >
> > > > > > > -Ewen
> > > > > > >
> > > > > > >
> > > > > > > On Sat, Dec 10, 2016 at 9:24 PM, Michael Pearce <
> > > > michael.pea...@ig.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Jay,
> > > > > > > >
> > > > > > > > Good point this detail is missing in the KIP write up.
> Ive
> > > > added this
> > > > > > > now.
> > > > > > > >
> > > > > > > > Essentially simply just upgrading the clients we do not
> > > expect
> > > > any
> > > > > > client
> > > > > > > > app code change needed.
> > > > > > > >
> > > > > > > > Cheers
> > > > > > > > Mike
> > > > > > > > ________________________________________
> > > > > > > > From: Jay Kreps <j...@confluent.io>
> > > > > > > > Sent: Saturday, December 10, 2016 10:51 PM
> > > > > > > > To: dev@kafka.apache.org
> > > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction Tombstone
> Flag
> > > > > > > >
> > > > > > > > Michael,
> > > > > > > >
> > > > > > > > The compatibility section goes through the migration
> path,
> > > but
> > > > isn't
> > > > > > the
> > > > > > > > bigger compatibility issue with existing apps? There are
> > many
> > > > > (probably
> > > > > > > > thousands) of apps in production that use this feature
> and
> > > > send null
> > > > > to
> > > > > > > > mean delete. It seems like this would break compatibility
> > > with
> > > > them,
> > > > > > and
> > > > > > > > they would have to be rewritten to right?
> > > > > > > >
> > > > > > > > -Jay
> > > > > > > >
> > > > > > > > On Thu, Dec 8, 2016 at 12:12 AM, Michael Pearce <
> > > > > michael.pea...@ig.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Jun,
> > > > > > > > >
> > > > > > > > > 4) On v3 we honour the tombstone. As such we expect it
> to
> > > be
> > > > set
> > > > > > > > correctly
> > > > > > > > > as per the KIP.
> > > > > > > > >
> > > > > > > > > 4.1) why would we want to produce an error when its v3?
> > > This
> > > > is the
> > > > > > > exact
> > > > > > > > > purpose to support non-null tombstone's
> > > > > > > > > 4.2) again here im unclear on the question on the v3,
> > > produce
> > > > > request
> > > > > > > we
> > > > > > > > > expect the tombstone flag to be set correctly.
> > > > > > > > >
> > > > > > > > > 4.4) compaction only occurs on compacted topics, the
> bit
> > > > makes no
> > > > > > > > > difference and not looked at on un-compacted (time/size
> > > based
> > > > > > > eviction).
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On 06/12/2016, 20:08, "Jun Rao" <j...@confluent.io>
> > wrote:
> > > > > > > > >
> > > > > > > > > Hi, Michael,
> > > > > > > > >
> > > > > > > > > 4. Then, I think I misunderstood this point. Could
> > you
> > > > document
> > > > > > the
> > > > > > > > > following points in the wiki?
> > > > > > > > > 4.1 If producer V3 sets tombstone, but provides a
> > > > non-null
> > > > > value,
> > > > > > > > does
> > > > > > > > > the
> > > > > > > > > send() get an error or does the producer
> > automatically
> > > > set the
> > > > > > > value
> > > > > > > > to
> > > > > > > > > null?
> > > > > > > > > 4.2 If producer V3 doesn't set tombstone, but
> > provides
> > > a
> > > > null
> > > > > > > value,
> > > > > > > > > does
> > > > > > > > > the send() get an error or does the producer
> > > > automatically sets
> > > > > > the
> > > > > > > > > tombstone?
> > > > > > > > > 4.3 Does the broker only expect messages that (a)
> > have
> > > no
> > > > > > tombstone
> > > > > > > > and
> > > > > > > > > non-null value; (b) have tombstone and null value
> and
> > > > reject
> > > > > the
> > > > > > > > > messages
> > > > > > > > > with an error code otherwise?
> > > > > > > > > 4.4 Do 4.1, 4.2, 4.3 depend on whether the topic
> is
> > > > compacted
> > > > > on
> > > > > > > > not?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Jun
> > > > > > > > >
> > > > > > > > > On Tue, Dec 6, 2016 at 10:35 AM, Michael Pearce <
> > > > > > > > michael.pea...@ig.com
> > > > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Not at all. This only acts on compacted topics
> > just
> > > > as what
> > > > > > > occurs
> > > > > > > > > today
> > > > > > > > > >
> > > > > > > > > > Sent using OWA for iPhone
> > > > > > > > > > ________________________________________
> > > > > > > > > > From: Jun Rao <j...@confluent.io>
> > > > > > > > > > Sent: Tuesday, December 6, 2016 6:25:28 PM
> > > > > > > > > > To: dev@kafka.apache.org
> > > > > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction
> > Tombstone
> > > > Flag
> > > > > > > > > >
> > > > > > > > > > Hi, Michael,
> > > > > > > > > >
> > > > > > > > > > 4. Hmm, does that mean the new client library can
> > > > never send
> > > > > a
> > > > > > > null
> > > > > > > > > message
> > > > > > > > > > even to a regular topic? This seems like a change
> > of
> > > > the
> > > > > > existing
> > > > > > > > > behavior.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > Jun
> > > > > > > > > >
> > > > > > > > > > On Tue, Dec 6, 2016 at 9:51 AM, Michael Pearce <
> > > > > > > > > michael.pea...@ig.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Jun,
> > > > > > > > > > >
> > > > > > > > > > > Re 4) That's because we expect the tombstone
> > value
> > > > to be
> > > > > set
> > > > > > > > > correctly if
> > > > > > > > > > > message bit is 2, as such if an older client
> > sends
> > > > in on
> > > > > old
> > > > > > > > > message the
> > > > > > > > > > > message is upcast and the bit is set correctly.
> > And
> > > > such no
> > > > > > > > longer
> > > > > > > > > need
> > > > > > > > > > to
> > > > > > > > > > > check the value. Mayuresh can you confirm my
> > > > thinking and
> > > > > > > > > understanding
> > > > > > > > > > of
> > > > > > > > > > > what we've discussed?
> > > > > > > > > > >
> > > > > > > > > > > The second point I understand what you're
> getting
> > > at
> > > > now my
> > > > > > > > > apologies.
> > > > > > > > > > Yes
> > > > > > > > > > > this makes sense to save on touching the
> message,
> > > if
> > > > we're
> > > > > > the
> > > > > > > > > only kip
> > > > > > > > > > > going in, in this release.
> > > > > > > > > > >
> > > > > > > > > > > Cheers
> > > > > > > > > > > Mike
> > > > > > > > > > >
> > > > > > > > > > > Sent using OWA for iPhone
> > > > > > > > > > > ________________________________________
> > > > > > > > > > > From: Jun Rao <j...@confluent.io>
> > > > > > > > > > > Sent: Tuesday, December 6, 2016 5:22:13 PM
> > > > > > > > > > > To: dev@kafka.apache.org
> > > > > > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction
> > > > Tombstone Flag
> > > > > > > > > > >
> > > > > > > > > > > Hi, Michael,
> > > > > > > > > > >
> > > > > > > > > > > 4. Is this updated in the wiki? The text "If
> the
> > > > magic byte
> > > > > > on
> > > > > > > > > message is
> > > > > > > > > > > 2, the broker should use the tombstone bit for
> > log
> > > > > > compaction."
> > > > > > > > > doesn't
> > > > > > > > > > > seem to have changed.
> > > > > > > > > > >
> > > > > > > > > > > 2. My point is that if we change the message
> > format
> > > > just
> > > > > for
> > > > > > > this
> > > > > > > > > KIP, we
> > > > > > > > > > > should consider whether it's worth optimizing
> the
> > > > down
> > > > > > > conversion
> > > > > > > > > path
> > > > > > > > > > > (i.e., decide whether a conversion is needed by
> > > just
> > > > > looking
> > > > > > at
> > > > > > > > the
> > > > > > > > > > > tombstone bit in the wrapper message) since
> > > > tombstone will
> > > > > be
> > > > > > > > used
> > > > > > > > > > rarely.
> > > > > > > > > > > However, if the message format change here is
> > > > combined with
> > > > > > > other
> > > > > > > > > KIPs,
> > > > > > > > > > > then this optimization likely won't be needed.
> > The
> > > > latter
> > > > > > > > probably
> > > > > > > > > makes
> > > > > > > > > > > the code simpler. Jiangjie, Mayuresh, what do
> you
> > > > think?
> > > > > > > > > > >
> > > > > > > > > > > Other than those, +1 from me,
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > >
> > > > > > > > > > > Jun
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Dec 6, 2016 at 8:54 AM, Michael Pearce
> <
> > > > > > > > > michael.pea...@ig.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Jun
> > > > > > > > > > > >
> > > > > > > > > > > > do we have your vote on this now?
> > > > > > > > > > > >
> > > > > > > > > > > > Any other concerns?
> > > > > > > > > > > >
> > > > > > > > > > > > Cheers
> > > > > > > > > > > > Mike
> > > > > > > > > > > >
> > > > > > > > > > > > Sent using OWA for iPhone
> > > > > > > > > > > > ________________________________________
> > > > > > > > > > > > From: Michael Pearce <michael.pea...@ig.com>
> > > > > > > > > > > > Sent: Saturday, December 3, 2016 1:37:45 AM
> > > > > > > > > > > > To: dev@kafka.apache.org
> > > > > > > > > > > > Subject: Re: [VOTE] KIP-87 - Add Compaction
> > > > Tombstone
> > > > > Flag
> > > > > > > > > > > >
> > > > > > > > > > > > Hi Jun,
> > > > > > > > > > > >
> > > > > > > > > > > > Have updated. Thanks again for the feedback.
> > > > > > > > > > > >
> > > > > > > > > > > > Agree yes we should align up when it gets to
> > > that,
> > > > I
> > > > > assume
> > > > > > > > > you've
> > > > > > > > > > > flagged
> > > > > > > > > > > > the same in the other KIP?
> > > > > > > > > > > >
> > > > > > > > > > > > I think for now let's get this KIP approved,
> > then
> > > > we can
> > > > > > > start
> > > > > > > > > the work
> > > > > > > > > > > to
> > > > > > > > > > > > get an initial PR, then we can discuss how to
> > > > align the
> > > > > two
> > > > > > > if
> > > > > > > > > needed.
> > > > > > > > > > > >
> > > > > > > > > > > > Cheers,
> > > > > > > > > > > > Mike
> > > > > > > > > > > >
> > > > > > > > > > > > On 02/12/2016, 18:26, "Jun Rao" <
> > > j...@confluent.io>
> > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi, Michael,
> > > > > > > > > > > >
> > > > > > > > > > > > For 2), this is fine. Could you update
> the
> > > KIP
> > > > wiki
> > > > > to
> > > > > > > make
> > > > > > > > > this
> > > > > > > > > > and
> > > > > > > > > > > > other
> > > > > > > > > > > > points clearer? Other than that, the KIP
> > > looks
> > > > good
> > > > > to
> > > > > > > me.
> > > > > > > > > > > >
> > > > > > > > > > > > An orthogonal thing is that there are
> other
> > > > KIPs such
> > > > > > as
> > > > > > > > > KIP-98
> > > > > > > > > > that
> > > > > > > > > > > > also
> > > > > > > > > > > > intend to change the message format. If
> > they
> > > > all get
> > > > > > > > > approved, we
> > > > > > > > > > > > should
> > > > > > > > > > > > think about whether it's better to just
> > bump
> > > > up the
> > > > > > magic
> > > > > > > > > byte once
> > > > > > > > > > > to
> > > > > > > > > > > > incorporate multiple format changes like
> we
> > > > did in
> > > > > > > > > KIP-31/KIP-32.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > >
> > > > > > > > > > > > Jun
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Dec 2, 2016 at 3:19 AM, Michael
> > > Pearce
> > > > <
> > > > > > > > > > > michael.pea...@ig.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi Jao
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks for the response. Sorry for slow
> > > > reply, both
> > > > > > > with
> > > > > > > > > personal
> > > > > > > > > > > > sickness
> > > > > > > > > > > > > and also battling some critical issues
> > > > encountered
> > > > > > > since
> > > > > > > > > > upgrading
> > > > > > > > > > > to
> > > > > > > > > > > > > 0.10.1.0
> > > > > > > > > > > > >
> > > > > > > > > > > > > 1) Thans for spotting, Document error
> > where
> > > > we
> > > > > > branched
> > > > > > > > > this KIP
> > > > > > > > > > > from
> > > > > > > > > > > > > KIP-82, will get that removed.
> > > > > > > > > > > > > 2) Intent is to do this just at the
> > record
> > > > message
> > > > > > > level.
> > > > > > > > > > > > > 3) Thanks for spotting, Will ensure
> this
> > is
> > > > > > corrected.
> > > > > > > > > > > > > 4) As per discussion thread we will
> > support
> > > > > > tombstone +
> > > > > > > > > null
> > > > > > > > > > value,
> > > > > > > > > > > > > tombstone + non null value, no
> tombstone
> > +
> > > > null
> > > > > > value.
> > > > > > > > > > > > > 5) I believe this was in the discussion
> > > > thread,
> > > > > > > @Mayuresh
> > > > > > > > > is this
> > > > > > > > > > > > > something we've overlooked? I thought
> we
> > > > would down
> > > > > > > > > convert and
> > > > > > > > > > > > remove the
> > > > > > > > > > > > > value so the old consumer had existing
> > > > behavior, or
> > > > > > is
> > > > > > > > > there
> > > > > > > > > > > > something we
> > > > > > > > > > > > > haven't thought about?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Cheers
> > > > > > > > > > > > > Mike
> > > > > > > > > > > > >
> > > > > > > > > > > > > On 30/11/2016, 18:12, "Jun Rao" <
> > > > j...@confluent.io>
> > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hi, Michael,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks for the KIP. A few comments
> > > below.
> > > > > > > > > > > > >
> > > > > > > > > > > > > 1. The message format change
> contains
> > > > > > > "HeadersLength
> > > > > > > > > > Headers".
> > > > > > > > > > > > Is that
> > > > > > > > > > > > > intended?
> > > > > > > > > > > > >
> > > > > > > > > > > > > 2. For compressed messageset, is
> the
> > > > tombstone
> > > > > > bit
> > > > > > > > > only set
> > > > > > > > > > at
> > > > > > > > > > > > the
> > > > > > > > > > > > > shallow
> > > > > > > > > > > > > level? Do we always leave that bit
> in
> > > the
> > > > > wrapper
> > > > > > > > > message
> > > > > > > > > > > unset?
> > > > > > > > > > > > An
> > > > > > > > > > > > > alternative is to set the tombstone
> > bit
> > > > in the
> > > > > > > > wrapper
> > > > > > > > > if at
> > > > > > > > > > > > least one
> > > > > > > > > > > > > inner message has the tombstone bit
> > > set.
> > > > This
> > > > > > makes
> > > > > > > > > things a
> > > > > > > > > > > bit
> > > > > > > > > > > > more
> > > > > > > > > > > > > complicated, but we could
> potentially
> > > > exploit
> > > > > > that
> > > > > > > > for
> > > > > > > > > > > > optimizing down
> > > > > > > > > > > > > conversion. For example, we only
> need
> > > to
> > > > > convert
> > > > > > > > > messages
> > > > > > > > > > with
> > > > > > > > > > > > magic 2
> > > > > > > > > > > > > to
> > > > > > > > > > > > > magic 1 if the wrapper's tombstone
> > bit
> > > > is set
> > > > > > > > > (conversion is
> > > > > > > > > > > > always
> > > > > > > > > > > > > needed
> > > > > > > > > > > > > from magic 2 to magic 0). Not sure
> if
> > > the
> > > > > > > > optimization
> > > > > > > > > is
> > > > > > > > > > worth
> > > > > > > > > > > > the
> > > > > > > > > > > > > complexity though.
> > > > > > > > > > > > >
> > > > > > > > > > > > > 3. The referencing of the new
> version
> > > of
> > > > > > > > > > > > ProducerRequest/FetchRequest
> > > > > > > > > > > > > is
> > > > > > > > > > > > > inconsistent (v4 vs v3). Since our
> > > > convention
> > > > > > > starts
> > > > > > > > at
> > > > > > > > > > version
> > > > > > > > > > > > at 0, I
> > > > > > > > > > > > > think the new version would be 3.
> > > > > > > > > > > > >
> > > > > > > > > > > > > 4. "If the magic byte on message is
> > 2,
> > > > the
> > > > > broker
> > > > > > > > > should use
> > > > > > > > > > > the
> > > > > > > > > > > > > tombstone
> > > > > > > > > > > > > bit for log compaction." What about
> > > null
> > > > value?
> > > > > > My
> > > > > > > > > > > understanding
> > > > > > > > > > > > is
> > > > > > > > > > > > > that
> > > > > > > > > > > > > null value will be treated the same
> > as
> > > > setting
> > > > > > the
> > > > > > > > > tombstone
> > > > > > > > > > > bit.
> > > > > > > > > > > > >
> > > > > > > > > > > > > 5. For the migration path, it would
> > be
> > > > useful
> > > > > to
> > > > > > > > > describe the
> > > > > > > > > > > > down
> > > > > > > > > > > > > conversion path to consumers (i.e.,
> > > > brokers on
> > > > > > > > message
> > > > > > > > > format
> > > > > > > > > > > > 0.10.2
> > > > > > > > > > > > > and
> > > > > > > > > > > > > consumers on older version).
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Jun
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Tue, Nov 29, 2016 at 3:18 AM,
> > > Michael
> > > > > Pearce <
> > > > > > > > > > > > michael.pea...@ig.com
> > > > > > > > > > > > > >
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi All,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > We have been discussing in the
> > below
> > > > thread
> > > > > and
> > > > > > > > final
> > > > > > > > > > changes
> > > > > > > > > > > > have
> > > > > > > > > > > > > been
> > > > > > > > > > > > > > made to the KIP wiki based on
> these
> > > > > > discussions.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > We would now like to put to the
> > vote
> > > > the
> > > > > > > following
> > > > > > > > > KIP:
> > > > > > > > > > > > > > https://cwiki.apache.org/
> > > > > > > > > confluence/display/KAFKA/KIP-
> > > > > > > > > > 87+-+
> > > > > > > > > > > > > > Add+Compaction+Tombstone+Flag
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > This kip is for having a distinct
> > > > compaction
> > > > > > > > > attribute
> > > > > > > > > > > > "tombstone"
> > > > > > > > > > > > > flag
> > > > > > > > > > > > > > instead of relying on null value,
> > > > allowing
> > > > > > > non-null
> > > > > > > > > value
> > > > > > > > > > > > delete
> > > > > > > > > > > > > messages.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Many thanks,
> > > > > > > > > > > > > > Michael
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On 22/11/2016, 15:52, "Michael
> > > Pearce"
> > > > <
> > > > > > > > > > > michael.pea...@ig.com>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi Mayuresh,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > LGTM. Ive just made one small
> > > > adjustment
> > > > > > > > > updating the
> > > > > > > > > > > wire
> > > > > > > > > > > > > protocol to
> > > > > > > > > > > > > > show the magic byte bump.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Do we think we're good to put
> > to
> > > a
> > > > vote?
> > > > > Is
> > > > > > > > > there any
> > > > > > > > > > > > other bits
> > > > > > > > > > > > > > needing discussion?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Cheers
> > > > > > > > > > > > > > Mike
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On 21/11/2016, 18:26,
> "Mayuresh
> > > > Gharat" <
> > > > > > > > > > > > > gharatmayures...@gmail.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi Michael,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I have updated the
> > migration
> > > > section
> > > > > of
> > > > > > > the
> > > > > > > > > KIP.
> > > > > > > > > > Can
> > > > > > > > > > > > you
> > > > > > > > > > > > > please
> > > > > > > > > > > > > > take a look?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Mayuresh
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Fri, Nov 18, 2016 at
> > 9:07
> > > > AM,
> > > > > > Mayuresh
> > > > > > > > > Gharat <
> > > > > > > > > > > > > > gharatmayures...@gmail.com
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hi Michael,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > That whilst sending
> > > > tombstone and
> > > > > non
> > > > > > > > null
> > > > > > > > > value,
> > > > > > > > > > > the
> > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > can expect
> > > > > > > > > > > > > > > only to receive the
> > > non-null
> > > > > message
> > > > > > > only
> > > > > > > > > in step
> > > > > > > > > > > > (3) is
> > > > > > > > > > > > > this
> > > > > > > > > > > > > > correct?
> > > > > > > > > > > > > > > ---> I do agree with
> you
> > > > here.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Becket, Ismael : can
> you
> > > guys
> > > > > review
> > > > > > > the
> > > > > > > > > > migration
> > > > > > > > > > > > plan
> > > > > > > > > > > > > listed
> > > > > > > > > > > > > > above using
> > > > > > > > > > > > > > > magic byte?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Mayuresh
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Fri, Nov 18, 2016 at
> > > 8:58
> > > > AM,
> > > > > > > Michael
> > > > > > > > > Pearce <
> > > > > > > > > > > > > > michael.pea...@ig.com>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >> Many thanks for this
> > > > Mayuresh. I
> > > > > > don't
> > > > > > > > > have any
> > > > > > > > > > > > > objections.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> I assume we should
> > state:
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> That whilst sending
> > > > tombstone and
> > > > > > non
> > > > > > > > null
> > > > > > > > > > value,
> > > > > > > > > > > > the
> > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > can expect
> > > > > > > > > > > > > > >> only to receive the
> > > non-null
> > > > > message
> > > > > > > > only
> > > > > > > > > in
> > > > > > > > > > step
> > > > > > > > > > > > (3) is
> > > > > > > > > > > > > this
> > > > > > > > > > > > > > correct?
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Cheers
> > > > > > > > > > > > > > >> Mike
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Sent using OWA for
> > iPhone
> > > > > > > > > > > > > > >>
> > > > ______________________________
> > > > > > > > __________
> > > > > > > > > > > > > > >> From: Mayuresh Gharat
> <
> > > > > > > > > > gharatmayures...@gmail.com
> > > > > > > > > > > >
> > > > > > > > > > > > > > >> Sent: Thursday,
> November
> > > > 17, 2016
> > > > > > > > 5:18:41
> > > > > > > > > PM
> > > > > > > > > > > > > > >> To:
> > dev@kafka.apache.org
> > > > > > > > > > > > > > >> Subject: Re: [DISCUSS]
> > > > KIP-87 -
> > > > > Add
> > > > > > > > > Compaction
> > > > > > > > > > > > Tombstone
> > > > > > > > > > > > > Flag
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Hi Ismael,
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Thanks for the
> > > explanation.
> > > > > > > > > > > > > > >> Specially I like this
> > part
> > > > where
> > > > > in
> > > > > > > you
> > > > > > > > > > mentioned
> > > > > > > > > > > > we can
> > > > > > > > > > > > > get
> > > > > > > > > > > > > > rid of the
> > > > > > > > > > > > > > >> older null value
> support
> > > > for log
> > > > > > > > > compaction
> > > > > > > > > > later
> > > > > > > > > > > > on,
> > > > > > > > > > > > > here :
> > > > > > > > > > > > > > >> We can't change
> > semantics
> > > > of the
> > > > > > > message
> > > > > > > > > format
> > > > > > > > > > > > without
> > > > > > > > > > > > > having
> > > > > > > > > > > > > > a long
> > > > > > > > > > > > > > >> transition period. And
> > we
> > > > can't
> > > > > rely
> > > > > > > > > > > > > > >> on people reading
> > > > documentation or
> > > > > > > > acting
> > > > > > > > > on a
> > > > > > > > > > > > warning for
> > > > > > > > > > > > > > something so
> > > > > > > > > > > > > > >> fundamental. As such,
> my
> > > > take is
> > > > > > that
> > > > > > > we
> > > > > > > > > need to
> > > > > > > > > > > > bump the
> > > > > > > > > > > > > magic
> > > > > > > > > > > > > > byte. The
> > > > > > > > > > > > > > >> good news is
> > > > > > > > > > > > > > >> that we don't have to
> > > > support all
> > > > > > > > versions
> > > > > > > > > > > forever.
> > > > > > > > > > > > We
> > > > > > > > > > > > > have
> > > > > > > > > > > > > > said that we
> > > > > > > > > > > > > > >> will support direct
> > > > upgrades for 2
> > > > > > > > years.
> > > > > > > > > That
> > > > > > > > > > > > means that
> > > > > > > > > > > > > > message format
> > > > > > > > > > > > > > >> version n could, in
> > > theory,
> > > > be
> > > > > > > removed 2
> > > > > > > > > years
> > > > > > > > > > > > after the
> > > > > > > > > > > > > it's
> > > > > > > > > > > > > > introduced.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Just a heads up, I
> would
> > > > like to
> > > > > > > mention
> > > > > > > > > that
> > > > > > > > > > even
> > > > > > > > > > > > without
> > > > > > > > > > > > > > bumping magic
> > > > > > > > > > > > > > >> byte, we will *NOT*
> > loose
> > > > zero
> > > > > copy
> > > > > > as
> > > > > > > > in
> > > > > > > > > the
> > > > > > > > > > > > client(x+1)
> > > > > > > > > > > > > in my
> > > > > > > > > > > > > > >> explanation
> > > > > > > > > > > > > > >> above will convert
> > > > internally a
> > > > > null
> > > > > > > > > value to
> > > > > > > > > > > have a
> > > > > > > > > > > > > tombstone
> > > > > > > > > > > > > > bit set and
> > > > > > > > > > > > > > >> a tombstone bit set to
> > > have
> > > > a null
> > > > > > > value
> > > > > > > > > > > > automatically
> > > > > > > > > > > > > > internally and by
> > > > > > > > > > > > > > >> the time we move to
> > > version
> > > > (x+2),
> > > > > > the
> > > > > > > > > clients
> > > > > > > > > > > > would have
> > > > > > > > > > > > > > upgraded.
> > > > > > > > > > > > > > >> Obviously if we
> support
> > a
> > > > request
> > > > > > from
> > > > > > > > > > > consumer(x),
> > > > > > > > > > > > we
> > > > > > > > > > > > > will
> > > > > > > > > > > > > > loose zero
> > > > > > > > > > > > > > >> copy
> > > > > > > > > > > > > > >> but that is the same
> > case
> > > > with
> > > > > magic
> > > > > > > > byte.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> But if magic byte bump
> > > > makes life
> > > > > > > easier
> > > > > > > > > for
> > > > > > > > > > > > transition
> > > > > > > > > > > > > for the
> > > > > > > > > > > > > > above
> > > > > > > > > > > > > > >> reasons that you
> > > explained,
> > > > I am
> > > > > OK
> > > > > > > with
> > > > > > > > > it
> > > > > > > > > > since
> > > > > > > > > > > > we are
> > > > > > > > > > > > > going
> > > > > > > > > > > > > > to meet the
> > > > > > > > > > > > > > >> end goal down the road
> > :)
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> On a side note can we
> > > > update the
> > > > > doc
> > > > > > > > here
> > > > > > > > > on
> > > > > > > > > > magic
> > > > > > > > > > > > byte
> > > > > > > > > > > > > to say
> > > > > > > > > > > > > > that "*it
> > > > > > > > > > > > > > >> should be bumped
> > whenever
> > > > the
> > > > > > message
> > > > > > > > > format is
> > > > > > > > > > > > changed
> > > > > > > > > > > > > or the
> > > > > > > > > > > > > > >> interpretation of
> > message
> > > > format
> > > > > > > (usage
> > > > > > > > > of the
> > > > > > > > > > > > reserved
> > > > > > > > > > > > > bits as
> > > > > > > > > > > > > > well) is
> > > > > > > > > > > > > > >> changed*".
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Hi Michael,
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Here is the update
> plan
> > > > that we
> > > > > > > > discussed
> > > > > > > > > > offline
> > > > > > > > > > > > > yesterday :
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Currently the
> magic-byte
> > > > which
> > > > > > > > > corresponds to
> > > > > > > > > > the
> > > > > > > > > > > > > > "message.format.version"
> > > > > > > > > > > > > > >> is set to 1.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> 1) On broker it will
> be
> > > set
> > > > to 1
> > > > > > > > > initially.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> 2) When a producer
> > client
> > > > sends a
> > > > > > > > message
> > > > > > > > > with
> > > > > > > > > > > > magic-byte
> > > > > > > > > > > > > = 2,
> > > > > > > > > > > > > > since the
> > > > > > > > > > > > > > >> broker is on
> magic-byte
> > =
> > > > 1, we
> > > > > will
> > > > > > > > down
> > > > > > > > > > convert
> > > > > > > > > > > > it,
> > > > > > > > > > > > > which
> > > > > > > > > > > > > > means if the
> > > > > > > > > > > > > > >> tombstone bit is set,
> > the
> > > > value
> > > > > will
> > > > > > > be
> > > > > > > > > set to
> > > > > > > > > > > > null. A
> > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > >> understanding
> > magic-byte =
> > > > 1, will
> > > > > > > still
> > > > > > > > > work
> > > > > > > > > > with
> > > > > > > > > > > > this. A
> > > > > > > > > > > > > > consumer
> > > > > > > > > > > > > > >> working
> > > > > > > > > > > > > > >> with magic-byte =2
> will
> > > > also be
> > > > > able
> > > > > > > to
> > > > > > > > > > understand
> > > > > > > > > > > > this,
> > > > > > > > > > > > > since
> > > > > > > > > > > > > > it
> > > > > > > > > > > > > > >> understands the
> > tombstone.
> > > > > > > > > > > > > > >> Now there is still the
> > > > question of
> > > > > > > > > supporting a
> > > > > > > > > > > > > non-tombstone
> > > > > > > > > > > > > > and null
> > > > > > > > > > > > > > >> value from producer
> > client
> > > > with
> > > > > > > > > magic-byte = 2.*
> > > > > > > > > > > (I
> > > > > > > > > > > > am
> > > > > > > > > > > > > not sure
> > > > > > > > > > > > > > if we
> > > > > > > > > > > > > > >> should support this.
> > > > Ismael/Becket
> > > > > > can
> > > > > > > > > comment
> > > > > > > > > > > > here)*
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> 3) When almost all the
> > > > clients
> > > > > have
> > > > > > > > > upgraded,
> > > > > > > > > > the
> > > > > > > > > > > > > > message.format.version
> > > > > > > > > > > > > > >> on
> > > > > > > > > > > > > > >> the broker can be
> > changed
> > > > to 2,
> > > > > > where
> > > > > > > in
> > > > > > > > > the
> > > > > > > > > > down
> > > > > > > > > > > > > conversion in
> > > > > > > > > > > > > > the above
> > > > > > > > > > > > > > >> step will not happen.
> If
> > > at
> > > > this
> > > > > > point
> > > > > > > > we
> > > > > > > > > get a
> > > > > > > > > > > > consumer
> > > > > > > > > > > > > > request from a
> > > > > > > > > > > > > > >> older consumer, we
> might
> > > > have to
> > > > > > down
> > > > > > > > > convert
> > > > > > > > > > > where
> > > > > > > > > > > > in we
> > > > > > > > > > > > > loose
> > > > > > > > > > > > > > zero copy,
> > > > > > > > > > > > > > >> but these cases should
> > be
> > > > rare.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Becket can you review
> > this
> > > > plan
> > > > > and
> > > > > > > add
> > > > > > > > > more
> > > > > > > > > > > > details if I
> > > > > > > > > > > > > have
> > > > > > > > > > > > > > >> missed/wronged
> > something,
> > > > before
> > > > > we
> > > > > > > put
> > > > > > > > > it on
> > > > > > > > > > KIP.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Thanks,
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Mayuresh
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> On Wed, Nov 16, 2016
> at
> > > > 11:07 PM,
> > > > > > > > Michael
> > > > > > > > > > Pearce <
> > > > > > > > > > > > > > michael.pea...@ig.com>
> > > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> > Thanks guys, for
> > > > discussing this
> > > > > > > > > offline and
> > > > > > > > > > > > getting
> > > > > > > > > > > > > some
> > > > > > > > > > > > > > consensus.
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > So its clear for
> > myself
> > > > and
> > > > > others
> > > > > > > > what
> > > > > > > > > is
> > > > > > > > > > > > proposed now
> > > > > > > > > > > > > (i
> > > > > > > > > > > > > > think i
> > > > > > > > > > > > > > >> > understand, but want
> > to
> > > > make
> > > > > sure)
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > Could i ask either
> > > > directly
> > > > > update
> > > > > > > the
> > > > > > > > > kip to
> > > > > > > > > > > > detail the
> > > > > > > > > > > > > > migration
> > > > > > > > > > > > > > >> > strategy, or
> > (re-)state
> > > > your
> > > > > > offline
> > > > > > > > > discussed
> > > > > > > > > > > and
> > > > > > > > > > > > > agreed
> > > > > > > > > > > > > > migration
> > > > > > > > > > > > > > >> > strategy based on a
> > > magic
> > > > byte
> > > > > is
> > > > > > in
> > > > > > > > > this
> > > > > > > > > > > thread.
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > The main original
> > driver
> > > > for the
> > > > > > KIP
> > > > > > > > > was to
> > > > > > > > > > > > support
> > > > > > > > > > > > > > compaction where
> > > > > > > > > > > > > > >> value
> > > > > > > > > > > > > > >> > isn't null, based
> off
> > > the
> > > > > > > discussions
> > > > > > > > on
> > > > > > > > > > KIP-82
> > > > > > > > > > > > thread.
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > We should be able to
> > > > support
> > > > > > > > > non-tombstone +
> > > > > > > > > > > null
> > > > > > > > > > > > value
> > > > > > > > > > > > > by the
> > > > > > > > > > > > > > >> completion
> > > > > > > > > > > > > > >> > of the KIP, as we
> > noted
> > > > when
> > > > > > > > discussing
> > > > > > > > > this
> > > > > > > > > > > kip,
> > > > > > > > > > > > having
> > > > > > > > > > > > > > logic based on
> > > > > > > > > > > > > > >> a
> > > > > > > > > > > > > > >> > null value isn't
> very
> > > > clean and
> > > > > > also
> > > > > > > > > separates
> > > > > > > > > > > the
> > > > > > > > > > > > > concerns.
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > As discussed already
> > > > though we
> > > > > can
> > > > > > > > > split this
> > > > > > > > > > > into
> > > > > > > > > > > > > KIP-87a
> > > > > > > > > > > > > > and KIP-87b
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > Where we look to
> > deliver
> > > > KIP-87a
> > > > > > on
> > > > > > > a
> > > > > > > > > > compacted
> > > > > > > > > > > > topic
> > > > > > > > > > > > > (to
> > > > > > > > > > > > > > address the
> > > > > > > > > > > > > > >> > immediate issues)
> > > > > > > > > > > > > > >> > * tombstone + null
> > value
> > > > > > > > > > > > > > >> > * tombstone +
> non-null
> > > > value
> > > > > > > > > > > > > > >> > * non-tombstone +
> > > > non-null value
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > Then we can discuss
> > once
> > > > KIP-87a
> > > > > > is
> > > > > > > > > completed
> > > > > > > > > > > > options
> > > > > > > > > > > > > later
> > > > > > > > > > > > > > and how we
> > > > > > > > > > > > > > >> > support the second
> > part
> > > > KIP-87b
> > > > > to
> > > > > > > > > deliver:
> > > > > > > > > > > > > > >> > * non-tombstone +
> null
> > > > value
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > Cheers
> > > > > > > > > > > > > > >> > Mike
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> >
> > > > ______________________________
> > > > > > > > > __________
> > > > > > > > > > > > > > >> > From: Becket Qin <
> > > > > > > > becket....@gmail.com>
> > > > > > > > > > > > > > >> > Sent: Thursday,
> > November
> > > > 17,
> > > > > 2016
> > > > > > > 1:43
> > > > > > > > > AM
> > > > > > > > > > > > > > >> > To:
> > > dev@kafka.apache.org
> > > > > > > > > > > > > > >> > Subject: Re:
> [DISCUSS]
> > > > KIP-87 -
> > > > > > Add
> > > > > > > > > Compaction
> > > > > > > > > > > > > Tombstone Flag
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > Renu, Mayuresh and I
> > had
> > > > an
> > > > > > offline
> > > > > > > > > > discussion,
> > > > > > > > > > > > and
> > > > > > > > > > > > > following
> > > > > > > > > > > > > > is a brief
> > > > > > > > > > > > > > >> > summary.
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > 1. We agreed that
> not
> > > > bumping up
> > > > > > > magic
> > > > > > > > > value
> > > > > > > > > > may
> > > > > > > > > > > > result
> > > > > > > > > > > > > in
> > > > > > > > > > > > > > losing zero
> > > > > > > > > > > > > > >> copy
> > > > > > > > > > > > > > >> > during migration.
> > > > > > > > > > > > > > >> > 2. Given that
> bumping
> > up
> > > > magic
> > > > > > value
> > > > > > > > is
> > > > > > > > > almost
> > > > > > > > > > > > free and
> > > > > > > > > > > > > has
> > > > > > > > > > > > > > benefit of
> > > > > > > > > > > > > > >> > avoiding potential
> > > > performance
> > > > > > > issue.
> > > > > > > > > It is
> > > > > > > > > > > > probably
> > > > > > > > > > > > > worth
> > > > > > > > > > > > > > doing.
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > One issue we still
> > need
> > > > to think
> > > > > > > about
> > > > > > > > > is
> > > > > > > > > > > whether
> > > > > > > > > > > > we
> > > > > > > > > > > > > want to
> > > > > > > > > > > > > > support a
> > > > > > > > > > > > > > >> > non-tombstone
> message
> > > > with null
> > > > > > > value.
> > > > > > > > > > > > > > >> > Currently it is not
> > > > supported by
> > > > > > > > Kafka.
> > > > > > > > > If we
> > > > > > > > > > > > allow a
> > > > > > > > > > > > > > non-tombstone null
> > > > > > > > > > > > > > >> > value message to
> exist
> > > > after
> > > > > > KIP-87.
> > > > > > > > The
> > > > > > > > > > problem
> > > > > > > > > > > > is
> > > > > > > > > > > > > that such
> > > > > > > > > > > > > > message
> > > > > > > > > > > > > > >> will
> > > > > > > > > > > > > > >> > not be supported by
> > the
> > > > > consumers
> > > > > > > > prior
> > > > > > > > > to
> > > > > > > > > > > KIP-87.
> > > > > > > > > > > > > Because a
> > > > > > > > > > > > > > null value
> > > > > > > > > > > > > > >> > will always be
> > > > interpreted to a
> > > > > > > > > tombstone.
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > One option is that
> we
> > > > keep the
> > > > > > > current
> > > > > > > > > way,
> > > > > > > > > > i.e.
> > > > > > > > > > > > do not
> > > > > > > > > > > > > > support such
> > > > > > > > > > > > > > >> > message. It would be
> > > good
> > > > to
> > > > > know
> > > > > > if
> > > > > > > > > there is
> > > > > > > > > > a
> > > > > > > > > > > > > concrete use
> > > > > > > > > > > > > > case for
> > > > > > > > > > > > > > >> such
> > > > > > > > > > > > > > >> > message. If there is
> > > not,
> > > > we can
> > > > > > > > > probably just
> > > > > > > > > > > not
> > > > > > > > > > > > > support it.
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > Thanks,
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > JIangjie (Becket)
> Qin
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > On Wed, Nov 16, 2016
> > at
> > > > 1:28 PM,
> > > > > > > > > Mayuresh
> > > > > > > > > > > Gharat <
> > > > > > > > > > > > > > >> >
> > > > gharatmayures...@gmail.com
> > > > > > > > > > > > > > >> > > wrote:
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >> > > Hi Ismael,
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > This is something
> I
> > > can
> > > > think
> > > > > of
> > > > > > > for
> > > > > > > > > > migration
> > > > > > > > > > > > plan:
> > > > > > > > > > > > > > >> > > So the migration
> > plan
> > > > can look
> > > > > > > > > something
> > > > > > > > > > like
> > > > > > > > > > > > this,
> > > > > > > > > > > > > with up
> > > > > > > > > > > > > > >> conversion :
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > 1) Currently lets
> > say
> > > > we have
> > > > > > > Broker
> > > > > > > > > at
> > > > > > > > > > > version
> > > > > > > > > > > > x.
> > > > > > > > > > > > > > >> > > 2) Currently we
> have
> > > > clients
> > > > > at
> > > > > > > > > version x.
> > > > > > > > > > > > > > >> > > 3) a) We move the
> > > > version to
> > > > > > > > > Broker(x+1) :
> > > > > > > > > > > > supports
> > > > > > > > > > > > > both
> > > > > > > > > > > > > > tombstone and
> > > > > > > > > > > > > > >> > null
> > > > > > > > > > > > > > >> > > for log
> compaction.
> > > > > > > > > > > > > > >> > > b) We upgrade
> > the
> > > > client
> > > > > to
> > > > > > > > > version
> > > > > > > > > > > > client(x+1) :
> > > > > > > > > > > > > if in
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> producer
> > > > > > > > > > > > > > >> > > client(x+1) the
> > value
> > > > is set
> > > > > to
> > > > > > > > null,
> > > > > > > > > we
> > > > > > > > > > will
> > > > > > > > > > > > > automatically
> > > > > > > > > > > > > > set the
> > > > > > > > > > > > > > >> > > Tombstone bit
> > > > internally. If
> > > > > the
> > > > > > > > > producer
> > > > > > > > > > > > client(x+1)
> > > > > > > > > > > > > sets
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > >> tombstone
> > > > > > > > > > > > > > >> > > itself, well and
> > good.
> > > > For
> > > > > > > producer
> > > > > > > > > > client(x),
> > > > > > > > > > > > the
> > > > > > > > > > > > > broker
> > > > > > > > > > > > > > will up
> > > > > > > > > > > > > > >> convert
> > > > > > > > > > > > > > >> > > to have the
> > tombstone
> > > > bit.
> > > > > > > > > Broker(x+1) is
> > > > > > > > > > > > supporting
> > > > > > > > > > > > > both.
> > > > > > > > > > > > > > Consumer
> > > > > > > > > > > > > > >> > > client(x+1) will
> be
> > > > aware of
> > > > > > this
> > > > > > > > and
> > > > > > > > > should
> > > > > > > > > > > be
> > > > > > > > > > > > able
> > > > > > > > > > > > > to
> > > > > > > > > > > > > > handle this.
> > > > > > > > > > > > > > >> For
> > > > > > > > > > > > > > >> > > consumer client(x)
> > we
> > > > will
> > > > > down
> > > > > > > > > convert the
> > > > > > > > > > > > message
> > > > > > > > > > > > > on the
> > > > > > > > > > > > > > broker
> > > > > > > > > > > > > > >> side.
> > > > > > > > > > > > > > >> > > c) At this
> point
> > > we
> > > > will
> > > > > > have
> > > > > > > to
> > > > > > > > > > specify a
> > > > > > > > > > > > > warning or
> > > > > > > > > > > > > > clearly
> > > > > > > > > > > > > > >> specify
> > > > > > > > > > > > > > >> > > in docs that this
> > > > behavior is
> > > > > > > about
> > > > > > > > > to be
> > > > > > > > > > > > changed for
> > > > > > > > > > > > > log
> > > > > > > > > > > > > > compaction.
> > > > > > > > > > > > > > >> > > 4) a) In next
> > release
> > > > of the
> > > > > > > > > Broker(x+2), we
> > > > > > > > > > > > say that
> > > > > > > > > > > > > only
> > > > > > > > > > > > > > Tombstone
> > > > > > > > > > > > > > >> is
> > > > > > > > > > > > > > >> > > used for log
> > > compaction
> > > > on the
> > > > > > > > Broker
> > > > > > > > > side.
> > > > > > > > > > > > > Clients(x+1)
> > > > > > > > > > > > > > still is
> > > > > > > > > > > > > > >> > > supported.
> > > > > > > > > > > > > > >> > > b) We upgrade
> > the
> > > > client
> > > > > to
> > > > > > > > > version
> > > > > > > > > > > > client(x+2) :
> > > > > > > > > > > > > if
> > > > > > > > > > > > > > value is set
> > > > > > > > > > > > > > >> to
> > > > > > > > > > > > > > >> > > null, tombstone
> will
> > > > not be
> > > > > set
> > > > > > > > > > automatically.
> > > > > > > > > > > > The
> > > > > > > > > > > > > client
> > > > > > > > > > > > > > will have to
> > > > > > > > > > > > > > >> > call
> > > > > > > > > > > > > > >> > > setTombstone() to
> > > > actually set
> > > > > > the
> > > > > > > > > > tombstone.
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > We should compare
> > this
> > > > > migration
> > > > > > > > plan
> > > > > > > > > with
> > > > > > > > > > the
> > > > > > > > > > > > > migration
> > > > > > > > > > > > > > plan for
> > > > > > > > > > > > > > >> magic
> > > > > > > > > > > > > > >> > > byte bump and do
> > > > whatever
> > > > > looks
> > > > > > > > good.
> > > > > > > > > > > > > > >> > > I am just worried
> > that
> > > > if we
> > > > > go
> > > > > > > down
> > > > > > > > > magic
> > > > > > > > > > > byte
> > > > > > > > > > > > route,
> > > > > > > > > > > > > > unless I am
> > > > > > > > > > > > > > >> > missing
> > > > > > > > > > > > > > >> > > something, it
> sounds
> > > > like
> > > > > kafka
> > > > > > > will
> > > > > > > > > be
> > > > > > > > > > stuck
> > > > > > > > > > > > with
> > > > > > > > > > > > > > supporting both
> > > > > > > > > > > > > > >> null
> > > > > > > > > > > > > > >> > > value and
> tombstone
> > > bit
> > > > for
> > > > > log
> > > > > > > > > compaction
> > > > > > > > > > for
> > > > > > > > > > > > life
> > > > > > > > > > > > > long,
> > > > > > > > > > > > > > which does
> > > > > > > > > > > > > > >> not
> > > > > > > > > > > > > > >> > > look like a good
> end
> > > > state.
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > Thanks,
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > Mayuresh
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > On Wed, Nov 16,
> 2016
> > > at
> > > > 9:32
> > > > > AM,
> > > > > > > > > Mayuresh
> > > > > > > > > > > > Gharat <
> > > > > > > > > > > > > > >> > >
> > > > gharatmayures...@gmail.com
> > > > > > > > > > > > > > >> > > > wrote:
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > > Hi Ismael,
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > That's a very
> good
> > > > point
> > > > > > which I
> > > > > > > > > might
> > > > > > > > > > have
> > > > > > > > > > > > not
> > > > > > > > > > > > > > considered earlier.
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > Here is a plan
> > that
> > > I
> > > > can
> > > > > > think
> > > > > > > > of:
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > Stage 1) The
> > broker
> > > > from now
> > > > > > on,
> > > > > > > > up
> > > > > > > > > > converts
> > > > > > > > > > > > the
> > > > > > > > > > > > > message
> > > > > > > > > > > > > > to have the
> > > > > > > > > > > > > > >> > > > tombstone
> marker.
> > > The
> > > > log
> > > > > > > > compaction
> > > > > > > > > > thread
> > > > > > > > > > > > does log
> > > > > > > > > > > > > > compaction
> > > > > > > > > > > > > > >> based
> > > > > > > > > > > > > > >> > on
> > > > > > > > > > > > > > >> > > > both null and
> > > > tombstone
> > > > > > marker.
> > > > > > > > > This is
> > > > > > > > > > our
> > > > > > > > > > > > > transition
> > > > > > > > > > > > > > period.
> > > > > > > > > > > > > > >> > > > Stage 2) The
> next
> > > > release we
> > > > > > > only
> > > > > > > > > say that
> > > > > > > > > > > log
> > > > > > > > > > > > > compaction
> > > > > > > > > > > > > > is based
> > > > > > > > > > > > > > >> on
> > > > > > > > > > > > > > >> > > > tombstone
> marker.
> > > > (Open
> > > > > source
> > > > > > > > > kafka makes
> > > > > > > > > > > > this as a
> > > > > > > > > > > > > > policy). By
> > > > > > > > > > > > > > >> this
> > > > > > > > > > > > > > >> > > time,
> > > > > > > > > > > > > > >> > > > the organization
> > > > which is
> > > > > > moving
> > > > > > > > to
> > > > > > > > > this
> > > > > > > > > > > > release
> > > > > > > > > > > > > will be
> > > > > > > > > > > > > > sure that
> > > > > > > > > > > > > > >> they
> > > > > > > > > > > > > > >> > > > have gone
> through
> > > the
> > > > entire
> > > > > > > > > transition
> > > > > > > > > > > > period.
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > My only goal of
> > > doing
> > > > this
> > > > > is
> > > > > > > that
> > > > > > > > > Kafka
> > > > > > > > > > > > clearly
> > > > > > > > > > > > > > specifies the end
> > > > > > > > > > > > > > >> > state
> > > > > > > > > > > > > > >> > > > about what log
> > > > compaction
> > > > > > means
> > > > > > > > (is
> > > > > > > > > it
> > > > > > > > > > null
> > > > > > > > > > > > value
> > > > > > > > > > > > > or a
> > > > > > > > > > > > > > tombstone
> > > > > > > > > > > > > > >> > marker,
> > > > > > > > > > > > > > >> > > > but not both).
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > What do you
> think?
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > Thanks,
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > Mayuresh
> > > > > > > > > > > > > > >> > > > .
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > On Wed, Nov 16,
> > 2016
> > > > at 9:17
> > > > > > AM,
> > > > > > > > > Ismael
> > > > > > > > > > > Juma <
> > > > > > > > > > > > > > ism...@juma.me.uk>
> > > > > > > > > > > > > > >> > wrote:
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > >> One comment
> > below.
> > > > > > > > > > > > > > >> > > >>
> > > > > > > > > > > > > > >> > > >> On Wed, Nov 16,
> > > 2016
> > > > at
> > > > > 5:08
> > > > > > > PM,
> > > > > > > > > Mayuresh
> > > > > > > > > > > > Gharat <
> > > > > > > > > > > > > > >> > > >>
> > > > gharatmayures...@gmail.com
> > > > > > > > > > > > > > >> > > >> > wrote:
> > > > > > > > > > > > > > >> > > >>
> > > > > > > > > > > > > > >> > > >> > - If we
> > don't
> > > > bump up
> > > > > > the
> > > > > > > > > magic
> > > > > > > > > > byte,
> > > > > > > > > > > > on the
> > > > > > > > > > > > > broker
> > > > > > > > > > > > > > side, the
> > > > > > > > > > > > > > >> > > broker
> > > > > > > > > > > > > > >> > > >> > will
> always
> > > > have to
> > > > > look
> > > > > > > at
> > > > > > > > > both
> > > > > > > > > > > > tombstone
> > > > > > > > > > > > > bit and
> > > > > > > > > > > > > > the value
> > > > > > > > > > > > > > >> when
> > > > > > > > > > > > > > >> > > do
> > > > > > > > > > > > > > >> > > >> the
> > > > > > > > > > > > > > >> > > >> >
> compaction.
> > > > Assuming
> > > > > we
> > > > > > do
> > > > > > > > > not bump
> > > > > > > > > > up
> > > > > > > > > > > > the
> > > > > > > > > > > > > magic
> > > > > > > > > > > > > > byte,
> > > > > > > > > > > > > > >> > > >> > imagine
> the
> > > > broker
> > > > > sees
> > > > > > a
> > > > > > > > > message
> > > > > > > > > > > which
> > > > > > > > > > > > does
> > > > > > > > > > > > > not
> > > > > > > > > > > > > > have a
> > > > > > > > > > > > > > >> tombstone
> > > > > > > > > > > > > > >> > > bit
> > > > > > > > > > > > > > >> > > >> > set. The
> > > broker
> > > > does
> > > > > not
> > > > > > > > know
> > > > > > > > > when
> > > > > > > > > > the
> > > > > > > > > > > > > message was
> > > > > > > > > > > > > > produced
> > > > > > > > > > > > > > >> (i.e.
> > > > > > > > > > > > > > >> > > >> > whether
> > > > > > > > > > > > > > >> > > >> > the
> message
> > > has
> > > > been
> > > > > up
> > > > > > > > > converted or
> > > > > > > > > > > > not), it
> > > > > > > > > > > > > has
> > > > > > > > > > > > > > to take a
> > > > > > > > > > > > > > >> > further
> > > > > > > > > > > > > > >> > > >> > look at
> > > > > > > > > > > > > > >> > > >> > the value
> to
> > > > see if it
> > > > > > is
> > > > > > > > > null or
> > > > > > > > > > not
> > > > > > > > > > > in
> > > > > > > > > > > > > order to
> > > > > > > > > > > > > > determine
> > > > > > > > > > > > > > >> if it
> > > > > > > > > > > > > > >> > > is
> > > > > > > > > > > > > > >> > > >> a
> > > > > > > > > > > > > > >> > > >> > tombstone.
> > The
> > > > same
> > > > > > logic
> > > > > > > > has
> > > > > > > > > to be
> > > > > > > > > > > put
> > > > > > > > > > > > on the
> > > > > > > > > > > > > > consumer as
> > > > > > > > > > > > > > >> well
> > > > > > > > > > > > > > >> > > >> because
> > > > > > > > > > > > > > >> > > >> > the
> > > > > > > > > > > > > > >> > > >> > consumer
> > does
> > > > not know
> > > > > > if
> > > > > > > > the
> > > > > > > > > > message
> > > > > > > > > > > > has
> > > > > > > > > > > > > been up
> > > > > > > > > > > > > > converted or
> > > > > > > > > > > > > > >> > not.
> > > > > > > > > > > > > > >> > > >> > - If we
> > > > upconvert
> > > > > > while
> > > > > > > > > > appending,
> > > > > > > > > > > > this is
> > > > > > > > > > > > > not
> > > > > > > > > > > > > > the case,
> > > > > > > > > > > > > > >> > right?
> > > > > > > > > > > > > > >> > > >>
> > > > > > > > > > > > > > >> > > >>
> > > > > > > > > > > > > > >> > > >> If I understand
> > you
> > > > > > correctly,
> > > > > > > > > this is
> > > > > > > > > > not
> > > > > > > > > > > > > sufficient
> > > > > > > > > > > > > > because the
> > > > > > > > > > > > > > >> log
> > > > > > > > > > > > > > >> > > may
> > > > > > > > > > > > > > >> > > >> have messages
> > > > appended
> > > > > before
> > > > > > > it
> > > > > > > > > was
> > > > > > > > > > > > upgraded to
> > > > > > > > > > > > > include
> > > > > > > > > > > > > > KIP-87.
> > > > > > > > > > > > > > >> > > >>
> > > > > > > > > > > > > > >> > > >> Ismael
> > > > > > > > > > > > > > >> > > >>
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > > > --
> > > > > > > > > > > > > > >> > > > -Regards,
> > > > > > > > > > > > > > >> > > > Mayuresh R.
> Gharat
> > > > > > > > > > > > > > >> > > > (862) 250-7125
> > > > > > > > > > > > > > >> > > >
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > > --
> > > > > > > > > > > > > > >> > > -Regards,
> > > > > > > > > > > > > > >> > > Mayuresh R. Gharat
> > > > > > > > > > > > > > >> > > (862) 250-7125
> > > > > > > > > > > > > > >> > >
> > > > > > > > > > > > > > >> > The information
> > > contained
> > > > in
> > > > > this
> > > > > > > > email
> > > > > > > > > is
> > > > > > > > > > > > strictly
> > > > > > > > > > > > > > confidential and for
> > > > > > > > > > > > > > >> > the use of the
> > addressee
> > > > only,
> > > > > > > unless
> > > > > > > > > > otherwise
> > > > > > > > > > > > > indicated. If
> > > > > > > > > > > > > > you are
> > > > > > > > > > > > > > >> not
> > > > > > > > > > > > > > >> > the intended
> > recipient,
> > > > please
> > > > > do
> > > > > > > not
> > > > > > > > > read,
> > > > > > > > > > > copy,
> > > > > > > > > > > > use or
> > > > > > > > > > > > > > disclose to
> > > > > > > > > > > > > > >> others
> > > > > > > > > > > > > > >> > this message or any
> > > > attachment.
> > > > > > > Please
> > > > > > > > > also
> > > > > > > > > > > > notify the
> > > > > > > > > > > > > sender
> > > > > > > > > > > > > > by
> > > > > > > > > > > > > > >> replying
> > > > > > > > > > > > > > >> > to this email or by
> > > > telephone
> > > > > > > (+44(020
> > > > > > > > > 7896
> > > > > > > > > > > 0011)
> > > > > > > > > > > > and
> > > > > > > > > > > > > then
> > > > > > > > > > > > > > delete the
> > > > > > > > > > > > > > >> email
> > > > > > > > > > > > > > >> > and any copies of
> it.
> > > > Opinions,
> > > > > > > > > conclusion
> > > > > > > > > > (etc)
> > > > > > > > > > > > that
> > > > > > > > > > > > > do not
> > > > > > > > > > > > > > relate to
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > official business of
> > > this
> > > > > company
> > > > > > > > shall
> > > > > > > > > be
> > > > > > > > > > > > understood as
> > > > > > > > > > > > > > neither given
> > > > > > > > > > > > > > >> nor
> > > > > > > > > > > > > > >> > endorsed by it. IG
> is
> > a
> > > > trading
> > > > > > name
> > > > > > > > of
> > > > > > > > > IG
> > > > > > > > > > > Markets
> > > > > > > > > > > > > Limited (a
> > > > > > > > > > > > > > company
> > > > > > > > > > > > > > >> > registered in
> England
> > > and
> > > > Wales,
> > > > > > > > company
> > > > > > > > > > number
> > > > > > > > > > > > > 04008957) and
> > > > > > > > > > > > > > IG Index
> > > > > > > > > > > > > > >> > Limited (a company
> > > > registered in
> > > > > > > > > England and
> > > > > > > > > > > > Wales,
> > > > > > > > > > > > > company
> > > > > > > > > > > > > > number
> > > > > > > > > > > > > > >> > 01190902).
> Registered
> > > > address at
> > > > > > > > Cannon
> > > > > > > > > Bridge
> > > > > > > > > > > > House, 25
> > > > > > > > > > > > > > Dowgate Hill,
> > > > > > > > > > > > > > >> > London EC4R 2YA.
> Both
> > IG
> > > > Markets
> > > > > > > > Limited
> > > > > > > > > > > (register
> > > > > > > > > > > > > number
> > > > > > > > > > > > > > 195355) and IG
> > > > > > > > > > > > > > >> > Index Limited
> > (register
> > > > number
> > > > > > > 114059)
> > > > > > > > > are
> > > > > > > > > > > > authorised
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > regulated by
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >> > Financial Conduct
> > > > Authority.
> > > > > > > > > > > > > > >> >
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> --
> > > > > > > > > > > > > > >> -Regards,
> > > > > > > > > > > > > > >> Mayuresh R. Gharat
> > > > > > > > > > > > > > >> (862) 250-7125
> > > > > > > > > > > > > > >> The information
> > contained
> > > > in this
> > > > > > > email
> > > > > > > > is
> > > > > > > > > > > strictly
> > > > > > > > > > > > > > confidential and for
> > > > > > > > > > > > > > >> the use of the
> addressee
> > > > only,
> > > > > > unless
> > > > > > > > > otherwise
> > > > > > > > > > > > > indicated. If
> > > > > > > > > > > > > > you are not
> > > > > > > > > > > > > > >> the intended
> recipient,
> > > > please do
> > > > > > not
> > > > > > > > > read,
> > > > > > > > > > copy,
> > > > > > > > > > > > use or
> > > > > > > > > > > > > > disclose to others
> > > > > > > > > > > > > > >> this message or any
> > > > attachment.
> > > > > > Please
> > > > > > > > > also
> > > > > > > > > > notify
> > > > > > > > > > > > the
> > > > > > > > > > > > > sender
> > > > > > > > > > > > > > by replying
> > > > > > > > > > > > > > >> to this email or by
> > > > telephone
> > > > > > (+44(020
> > > > > > > > > 7896
> > > > > > > > > > 0011)
> > > > > > > > > > > > and then
> > > > > > > > > > > > > > delete the email
> > > > > > > > > > > > > > >> and any copies of it.
> > > > Opinions,
> > > > > > > > > conclusion (etc)
> > > > > > > > > > > > that do
> > > > > > > > > > > > > not
> > > > > > > > > > > > > > relate to the
> > > > > > > > > > > > > > >> official business of
> > this
> > > > company
> > > > > > > shall
> > > > > > > > be
> > > > > > > > > > > > understood as
> > > > > > > > > > > > > > neither given nor
> > > > > > > > > > > > > > >> endorsed by it. IG is
> a
> > > > trading
> > > > > name
> > > > > > > of
> > > > > > > > IG
> > > > > > > > > > Markets
> > > > > > > > > > > > > Limited (a
> > > > > > > > > > > > > > company
> > > > > > > > > > > > > > >> registered in England
> > and
> > > > Wales,
> > > > > > > company
> > > > > > > > > number
> > > > > > > > > > > > 04008957)
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > IG Index
> > > > > > > > > > > > > > >> Limited (a company
> > > > registered in
> > > > > > > England
> > > > > > > > > and
> > > > > > > > > > > Wales,
> > > > > > > > > > > > > company
> > > > > > > > > > > > > > number
> > > > > > > > > > > > > > >> 01190902). Registered
> > > > address at
> > > > > > > Cannon
> > > > > > > > > Bridge
> > > > > > > > > > > > House, 25
> > > > > > > > > > > > > > Dowgate Hill,
> > > > > > > > > > > > > > >> London EC4R 2YA. Both
> IG
> > > > Markets
> > > > > > > Limited
> > > > > > > > > > (register
> > > > > > > > > > > > number
> > > > > > > > > > > > > > 195355) and IG
> > > > > > > > > > > > > > >> Index Limited
> (register
> > > > number
> > > > > > 114059)
> > > > > > > > are
> > > > > > > > > > > > authorised and
> > > > > > > > > > > > > > regulated by the
> > > > > > > > > > > > > > >> Financial Conduct
> > > Authority.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > -Regards,
> > > > > > > > > > > > > > > Mayuresh R. Gharat
> > > > > > > > > > > > > > > (862) 250-7125
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > -Regards,
> > > > > > > > > > > > > > Mayuresh R. Gharat
> > > > > > > > > > > > > > (862) 250-7125
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The information contained in
> > this
> > > > email
> > > > > is
> > > > > > > > > strictly
> > > > > > > > > > > > confidential
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > for the use of the addressee
> only,
> > > > unless
> > > > > > > otherwise
> > > > > > > > > > > indicated.
> > > > > > > > > > > > If
> > > > > > > > > > > > > you are
> > > > > > > > > > > > > > not the intended recipient,
> please
> > do
> > > > not
> > > > > read,
> > > > > > > > > copy, use
> > > > > > > > > > or
> > > > > > > > > > > > > disclose to
> > > > > > > > > > > > > > others this message or any
> > > attachment.
> > > > Please
> > > > > > > also
> > > > > > > > > notify
> > > > > > > > > > the
> > > > > > > > > > > > sender
> > > > > > > > > > > > > by
> > > > > > > > > > > > > > replying to this email or by
> > > telephone
> > > > > (+44(020
> > > > > > > > 7896
> > > > > > > > > 0011)
> > > > > > > > > > > and
> > > > > > > > > > > > then
> > > > > > > > > > > > > delete
> > > > > > > > > > > > > > the email and any copies of it.
> > > > Opinions,
> > > > > > > > conclusion
> > > > > > > > > (etc)
> > > > > > > > > > > > that do
> > > > > > > > > > > > > not
> > > > > > > > > > > > > > relate to the official business
> of
> > > this
> > > > > company
> > > > > > > > > shall be
> > > > > > > > > > > > understood
> > > > > > > > > > > > > as
> > > > > > > > > > > > > > neither given nor endorsed by it.
> > IG
> > > > is a
> > > > > > trading
> > > > > > > > > name of
> > > > > > > > > > IG
> > > > > > > > > > > > Markets
> > > > > > > > > > > > > > Limited (a company registered in
> > > > England and
> > > > > > > Wales,
> > > > > > > > > company
> > > > > > > > > > > > number
> > > > > > > > > > > > > > 04008957) and IG Index Limited (a
> > > > company
> > > > > > > > registered
> > > > > > > > > in
> > > > > > > > > > > > England and
> > > > > > > > > > > > > Wales,
> > > > > > > > > > > > > > company number 01190902).
> > Registered
> > > > address
> > > > > at
> > > > > > > > > Cannon
> > > > > > > > > > Bridge
> > > > > > > > > > > > House,
> > > > > > > > > > > > > 25
> > > > > > > > > > > > > > Dowgate Hill, London EC4R 2YA.
> Both
> > > IG
> > > > > Markets
> > > > > > > > > Limited
> > > > > > > > > > > > (register
> > > > > > > > > > > > > number
> > > > > > > > > > > > > > 195355) and IG Index Limited
> > > (register
> > > > number
> > > > > > > > > 114059) are
> > > > > > > > > > > > authorised
> > > > > > > > > > > > > and
> > > > > > > > > > > > > > regulated by the Financial
> Conduct
> > > > Authority.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > The information contained in this email
> > is
> > > > strictly
> > > > > > > > > confidential
> > > > > > > > > > > and
> > > > > > > > > > > > for
> > > > > > > > > > > > > the use of the addressee only, unless
> > > > otherwise
> > > > > > > > indicated.
> > > > > > > > > If you
> > > > > > > > > > > > are not
> > > > > > > > > > > > > the intended recipient, please do not
> > read,
> > > > copy,
> > > > > use
> > > > > > > or
> > > > > > > > > disclose
> > > > > > > > > > > to
> > > > > > > > > > > > others
> > > > > > > > > > > > > this message or any attachment. Please
> > also
> > > > notify
> > > > > > the
> > > > > > > > > sender by
> > > > > > > > > > > > replying
> > > > > > > > > > > > > to this email or by telephone (+44(020
> > 7896
> > > > 0011
> > > > > <020%207896%200011>) and
> > > > > > > > then
> > > > > > > > > delete
> > > > > > > > > > > > the email
> > > > > > > > > > > > > and any copies of it. Opinions,
> > conclusion
> > > > (etc)
> > > > > that
> > > > > > > do
> > > > > > > > > not
> > > > > > > > > > relate
> > > > > > > > > > > > to the
> > > > > > > > > > > > > official business of this company shall
> > be
> > > > > understood
> > > > > > > as
> > > > > > > > > neither
> > > > > > > > > > > > given nor
> > > > > > > > > > > > > endorsed by it. IG is a trading name of
> > IG
> > > > Markets
> > > > > > > > Limited
> > > > > > > > > (a
> > > > > > > > > > > company
> > > > > > > > > > > > > registered in England and Wales,
> company
> > > > number
> > > > > > > 04008957)
> > > > > > > > > and IG
> > > > > > > > > > > > Index
> > > > > > > > > > > > > Limited (a company registered in
> England
> > > and
> > > > Wales,
> > > > > > > > company
> > > > > > > > > > number
> > > > > > > > > > > > > 01190902). Registered address at Cannon
> > > > Bridge
> > > > > House,
> > > > > > > 25
> > > > > > > > > Dowgate
> > > > > > > > > > > > Hill,
> > > > > > > > > > > > > London EC4R 2YA. Both IG Markets
> Limited
> > > > (register
> > > > > > > number
> > > > > > > > > 195355)
> > > > > > > > > > > > and IG
> > > > > > > > > > > > > Index Limited (register number 114059)
> > are
> > > > > authorised
> > > > > > > and
> > > > > > > > > > regulated
> > > > > > > > > > > > by the
> > > > > > > > > > > > > Financial Conduct Authority.
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > The information contained in this email is
> > > strictly
> > > > > > > > confidential
> > > > > > > > > and
> > > > > > > > > > for
> > > > > > > > > > > > the use of the addressee only, unless
> otherwise
> > > > > indicated.
> > > > > > If
> > > > > > > > > you are
> > > > > > > > > > not
> > > > > > > > > > > > the intended recipient, please do not read,
> > copy,
> > > > use or
> > > > > > > > > disclose to
> > > > > > > > > > > others
> > > > > > > > > > > > this message or any attachment. Please also
> > > notify
> > > > the
> > > > > > sender
> > > > > > > > by
> > > > > > > > > > replying
> > > > > > > > > > > > to this email or by telephone (+44(020 7896
> > 0011
> > > > > <020%207896%200011>) and then
> > > > > > > > > delete the
> > > > > > > > > > > email
> > > > > > > > > > > > and any copies of it. Opinions, conclusion
> > (etc)
> > > > that do
> > > > > > not
> > > > > > > > > relate to
> > > > > > > > > > > the
> > > > > > > > > > > > official business of this company shall be
> > > > understood as
> > > > > > > > neither
> > > > > > > > > given
> > > > > > > > > > > nor
> > > > > > > > > > > > endorsed by it. IG is a trading name of IG
> > > Markets
> > > > > Limited
> > > > > > (a
> > > > > > > > > company
> > > > > > > > > > > > registered in England and Wales, company
> number
> > > > 04008957)
> > > > > > and
> > > > > > > > IG
> > > > > > > > > Index
> > > > > > > > > > > > Limited (a company registered in England and
> > > Wales,
> > > > > company
> > > > > > > > > number
> > > > > > > > > > > > 01190902). Registered address at Cannon
> Bridge
> > > > House, 25
> > > > > > > > Dowgate
> > > > > > > > > Hill,
> > > > > > > > > > > > London EC4R 2YA. Both IG Markets Limited
> > > (register
> > > > number
> > > > > > > > > 195355) and
> > > > > > > > > > IG
> > > > > > > > > > > > Index Limited (register number 114059) are
> > > > authorised and
> > > > > > > > > regulated by
> > > > > > > > > > > the
> > > > > > > > > > > > Financial Conduct Authority.
> > > > > > > > > > > >
> > > > > > > > > > > The information contained in this email is
> > strictly
> > > > > > > confidential
> > > > > > > > > and for
> > > > > > > > > > > the use of the addressee only, unless otherwise
> > > > indicated.
> > > > > If
> > > > > > > you
> > > > > > > > > are not
> > > > > > > > > > > the intended recipient, please do not read,
> copy,
> > > > use or
> > > > > > > disclose
> > > > > > > > > to
> > > > > > > > > > others
> > > > > > > > > > > this message or any attachment. Please also
> > notify
> > > > the
> > > > > sender
> > > > > > > by
> > > > > > > > > replying
> > > > > > > > > > > to this email or by telephone (+44(020 7896
> 0011
> > > > > <020%207896%200011>) and then
> > > > > > > delete
> > > > > > > > > the
> > > > > > > > > > email
> > > > > > > > > > > and any copies of it. Opinions, conclusion
> (etc)
> > > > that do
> > > > > not
> > > > > > > > > relate to
> > > > > > > > > > the
> > > > > > > > > > > official business of this company shall be
> > > > understood as
> > > > > > > neither
> > > > > > > > > given
> > > > > > > > > > nor
> > > > > > > > > > > endorsed by it. IG is a trading name of IG
> > Markets
> > > > Limited
> > > > > (a
> > > > > > > > > company
> > > > > > > > > > > registered in England and Wales, company number
> > > > 04008957)
> > > > > and
> > > > > > > IG
> > > > > > > > > Index
> > > > > > > > > > > Limited (a company registered in England and
> > Wales,
> > > > company
> > > > > > > > number
> > > > > > > > > > > 01190902). Registered address at Cannon Bridge
> > > > House, 25
> > > > > > > Dowgate
> > > > > > > > > Hill,
> > > > > > > > > > > London EC4R 2YA. Both IG Markets Limited
> > (register
> > > > number
> > > > > > > 195355)
> > > > > > > > > and IG
> > > > > > > > > > > Index Limited (register number 114059) are
> > > > authorised and
> > > > > > > > > regulated by
> > > > > > > > > > the
> > > > > > > > > > > Financial Conduct Authority.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > The information contained in this email is
> strictly
> > > > > > confidential
> > > > > > > > and
> > > > > > > > > for
> > > > > > > > > > the use of the addressee only, unless otherwise
> > > > indicated. If
> > > > > > you
> > > > > > > > > are not
> > > > > > > > > > the intended recipient, please do not read, copy,
> > use
> > > > or
> > > > > > disclose
> > > > > > > > to
> > > > > > > > > others
> > > > > > > > > > this message or any attachment. Please also
> notify
> > > the
> > > > sender
> > > > > > by
> > > > > > > > > replying
> > > > > > > > > > to this email or by telephone (+44(020 7896 0011
> > > > > <020%207896%200011>) and then
> > > > > > delete
> > > > > > > > > the email
> > > > > > > > > > and any copies of it. Opinions, conclusion (etc)
> > that
> > > > do not
> > > > > > > relate
> > > > > > > > > to the
> > > > > > > > > > official business of this company shall be
> > understood
> > > > as
> > > > > > neither
> > > > > > > > > given nor
> > > > > > > > > > endorsed by it. IG is a trading name of IG
> Markets
> > > > Limited (a
> > > > > > > > company
> > > > > > > > > > registered in England and Wales, company number
> > > > 04008957) and
> > > > > > IG
> > > > > > > > > Index
> > > > > > > > > > Limited (a company registered in England and
> Wales,
> > > > company
> > > > > > > number
> > > > > > > > > > 01190902). Registered address at Cannon Bridge
> > House,
> > > > 25
> > > > > > Dowgate
> > > > > > > > > Hill,
> > > > > > > > > > London EC4R 2YA. Both IG Markets Limited
> (register
> > > > number
> > > > > > 195355)
> > > > > > > > > and IG
> > > > > > > > > > Index Limited (register number 114059) are
> > authorised
> > > > and
> > > > > > > regulated
> > > > > > > > > by the
> > > > > > > > > > Financial Conduct Authority.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > The information contained in this email is strictly
> > > > confidential
> > > > > and
> > > > > > > for
> > > > > > > > > the use of the addressee only, unless otherwise
> > indicated.
> > > > If you
> > > > > are
> > > > > > > not
> > > > > > > > > the intended recipient, please do not read, copy, use
> or
> > > > disclose
> > > > > to
> > > > > > > > others
> > > > > > > > > this message or any attachment. Please also notify the
> > > > sender by
> > > > > > > replying
> > > > > > > > > to this email or by telephone (+44(020 7896 0011
> > > > > <020%207896%200011>) and then delete the
> > > > > > > > email
> > > > > > > > > and any copies of it. Opinions, conclusion (etc) that
> do
> > > not
> > > > relate
> > > > > > to
> > > > > > > > the
> > > > > > > > > official business of this company shall be understood
> as
> > > > neither
> > > > > > given
> > > > > > > > nor
> > > > > > > > > endorsed by it. IG is a trading name of IG Markets
> > Limited
> > > (a
> > > > > company
> > > > > > > > > registered in England and Wales, company number
> 04008957)
> > > > and IG
> > > > > > Index
> > > > > > > > > Limited (a company registered in England and Wales,
> > company
> > > > number
> > > > > > > > > 01190902). Registered address at Cannon Bridge House,
> 25
> > > > Dowgate
> > > > > > Hill,
> > > > > > > > > London EC4R 2YA. Both IG Markets Limited (register
> number
> > > > 195355)
> > > > > and
> > > > > > > IG
> > > > > > > > > Index Limited (register number 114059) are authorised
> and
> > > > regulated
> > > > > > by
> > > > > > > > the
> > > > > > > > > Financial Conduct Authority.
> > > > > > > > >
> > > > > > > > The information contained in this email is strictly
> > > > confidential and
> > > > > > for
> > > > > > > > the use of the addressee only, unless otherwise
> indicated.
> > If
> > > > you are
> > > > > > not
> > > > > > > > the intended recipient, please do not read, copy, use or
> > > > disclose to
> > > > > > > others
> > > > > > > > this message or any attachment. Please also notify the
> > sender
> > > > by
> > > > > > replying
> > > > > > > > to this email or by telephone (+44(020 7896 0011
> > > > <020%207896%200011>)
> > > > > and then delete the
> > > > > > > email
> > > > > > > > and any copies of it. Opinions, conclusion (etc) that do
> > not
> > > > relate
> > > > > to
> > > > > > > the
> > > > > > > > official business of this company shall be understood as
> > > > neither
> > > > > given
> > > > > > > nor
> > > > > > > > endorsed by it. IG is a trading name of IG Markets
> Limited
> > (a
> > > > company
> > > > > > > > registered in England and Wales, company number 04008957)
> > and
> > > > IG
> > > > > Index
> > > > > > > > Limited (a company registered in England and Wales,
> company
> > > > number
> > > > > > > > 01190902). Registered address at Cannon Bridge House, 25
> > > > Dowgate
> > > > > Hill,
> > > > > > > > London EC4R 2YA. Both IG Markets Limited (register number
> > > > 195355) and
> > > > > > IG
> > > > > > > > Index Limited (register number 114059) are authorised and
> > > > regulated
> > > > > by
> > > > > > > the
> > > > > > > > Financial Conduct Authority.
> > > > > > > >
> > > > > > > The information contained in this email is strictly
> > > confidential
> > > > and
> > > > > for
> > > > > > > the use of the addressee only, unless otherwise indicated.
> If
> > > > you are
> > > > > not
> > > > > > > the intended recipient, please do not read, copy, use or
> > > > disclose to
> > > > > > others
> > > > > > > this message or any attachment. Please also notify the
> sender
> > > by
> > > > > replying
> > > > > > > to this email or by telephone (+44(020 7896 0011
> > > > <020%207896%200011>)
> > > > > and then delete the
> > > > > > email
> > > > > > > and any copies of it. Opinions, conclusion (etc) that do
> not
> > > > relate to
> > > > > > the
> > > > > > > official business of this company shall be understood as
> > > neither
> > > > given
> > > > > > nor
> > > > > > > endorsed by it. IG is a trading name of IG Markets Limited
> (a
> > > > company
> > > > > > > registered in England and Wales, company number 04008957)
> and
> > > IG
> > > > Index
> > > > > > > Limited (a company registered in England and Wales, company
> > > > number
> > > > > > > 01190902). Registered address at Cannon Bridge House, 25
> > > Dowgate
> > > > Hill,
> > > > > > > London EC4R 2YA. Both IG Markets Limited (register number
> > > > 195355) and
> > > > > IG
> > > > > > > Index Limited (register number 114059) are authorised and
> > > > regulated by
> > > > > > the
> > > > > > > Financial Conduct Authority.
> > > > > > >
> > > > > > The information contained in this email is strictly
> > confidential
> > > > and for
> > > > > > the use of the addressee only, unless otherwise indicated. If
> > you
> > > > are not
> > > > > > the intended recipient, please do not read, copy, use or
> > disclose
> > > > to
> > > > > others
> > > > > > this message or any attachment. Please also notify the sender
> > by
> > > > replying
> > > > > > to this email or by telephone (+44(020 7896 0011
> > > > <020%207896%200011>)
> > > > > and then delete the email
> > > > > > and any copies of it. Opinions, conclusion (etc) that do not
> > > > relate to
> > > > > the
> > > > > > official business of this company shall be understood as
> > neither
> > > > given
> > > > > nor
> > > > > > endorsed by it. IG is a trading name of IG Markets Limited (a
> > > > company
> > > > > > registered in England and Wales, company number 04008957) and
> > IG
> > > > Index
> > > > > > Limited (a company registered in England and Wales, company
> > > number
> > > > > > 01190902). Registered address at Cannon Bridge House, 25
> > Dowgate
> > > > Hill,
> > > > > > London EC4R 2YA. Both IG Markets Limited (register number
> > 195355)
> > > > and IG
> > > > > > Index Limited (register number 114059) are authorised and
> > > > regulated by
> > > > > the
> > > > > > Financial Conduct Authority.
> > > > > >
> > > > >
> > > >
> > > >
> > > > The information contained in this email is strictly confidential and
> > for
> > > > the use of the addressee only, unless otherwise indicated. If you are
> > not
> > > > the intended recipient, please do not read, copy, use or disclose to
> > > others
> > > > this message or any attachment. Please also notify the sender by
> > replying
> > > > to this email or by telephone (+44(020 7896 0011) and then delete the
> > > email
> > > > and any copies of it. Opinions, conclusion (etc) that do not relate
> to
> > > the
> > > > official business of this company shall be understood as neither
> given
> > > nor
> > > > endorsed by it. IG is a trading name of IG Markets Limited (a company
> > > > registered in England and Wales, company number 04008957) and IG
> Index
> > > > Limited (a company registered in England and Wales, company number
> > > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate
> Hill,
> > > > London EC4R 2YA. Both IG Markets Limited (register number 195355) and
> > IG
> > > > Index Limited (register number 114059) are authorised and regulated
> by
> > > the
> > > > Financial Conduct Authority.
> > > >
> > > The information contained in this email is strictly confidential and
> for
> > > the use of the addressee only, unless otherwise indicated. If you are
> not
> > > the intended recipient, please do not read, copy, use or disclose to
> > others
> > > this message or any attachment. Please also notify the sender by
> replying
> > > to this email or by telephone (+44(020 7896 0011) and then delete the
> > email
> > > and any copies of it. Opinions, conclusion (etc) that do not relate to
> > the
> > > official business of this company shall be understood as neither given
> > nor
> > > endorsed by it. IG is a trading name of IG Markets Limited (a company
> > > registered in England and Wales, company number 04008957) and IG Index
> > > Limited (a company registered in England and Wales, company number
> > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
> > > London EC4R 2YA. Both IG Markets Limited (register number 195355) and
> IG
> > > Index Limited (register number 114059) are authorised and regulated by
> > the
> > > Financial Conduct Authority.
> > >
> > The information contained in this email is strictly confidential and for
> > the use of the addressee only, unless otherwise indicated. If you are not
> > the intended recipient, please do not read, copy, use or disclose to
> others
> > this message or any attachment. Please also notify the sender by replying
> > to this email or by telephone (+44(020 7896 0011) and then delete the
> email
> > and any copies of it. Opinions, conclusion (etc) that do not relate to
> the
> > official business of this company shall be understood as neither given
> nor
> > endorsed by it. IG is a trading name of IG Markets Limited (a company
> > registered in England and Wales, company number 04008957) and IG Index
> > Limited (a company registered in England and Wales, company number
> > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill,
> > London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG
> > Index Limited (register number 114059) are authorised and regulated by
> the
> > Financial Conduct Authority.
> >
>
