I am not sure if we are bumping magicByte value as the KIP does not mention it (If I didn't miss anything).
@Nacho : I did not understand what you meant by : Conversion from one to the other may be possible but I would rather not have to do it. You will have to do the conversion for the scenario that Becket has mentioned above where an old consumer talks to the new broker, right? Thanks, Mayuresh On Thu, Nov 10, 2016 at 11:54 AM, Becket Qin <becket....@gmail.com> wrote: > Nacho, > > In Kafka protocol, a negative length is null, a zero length means empty > byte array. > > I am still confused about the two stages. It seems that the broker only > needs to understand three versions of messages. > 1. MagicValue=0 - no timestamp, absolute offset, no tombstone flag > 2. MagicValue=1 - with timestamp, relative offsets, no tombstone flag (null > value = tombstone) > 3. MagicValue=2 - with timestamp, relative offsets, tombstone flag > > We are talking about two flavors for 3: > 3.1 tombstone flag set = tombstone (Allows a key with null value in the > compacted topics) > 3.2 tombstone flag set OR null value = tombstone ( Do not allow a key with > null value in the compacted topics) > > No matter which flavor we choose, we just need to stick to that way of > interpretation, right? Why would we need a second stage? > > Jiangjie (Becket) Qin > > On Thu, Nov 10, 2016 at 10:37 AM, Ignacio Solis <iso...@igso.net> wrote: > > > A quick differentiation I would like to make is null is not the same as > > size 0. > > > > Many times these are made equal, but they're not. When serializing data, > > we have to make a choice in null values and many times these are > translated > > to zero length blobs. This is not really the same thing. > > > > From this perspective, what can Kafka represent and what is considered a > > valid value? > > > > If the user sends a byte array of length 0, or if the serializer sends > > something of length 0, this should be a valid value. It is not kafka's > job > > to determine what the user is trying to send. For all we know, the user > has > > a really good compression serializer that sends 0 bytes when nothing has > > changed. > > > > If the user is allowed to send null then some behavior should be defined. > > However, this should semantically be different than sending a command. It > > is possible for a null value could signify some form of delete, like > > "delete all messages with this key". However, if kafka has a goal to > write > > good, readable code, then this should not be allowed. > > > > A delete or a purge is a call that can have certain side effects or > > encounter errors that are unrelated to a send call. > > > > A couple of bullets from the Kafka style guide ( > > http://kafka.apache.org/coding-guide.html ): > > > > - Clear code is preferable to comments. When possible make your naming so > > good you don't need comments. > > - Logging, configuration, and public APIs are our "UI". Make them pretty, > > consistent, and usable. > > > > If you want sendTombstone() functionality make the protocol reflect that. > > > > Right now we're struggling because we did not have a clear separation of > > concerns. > > > > I don't have a good solution for deployment. I'm in favor of a harsh > road: > > - Add the flag/variable/header for tombstone > > - Add an API for tombstone behavior if needed > > - Error if somebody tries to API send null > > - Accept if somebody tries to API send byte[] length 0 > > - Rev broker version > > - broker accept flag/variable/header as tombstone > > - broker accept zero length values as normal messages > > > > The more gentle road would be: > > - Add the flag/variable/header for tombstone > > - Add an API for tombstone behavior if needed > > - warn/deprecate/etc if somebody sends null > > - broker accepts flags/variable/headers and zero length values as > > tombstones > > > > Conversion from one to the other may be possible but I would rather not > > have to do it. > > > > Nacho > > > > > > On Wed, Nov 9, 2016 at 9:03 PM, radai <radai.rosenbl...@gmail.com> > wrote: > > > > > my personal opinion - a log compacted topic is basically a kv-store, > so a > > > map API. > > > map.put(key, null) is not the same as map.remove(key), which to me > means > > a > > > null value should not represent a delete. a delete should be explicit > > > (meaning flag). > > > > > > > > > On Wed, Nov 9, 2016 at 11:01 AM, Mayuresh Gharat < > > > gharatmayures...@gmail.com > > > > wrote: > > > > > > > I see the reasoning and might be inclined to agree a bit : > > > > If we go to stage 2, the only difference is that we can theoretically > > > > support a null value non-tombstone message in a log compacted topic, > > but > > > I > > > > am not sure if that has any use case. > > > > > > > > But as an end goal I see that kafka should clearly specify what it > > means > > > by > > > > a tombstone : is it the attribute flag OR is it the null value. If we > > > just > > > > do stage 1, I don't think we are defining the end-goal completely. > > > > Again this is more about semantics of correctness of end state. > > > > > > > > Thanks, > > > > > > > > Mayuresh > > > > > > > > On Wed, Nov 9, 2016 at 10:49 AM, Becket Qin <becket....@gmail.com> > > > wrote: > > > > > > > > > I am not sure if we need the second stage. Wouldn't it be enough to > > say > > > > > that a message is a tombstone if one of the following is true? > > > > > 1. tombstone flag is set. > > > > > 2. value is null. > > > > > > > > > > If we go to stage 2, the only difference is that we can > theoretically > > > > > support a null value non-tombstone message in a log compacted > topic, > > > but > > > > I > > > > > am not sure if that has any use case. > > > > > > > > > > Thanks, > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > On Wed, Nov 9, 2016 at 9:23 AM, Mayuresh Gharat < > > > > > gharatmayures...@gmail.com> > > > > > wrote: > > > > > > > > > > > I think it will be a good idea. +1 > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > On Wed, Nov 9, 2016 at 9:13 AM, Michael Pearce < > > > michael.pea...@ig.com> > > > > > > wrote: > > > > > > > > > > > > > +1 Mayuresh, I think this is a good solution/strategy. > > > > > > > > > > > > > > Shall we update the KIP with this? Becket/Jun/Joel any comments > > to > > > > add > > > > > > > before we do? > > > > > > > > > > > > > > On 08/11/2016, 17:29, "Mayuresh Gharat" < > > > gharatmayures...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > I think the migration can be done in 2 stages : > > > > > > > > > > > > > > 1) In first stage the broker should understand the > attribute > > > flag > > > > > as > > > > > > > well > > > > > > > as Null for the value for log compaction. > > > > > > > 2) In second stage we move on to supporting only the > > attribute > > > > flag > > > > > > > for log > > > > > > > compaction. > > > > > > > > > > > > > > I agree with Becket that for older clients (consumers) the > > > broker > > > > > > might > > > > > > > have to down convert a message that has the attribute flag > > set > > > > for > > > > > > log > > > > > > > compacting but has a non null value. But this should be in > > > first > > > > > > stage. > > > > > > > Once all the clients have upgraded (clients start > recognizing > > > the > > > > > > > attribute > > > > > > > flag), we can move the broker to stage 2. > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > On Tue, Nov 8, 2016 at 12:06 AM, Michael Pearce < > > > > > > michael.pea...@ig.com > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Also we can add further guidance: > > > > > > > > > > > > > > > > To avoid the below caveat to organisations by promoting > of > > > > > > > upgrading all > > > > > > > > consumers first before relying on producing tombstone > > > messages > > > > > with > > > > > > > data > > > > > > > > > > > > > > > > Sent using OWA for iPhone > > > > > > > > ________________________________________ > > > > > > > > From: Michael Pearce > > > > > > > > Sent: Tuesday, November 8, 2016 8:03:32 AM > > > > > > > > To: dev@kafka.apache.org > > > > > > > > Subject: Re: [DISCUSS] KIP-87 - Add Compaction Tombstone > > Flag > > > > > > > > > > > > > > > > Thanks Jun on the feedback, I think I understand the > > > > issue/point > > > > > > now. > > > > > > > > > > > > > > > > We def can add that on older client version if tombstone > > > marker > > > > > > make > > > > > > > the > > > > > > > > value null to preserve behaviour. > > > > > > > > > > > > > > > > There is one caveats to this: > > > > > > > > > > > > > > > > * we have to be clear that data is lost if reading via > old > > > > > > > client/message > > > > > > > > format - I don't think this is a big issue as mostly the > > > > idea/use > > > > > > > case is > > > > > > > > around meta data transport as such would only be as bad > as > > > > > current > > > > > > > situation > > > > > > > > > > > > > > > > Re having configurable broker this was to handle cases > like > > > you > > > > > > > described > > > > > > > > but in another way by allowing organisation choose the > > > > behaviour > > > > > of > > > > > > > the > > > > > > > > compaction per broker or per topic so they could manage > > their > > > > > > > transition to > > > > > > > > using tombstone markers. > > > > > > > > > > > > > > > > On hind sight it maybe easier to just upgrade and > downgrade > > > the > > > > > > > messages > > > > > > > > on version as you propose. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Sent using OWA for iPhone > > > > > > > > ________________________________________ > > > > > > > > From: Jun Rao <j...@confluent.io> > > > > > > > > Sent: Tuesday, November 8, 2016 12:34:41 AM > > > > > > > > To: dev@kafka.apache.org > > > > > > > > Subject: Re: [DISCUSS] KIP-87 - Add Compaction Tombstone > > Flag > > > > > > > > > > > > > > > > For the use case, one potential use case is for schema > > > > > > registration. > > > > > > > For > > > > > > > > example, in Avro, a null value corresponds to a Null > > schema. > > > > So, > > > > > if > > > > > > > you > > > > > > > > want to be able to keep the schema id in a delete > message, > > > the > > > > > > value > > > > > > > can't > > > > > > > > be null. We could get around this issue by specializing > > null > > > > > value > > > > > > > during > > > > > > > > schema registration though. > > > > > > > > > > > > > > > > Now for the proposed changes. We probably should preserve > > > > client > > > > > > > > compatibility. If a client application is sending a null > > > value > > > > > to a > > > > > > > > compacted topic, ideally, it should work the same after > the > > > > > client > > > > > > > > upgrades. > > > > > > > > > > > > > > > > I am not sure about making the tombstone marker > > configurable, > > > > > > > especially at > > > > > > > > the topic level. Should we allow users to change the > config > > > > > values > > > > > > > back and > > > > > > > > forth, and what would be the implication? > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > On Mon, Nov 7, 2016 at 10:48 AM, Becket Qin < > > > > > becket....@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi Michael, > > > > > > > > > > > > > > > > > > Yes, changing the logic in the log cleaner makes sense. > > > There > > > > > > > could be > > > > > > > > some > > > > > > > > > other thing worth thinking (e.g. the message size > change > > > > after > > > > > > > > conversion), > > > > > > > > > though. > > > > > > > > > > > > > > > > > > The scenario I was thinking is the following: > > > > > > > > > Imagine a distributed caching system built on top of > > > Kafka. A > > > > > > user > > > > > > > is > > > > > > > > > consuming from a topic and it is guaranteed that if the > > > user > > > > > > > consume to > > > > > > > > the > > > > > > > > > log end it will get the latest value for all the keys. > > > > > Currently > > > > > > > if the > > > > > > > > > consumer sees a null value it knows the key has been > > > removed. > > > > > Now > > > > > > > let's > > > > > > > > say > > > > > > > > > we rolled out this change. And the producer applies a > > > message > > > > > > with > > > > > > > the > > > > > > > > > tombstone flag set, but the value was not null. When we > > > > append > > > > > > that > > > > > > > > message > > > > > > > > > to the log I suppose we will not do the down conversion > > if > > > > the > > > > > > > broker has > > > > > > > > > set the message.format.version to the latest. Because > the > > > log > > > > > > > cleaner > > > > > > > > won't > > > > > > > > > touch the active log segment, so that message will be > > > sitting > > > > > in > > > > > > > the > > > > > > > > active > > > > > > > > > segment as is. Now when a consumer that hasn't upgraded > > yet > > > > > > > consumes that > > > > > > > > > tombstone message in the active segment, it seems that > > the > > > > > broker > > > > > > > will > > > > > > > > need > > > > > > > > > to down convert that message to remove the value, > right? > > In > > > > > this > > > > > > > case, we > > > > > > > > > cannot wait for the log cleaner to do the down > conversion > > > > > because > > > > > > > that > > > > > > > > > message may have already been consumed before the log > > > > > compaction > > > > > > > happens. > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 7, 2016 at 9:59 AM, Michael Pearce < > > > > > > > michael.pea...@ig.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi Becket, > > > > > > > > > > > > > > > > > > > > We were thinking more about having the logic that’s > in > > > the > > > > > > method > > > > > > > > > > shouldRetainMessage configurable via > > > > > http://kafka.apache.org/ > > > > > > > > > > documentation.html#brokerconfigs at a broker/topic > > > level. > > > > > And > > > > > > > then > > > > > > > > > scrap > > > > > > > > > > auto converting the message, and allow organisations > to > > > > > manage > > > > > > > the > > > > > > > > > rollout > > > > > > > > > > of enabling of the feature. > > > > > > > > > > (this isn’t in documentation but in response to the > > > > > discussion > > > > > > > thread > > > > > > > > as > > > > > > > > > > an alternative approach to roll out the feature) > > > > > > > > > > > > > > > > > > > > Does this make any more sense? > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > > > On 11/3/16, 2:27 PM, "Becket Qin" < > > becket....@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Hi Michael, > > > > > > > > > > > > > > > > > > > > Do you mean using a new configuration it is just > > the > > > > > > exiting > > > > > > > > > > message.format.version config? It seems the > > > > > > > message.format.version > > > > > > > > > > config > > > > > > > > > > is enough in this case. And the default value > would > > > > > always > > > > > > > be the > > > > > > > > > > latest > > > > > > > > > > version. > > > > > > > > > > > > > > > > > > > > > Message version migration would be handled as > > like > > > in > > > > > > > KIP-32 > > > > > > > > > > > > > > > > > > > > Also just want to confirm on this. Today if an > old > > > > > consumer > > > > > > > > consumes > > > > > > > > > a > > > > > > > > > > log > > > > > > > > > > compacted topic and sees an empty value, it knows > > > that > > > > > is a > > > > > > > > > tombstone. > > > > > > > > > > After we start to use the attribute bit, a > > tombstone > > > > > > message > > > > > > > can > > > > > > > > > have a > > > > > > > > > > non-empty value. So by "like in KIP-32" you mean > we > > > > will > > > > > > > remove the > > > > > > > > > > value > > > > > > > > > > to down convert the message if the consumer > version > > > is > > > > > old, > > > > > > > right? > > > > > > > > > > > > > > > > > > > > Thanks. > > > > > > > > > > > > > > > > > > > > Jiangjie (Becket) Qin > > > > > > > > > > > > > > > > > > > > On Wed, Nov 2, 2016 at 1:37 AM, Michael Pearce < > > > > > > > > > michael.pea...@ig.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Joel , et al. > > > > > > > > > > > > > > > > > > > > > > Any comments on the below idea to handle roll > > out / > > > > > > > compatibility > > > > > > > > > of > > > > > > > > > > this > > > > > > > > > > > feature, using a configuration? > > > > > > > > > > > > > > > > > > > > > > Does it make sense/clear? > > > > > > > > > > > Does it add value? > > > > > > > > > > > Do we want to enforce flag by default, or value > > by > > > > > > > default, or > > > > > > > > > both? > > > > > > > > > > > > > > > > > > > > > > Cheers > > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 10/27/16, 4:47 PM, "Michael Pearce" < > > > > > > > michael.pea...@ig.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > Thanks, James, I think this is a really > good > > > > > addition > > > > > > > to the > > > > > > > > > KIP > > > > > > > > > > > details, please feel free to amend the wiki/add > > the > > > > use > > > > > > > cases, > > > > > > > > also > > > > > > > > > > if any > > > > > > > > > > > others you think of. I definitely think its > > > > worthwhile > > > > > > > > documenting. > > > > > > > > > > If you > > > > > > > > > > > can’t let me know ill add them next week (just > > > > leaving > > > > > > for > > > > > > > a long > > > > > > > > > > weekend > > > > > > > > > > > off) > > > > > > > > > > > > > > > > > > > > > > Re Joel and others comments about upgrade > and > > > > > > > compatibility. > > > > > > > > > > > > > > > > > > > > > > Rather than trying to auto manage this. > > > > > > > > > > > > > > > > > > > > > > Actually maybe we make a configuration > > option, > > > > both > > > > > > at > > > > > > > server > > > > > > > > > > and per > > > > > > > > > > > topic level to control the behavior of how the > > > server > > > > > > logic > > > > > > > > should > > > > > > > > > > work out > > > > > > > > > > > if the record, is a tombstone record . > > > > > > > > > > > > > > > > > > > > > > e.g. > > > > > > > > > > > > > > > > > > > > > > key = compation.tombstone.marker > > > > > > > > > > > > > > > > > > > > > > value options: > > > > > > > > > > > > > > > > > > > > > > value (continues to use null value as > > > tombstone > > > > > > > marker) > > > > > > > > > > > flag (expects to use the tombstone flag) > > > > > > > > > > > value_or_flag (if either is true it treats > > the > > > > > record > > > > > > > as a > > > > > > > > > > tombstone) > > > > > > > > > > > > > > > > > > > > > > This way on upgrade users can keep current > > > > > behavior, > > > > > > > and > > > > > > > > slowly > > > > > > > > > > > migrate to the new. Having a transition period > of > > > > using > > > > > > > > > > value_or_flag, > > > > > > > > > > > finally having flag only if an organization > > wishes > > > to > > > > > use > > > > > > > null > > > > > > > > > values > > > > > > > > > > > without it being treated as a tombstone marker > > (use > > > > > case > > > > > > > noted > > > > > > > > > below) > > > > > > > > > > > > > > > > > > > > > > Having it both global broker level and > topic > > > > > override > > > > > > > also > > > > > > > > > > allows some > > > > > > > > > > > flexibility here. > > > > > > > > > > > > > > > > > > > > > > Cheers > > > > > > > > > > > Mike > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 10/27/16, 8:03 AM, "James Cheng" < > > > > > > > wushuja...@gmail.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > This KIP would definitely address a gap > > in > > > > the > > > > > > > current > > > > > > > > > > > functionality, where you currently can't have a > > > > > tombstone > > > > > > > with > > > > > > > > any > > > > > > > > > > > associated content. > > > > > > > > > > > > > > > > > > > > > > That said, I'd like to talk about use > > > cases, > > > > to > > > > > > > make sure > > > > > > > > > > that > > > > > > > > > > > this is in fact useful. The KIP should be > updated > > > > with > > > > > > > whatever > > > > > > > > use > > > > > > > > > > cases > > > > > > > > > > > we come up with. > > > > > > > > > > > > > > > > > > > > > > First of all, an observation: When we > > speak > > > > > about > > > > > > > log > > > > > > > > > > compaction, > > > > > > > > > > > we typically think of "the latest message for a > > key > > > > is > > > > > > > retained". > > > > > > > > > In > > > > > > > > > > that > > > > > > > > > > > respect, a delete tombstone (i.e. a message > with > > a > > > > null > > > > > > > payload) > > > > > > > > is > > > > > > > > > > treated > > > > > > > > > > > the same as any other Kafka message: the latest > > > > message > > > > > > is > > > > > > > > > retained. > > > > > > > > > > It > > > > > > > > > > > doesn't matter whether the latest message is > > null, > > > or > > > > > if > > > > > > > the > > > > > > > > latest > > > > > > > > > > message > > > > > > > > > > > has actual content. In all cases, the last > > message > > > is > > > > > > > retained. > > > > > > > > > > > > > > > > > > > > > > The only way a delete tombstone is > > treated > > > > > > > differently > > > > > > > > from > > > > > > > > > > other > > > > > > > > > > > Kafka messages is that it automatically > > disappears > > > > > after > > > > > > a > > > > > > > while. > > > > > > > > > > The time > > > > > > > > > > > of deletion is specified using > > delete.retention.ms > > > . > > > > > > > > > > > > > > > > > > > > > > So what we're really talking about is, > do > > > we > > > > > want > > > > > > > to > > > > > > > > > support > > > > > > > > > > > messages in a log-compacted topic that > > auto-delete > > > > > > > themselves > > > > > > > > after > > > > > > > > > > a while? > > > > > > > > > > > > > > > > > > > > > > In a thread from 2015, there was a > > > discussion > > > > > on > > > > > > > > > first-class > > > > > > > > > > > support of headers between Roger Hoover, Felix > > GV, > > > > Jun > > > > > > > Rao, and > > > > > > > > I. > > > > > > > > > > See > > > > > > > > > > > thread at https://groups.google.com/d/ > > > > > > > msg/confluent-platform/ > > > > > > > > > > > 8xPbjyUE_7E/yQ1AeCufL_gJ < > > > > https://groups.google.com/d/ > > > > > > > > > > > msg/confluent-platform/ > 8xPbjyUE_7E/yQ1AeCufL_gJ> > > . > > > > In > > > > > > that > > > > > > > > thread, > > > > > > > > > > Jun > > > > > > > > > > > raised a good question that I didn't have a > good > > > > answer > > > > > > > for at > > > > > > > > the > > > > > > > > > > time: If > > > > > > > > > > > a message is going to auto-delete itself after > a > > > > while, > > > > > > how > > > > > > > > > > important was > > > > > > > > > > > the message? That is, what information did the > > > > message > > > > > > > contain > > > > > > > > that > > > > > > > > > > was > > > > > > > > > > > important *for a while* but not so important > that > > > it > > > > > > > needed to be > > > > > > > > > > kept > > > > > > > > > > > around forever? > > > > > > > > > > > > > > > > > > > > > > Some use cases that I can think of: > > > > > > > > > > > > > > > > > > > > > > 1) Tracability. I would like to know > who > > > > issued > > > > > > > this > > > > > > > > delete > > > > > > > > > > > tombstone. It might include the hostname, IP of > > the > > > > > > > producer of > > > > > > > > the > > > > > > > > > > delete. > > > > > > > > > > > 2) Timestamps. I would like to know > when > > > this > > > > > > > delete was > > > > > > > > > > issued. > > > > > > > > > > > This use case is already addressed by the > > > > availability > > > > > of > > > > > > > > > per-message > > > > > > > > > > > timestamps that came in 0.10.0 > > > > > > > > > > > 3) Data provenance. I hope I'm using > this > > > > > phrase > > > > > > > > correctly, > > > > > > > > > > but > > > > > > > > > > > what I mean is, where did this delete come > from? > > > What > > > > > > > processing > > > > > > > > > job > > > > > > > > > > > emitted it? What input to the processing job > > caused > > > > > this > > > > > > > delete > > > > > > > > to > > > > > > > > > be > > > > > > > > > > > produced? For example, if a record in topic A > was > > > > > > > processed and > > > > > > > > > > caused a > > > > > > > > > > > delete tombstone to be emitted to topic B, I > > might > > > > like > > > > > > the > > > > > > > > offset > > > > > > > > > > of the > > > > > > > > > > > topic A message to be attached to the topic B > > > > message. > > > > > > > > > > > 4) Distributed tracing for stream > > > topologies. > > > > > > This > > > > > > > might > > > > > > > > > be a > > > > > > > > > > > slight repeat of the above use cases. In the > > > > > > microservices > > > > > > > world, > > > > > > > > > we > > > > > > > > > > can > > > > > > > > > > > generate call-graphs of webservices using tools > > > like > > > > > > > Zipkin/ > > > > > > > > > > opentracing.io > > > > > > > > > > > <http://opentracing.io/>, or something > homegrown > > > > like > > > > > > > > > > > https://engineering.linkedin. > > > > > > com/distributed-service-call- > > > > > > > > > > > graph/real-time-distributed- > > > > > tracing-website-performance- > > > > > > > > > and-efficiency > > > > > > > > > > < > > > > > > > > > > > https://engineering.linkedin. > > > > > > com/distributed-service-call- > > > > > > > > > > > graph/real-time-distributed- > > > > > tracing-website-performance- > > > > > > > > > > and-efficiency>. > > > > > > > > > > > I can imagine that you might want to do > something > > > > > similar > > > > > > > for > > > > > > > > > stream > > > > > > > > > > > processing topologies, where stream processing > > jobs > > > > > carry > > > > > > > along > > > > > > > > and > > > > > > > > > > forward > > > > > > > > > > > along a globally unique identifier, and a > > > distributed > > > > > > > topology > > > > > > > > > graph > > > > > > > > > > is > > > > > > > > > > > generated. > > > > > > > > > > > 5) Cases where processing a delete > > requires > > > > > data > > > > > > > that is > > > > > > > > > not > > > > > > > > > > > available in the message key. I'm not sure I > > have a > > > > > good > > > > > > > example > > > > > > > > of > > > > > > > > > > this, > > > > > > > > > > > though. One hand-wavy example might be where I > am > > > > > > > publishing > > > > > > > > > > documents into > > > > > > > > > > > Kafka where the documentId is the message key, > > and > > > > the > > > > > > text > > > > > > > > > contents > > > > > > > > > > of the > > > > > > > > > > > document are in the message body. And I have a > > > > > consuming > > > > > > > job that > > > > > > > > > > does some > > > > > > > > > > > analytics on the message body. If that document > > > gets > > > > > > > deleted, > > > > > > > > then > > > > > > > > > > the > > > > > > > > > > > consuming job might need the original message > > body > > > in > > > > > > > order to > > > > > > > > > > "delete" > > > > > > > > > > > that message's impact from the analytics. But > I'm > > > not > > > > > > sure > > > > > > > that > > > > > > > > is > > > > > > > > > a > > > > > > > > > > great > > > > > > > > > > > example. If the consumer was worried about > that, > > > the > > > > > > > consumer > > > > > > > > would > > > > > > > > > > > probably keep the original message around, > stored > > > by > > > > > > > primary key. > > > > > > > > > > And then > > > > > > > > > > > all it would need from a delete message would > be > > > the > > > > > > > primary key > > > > > > > > of > > > > > > > > > > the > > > > > > > > > > > message. > > > > > > > > > > > > > > > > > > > > > > Do people think these are valid use > > cases? > > > > > > > > > > > > > > > > > > > > > > What are other use cases that people > can > > > > think > > > > > > of? > > > > > > > > > > > > > > > > > > > > > > -James > > > > > > > > > > > > > > > > > > > > > > > On Oct 26, 2016, at 3:46 PM, Mayuresh > > > > Gharat > > > > > < > > > > > > > > > > > gharatmayures...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > +1 @Joel. > > > > > > > > > > > > I think a clear migration plan of > > > upgrading > > > > > and > > > > > > > > > > downgrading of > > > > > > > > > > > server and > > > > > > > > > > > > clients along with handling of issues > > > that > > > > > Joel > > > > > > > > > mentioned, > > > > > > > > > > on > > > > > > > > > > > the KIP would > > > > > > > > > > > > be really great. > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > Mayuresh > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Oct 26, 2016 at 3:31 PM, Joel > > > > Koshy < > > > > > > > > > > jjkosh...@gmail.com> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > >> I'm not sure why it would be useful, > > but > > > > it > > > > > > > should be > > > > > > > > > > > theoretically > > > > > > > > > > > >> possible if the attribute bit alone > is > > > > > enough > > > > > > > to mark > > > > > > > > a > > > > > > > > > > > tombstone. OTOH, we > > > > > > > > > > > >> could consider that as invalid if we > > > wish. > > > > > > > These are > > > > > > > > > > relevant > > > > > > > > > > > details that > > > > > > > > > > > >> I think should be added to the KIP. > > > > > > > > > > > >> > > > > > > > > > > > >> Also, in the few odd scenarios that > I > > > > > > mentioned > > > > > > > we > > > > > > > > > should > > > > > > > > > > also > > > > > > > > > > > consider > > > > > > > > > > > >> that fetches could be coming from > > other > > > > > > > > > yet-to-be-upgraded > > > > > > > > > > > brokers in a > > > > > > > > > > > >> cluster that is being upgraded. So > we > > > > would > > > > > > > probably > > > > > > > > > want > > > > > > > > > > to > > > > > > > > > > > continue to > > > > > > > > > > > >> support nulls as tombstones or > > > > down-convert > > > > > in > > > > > > > a way > > > > > > > > > that > > > > > > > > > > we > > > > > > > > > > > are sure works > > > > > > > > > > > >> with least surprise to fetchers. > > > > > > > > > > > >> > > > > > > > > > > > >> There is a slightly vague statement > > > under > > > > > > > > > "Compatibility, > > > > > > > > > > > Deprecation, and > > > > > > > > > > > >> Migration Plan" that could benefit > > more > > > > > > details: > > > > > > > > *Logic > > > > > > > > > > would > > > > > > > > > > > base on > > > > > > > > > > > >> current behavior of null value or if > > > > > tombstone > > > > > > > flag > > > > > > > > set > > > > > > > > > to > > > > > > > > > > > true, as such > > > > > > > > > > > >> wouldn't impact any existing flows > > > simply > > > > > > allow > > > > > > > new > > > > > > > > > > producers > > > > > > > > > > > to make use > > > > > > > > > > > >> of the feature*. It is unclear to me > > > based > > > > > on > > > > > > > that > > > > > > > > > > whether you > > > > > > > > > > > would > > > > > > > > > > > >> interpret null as a tombstone if the > > > > > tombstone > > > > > > > > attribute > > > > > > > > > > bit is > > > > > > > > > > > off. > > > > > > > > > > > >> > > > > > > > > > > > >> On Wed, Oct 26, 2016 at 3:10 PM, > > Xavier > > > > > > Léauté < > > > > > > > > > > > xav...@confluent.io> > > > > > > > > > > > >> wrote: > > > > > > > > > > > >> > > > > > > > > > > > >>> Does this mean that starting with > V4 > > > > > requests > > > > > > > we > > > > > > > > would > > > > > > > > > > allow > > > > > > > > > > > storing null > > > > > > > > > > > >>> messages in compacted topics? The > KIP > > > > > should > > > > > > > probably > > > > > > > > > > clarify > > > > > > > > > > > the > > > > > > > > > > > >> behavior > > > > > > > > > > > >>> for null messages where the > tombstone > > > > flag > > > > > is > > > > > > > not > > > > > > > > net. > > > > > > > > > > > >>> > > > > > > > > > > > >>> On Wed, Oct 26, 2016 at 1:32 AM > > Magnus > > > > > > > Edenhill < > > > > > > > > > > > mag...@edenhill.se> > > > > > > > > > > > >>> wrote: > > > > > > > > > > > >>> > > > > > > > > > > > >>>> 2016-10-25 21:36 GMT+02:00 Nacho > > Solis > > > > > > > > > > > <nso...@linkedin.com.invalid>: > > > > > > > > > > > >>>> > > > > > > > > > > > >>>>> I think you probably require a > > > > MagicByte > > > > > > > bump if > > > > > > > > you > > > > > > > > > > expect > > > > > > > > > > > correct > > > > > > > > > > > >>>>> behavior of the system as a > whole. > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> From a client perspective you > want > > to > > > > > make > > > > > > > sure > > > > > > > > that > > > > > > > > > > when you > > > > > > > > > > > >> deliver a > > > > > > > > > > > >>>>> message that the broker supports > > the > > > > > > feature > > > > > > > you're > > > > > > > > > > expecting > > > > > > > > > > > >>>>> (compaction). So, depending on > the > > > > > > behavior > > > > > > > of the > > > > > > > > > > broker on > > > > > > > > > > > >>>> encountering > > > > > > > > > > > >>>>> a previously undefined bit flag I > > > would > > > > > > > suggest > > > > > > > > > making > > > > > > > > > > some > > > > > > > > > > > change to > > > > > > > > > > > >>>> make > > > > > > > > > > > >>>>> certain that flag-based > compaction > > is > > > > > > > supported. > > > > > > > > I'm > > > > > > > > > > going > > > > > > > > > > > to guess > > > > > > > > > > > >>> that > > > > > > > > > > > >>>>> the MagicByte would do this. > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> I dont believe this is needed > since > > it > > > > is > > > > > > > already > > > > > > > > > > attributed > > > > > > > > > > > through > > > > > > > > > > > >> the > > > > > > > > > > > >>>> request's API version. > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> Producer: > > > > > > > > > > > >>>> * if a client sends ProduceRequest > > V4 > > > > then > > > > > > > > > > attributes.bit5 > > > > > > > > > > > indicates a > > > > > > > > > > > >>>> tombstone > > > > > > > > > > > >>>> * if a clients sends > ProduceRequest > > > <V4 > > > > > then > > > > > > > > > > attributes.bit5 > > > > > > > > > > > is > > > > > > > > > > > >> ignored > > > > > > > > > > > >>>> and value==null indicates a > > tombstone > > > > > > > > > > > >>>> * in both cases the on-disk > messages > > > are > > > > > > > stored with > > > > > > > > > > > attributes.bit5 > > > > > > > > > > > >> (I > > > > > > > > > > > >>>> assume?) > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> Consumer: > > > > > > > > > > > >>>> * if a clients sends FetchRequest > V4 > > > > > > messages > > > > > > > are > > > > > > > > > > > sendfile():ed > > > > > > > > > > > >> directly > > > > > > > > > > > >>>> from disk (with attributes.bit5) > > > > > > > > > > > >>>> * if a client sends FetchRequest > <V4 > > > > > > messages > > > > > > > are > > > > > > > > > > slowpathed > > > > > > > > > > > and > > > > > > > > > > > >>>> translated from attributes.bit5 to > > > > > > value=null > > > > > > > as > > > > > > > > > > required. > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> That's my understanding anyway, > > please > > > > > > > correct me if > > > > > > > > > I'm > > > > > > > > > > > wrong. > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> /Magnus > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> > > > > > > > > > > > >>>> > > > > > > > > > > > >>>>> On Tue, Oct 25, 2016 at 10:17 AM, > > > > Magnus > > > > > > > Edenhill < > > > > > > > > > > > >> mag...@edenhill.se> > > > > > > > > > > > >>>>> wrote: > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>>> It is safe to assume that a > > > previously > > > > > > > undefined > > > > > > > > > > attributes > > > > > > > > > > > bit > > > > > > > > > > > >> will > > > > > > > > > > > >>> be > > > > > > > > > > > >>>>>> unset in protocol requests from > > > > existing > > > > > > > clients, > > > > > > > > if > > > > > > > > > > not, > > > > > > > > > > > such a > > > > > > > > > > > >>> client > > > > > > > > > > > >>>>> is > > > > > > > > > > > >>>>>> already violating the protocol > and > > > > needs > > > > > > to > > > > > > > be > > > > > > > > > fixed. > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>> So I dont see a need for a > > MagicByte > > > > > bump, > > > > > > > both > > > > > > > > > > broker and > > > > > > > > > > > client > > > > > > > > > > > >> has > > > > > > > > > > > >>>> the > > > > > > > > > > > >>>>>> information it needs to > construct > > or > > > > > parse > > > > > > > the > > > > > > > > > message > > > > > > > > > > > according to > > > > > > > > > > > >>>>> request > > > > > > > > > > > >>>>>> version. > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>> 2016-10-25 18:48 GMT+02:00 > Michael > > > > > Pearce > > > > > > < > > > > > > > > > > > michael.pea...@ig.com>: > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>>>> Hi Magnus, > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> I was wondering if I even > needed > > to > > > > > > change > > > > > > > those > > > > > > > > > > also, as > > > > > > > > > > > >>> technically > > > > > > > > > > > >>>>>>> we’re just making use of a non > > used > > > > > > > attribute > > > > > > > > bit, > > > > > > > > > > but im > > > > > > > > > > > not > > > > > > > > > > > >> 100% > > > > > > > > > > > >>>> that > > > > > > > > > > > >>>>>> it > > > > > > > > > > > >>>>>>> be always false currently. > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> If someone can say 100% it will > > > > already > > > > > > be > > > > > > > set > > > > > > > > > false > > > > > > > > > > with > > > > > > > > > > > current > > > > > > > > > > > >>> and > > > > > > > > > > > >>>>>>> historic bit wise masking > > > techniques > > > > > used > > > > > > > over > > > > > > > > the > > > > > > > > > > time, > > > > > > > > > > > we could > > > > > > > > > > > >>> do > > > > > > > > > > > >>>>> away > > > > > > > > > > > >>>>>>> with both, and simply just > start > > to > > > > use > > > > > > it. > > > > > > > > > > Unfortunately > > > > > > > > > > > I don’t > > > > > > > > > > > >>>> have > > > > > > > > > > > >>>>>> that > > > > > > > > > > > >>>>>>> historic knowledge so was > hoping > > it > > > > > would > > > > > > > be > > > > > > > > > flagged > > > > > > > > > > up in > > > > > > > > > > > this > > > > > > > > > > > >>>>>> discussion > > > > > > > > > > > >>>>>>> thread ? > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> Cheers > > > > > > > > > > > >>>>>>> Mike > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> On 10/25/16, 5:36 PM, "Magnus > > > > > Edenhill" < > > > > > > > > > > > mag...@edenhill.se> > > > > > > > > > > > >>> wrote: > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> Hi Michael, > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> With the version bumps for > > > Produce > > > > > and > > > > > > > Fetch > > > > > > > > > > requests, > > > > > > > > > > > do you > > > > > > > > > > > >>>>> really > > > > > > > > > > > >>>>>>> need > > > > > > > > > > > >>>>>>> to bump MagicByte too? > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> Regards, > > > > > > > > > > > >>>>>>> Magnus > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> 2016-10-25 18:09 GMT+02:00 > > > Michael > > > > > > > Pearce < > > > > > > > > > > > >>> michael.pea...@ig.com > > > > > > > > > > > >>>>> : > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>>> Hi All, > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > >>>>>>>> I would like to discuss the > > > > following > > > > > > KIP > > > > > > > > > proposal: > > > > > > > > > > > >>>>>>>> https://cwiki.apache.org/ > > > > > > > > > > confluence/display/KAFKA/KIP- > > > > > > > > > > > >>>>>>>> 87+-+Add+Compaction+Tombstone+ > > > Flag > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > >>>>>>>> This is off the back of the > > > > discussion > > > > > > on > > > > > > > KIP-82 > > > > > > > > > / > > > > > > > > > > KIP > > > > > > > > > > > >>> meeting > > > > > > > > > > > >>>>>>> where it > > > > > > > > > > > >>>>>>>> was agreed to separate this > > issue > > > > and > > > > > > > feature. > > > > > > > > > See: > > > > > > > > > > > >>>>>>>> http://mail-archives.apache. > > > > > > > > > > org/mod_mbox/kafka-dev/201610 > > > > > > > > > > > . > > > > > > > > > > > >>>>>>>> mbox/%3cCAJS3ho8OcR== > > > > > > > EcxsJ8OP99pD2hz=iiGecWsv- > > > > > > > > > > > >>>>>>>> EZsBsNyDcKr=g...@mail.gmail.com% > 3e > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > >>>>>>>> Thanks > > > > > > > > > > > >>>>>>>> Mike > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > >>>>>>>> The information contained in > > this > > > > > email > > > > > > is > > > > > > > > > strictly > > > > > > > > > > > >>>> confidential > > > > > > > > > > > >>>>>> and > > > > > > > > > > > >>>>>>> for > > > > > > > > > > > >>>>>>>> the use of the addressee only, > > > > unless > > > > > > > otherwise > > > > > > > > > > indicated. > > > > > > > > > > > >> If > > > > > > > > > > > >>>> you > > > > > > > > > > > >>>>>>> are not > > > > > > > > > > > >>>>>>>> the intended recipient, please > > do > > > > not > > > > > > > read, > > > > > > > > copy, > > > > > > > > > > use or > > > > > > > > > > > >>>> disclose > > > > > > > > > > > >>>>>> to > > > > > > > > > > > >>>>>>> others > > > > > > > > > > > >>>>>>>> this message or any > attachment. > > > > Please > > > > > > > also > > > > > > > > notify > > > > > > > > > > the > > > > > > > > > > > >> sender > > > > > > > > > > > >>>> by > > > > > > > > > > > >>>>>>> replying > > > > > > > > > > > >>>>>>>> to this email or by telephone > > > > (+44(020 > > > > > > > 7896 > > > > > > > > 0011) > > > > > > > > > > and then > > > > > > > > > > > >>>> delete > > > > > > > > > > > >>>>>>> the email > > > > > > > > > > > >>>>>>>> and any copies of it. > Opinions, > > > > > > > conclusion (etc) > > > > > > > > > > that do > > > > > > > > > > > >> not > > > > > > > > > > > >>>>> relate > > > > > > > > > > > >>>>>>> to the > > > > > > > > > > > >>>>>>>> official business of this > > company > > > > > shall > > > > > > be > > > > > > > > > > understood as > > > > > > > > > > > >>>> neither > > > > > > > > > > > >>>>>>> given nor > > > > > > > > > > > >>>>>>>> endorsed by it. IG is a > trading > > > name > > > > > of > > > > > > IG > > > > > > > > Markets > > > > > > > > > > Limited > > > > > > > > > > > >> (a > > > > > > > > > > > >>>>>> company > > > > > > > > > > > >>>>>>>> registered in England and > Wales, > > > > > company > > > > > > > number > > > > > > > > > > 04008957) > > > > > > > > > > > >> and > > > > > > > > > > > >>>> IG > > > > > > > > > > > >>>>>>> Index > > > > > > > > > > > >>>>>>>> Limited (a company registered > in > > > > > England > > > > > > > and > > > > > > > > > Wales, > > > > > > > > > > > company > > > > > > > > > > > >>>>> number > > > > > > > > > > > >>>>>>>> 01190902). Registered address > at > > > > > Cannon > > > > > > > Bridge > > > > > > > > > > House, 25 > > > > > > > > > > > >>>> Dowgate > > > > > > > > > > > >>>>>>> Hill, > > > > > > > > > > > >>>>>>>> London EC4R 2YA. Both IG > Markets > > > > > Limited > > > > > > > > (register > > > > > > > > > > number > > > > > > > > > > > >>>> 195355) > > > > > > > > > > > >>>>>>> and IG > > > > > > > > > > > >>>>>>>> Index Limited (register number > > > > 114059) > > > > > > are > > > > > > > > > > authorised and > > > > > > > > > > > >>>>> regulated > > > > > > > > > > > >>>>>>> by the > > > > > > > > > > > >>>>>>>> Financial Conduct Authority. > > > > > > > > > > > >>>>>>>> > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>>> The information contained in > this > > > > email > > > > > > is > > > > > > > > strictly > > > > > > > > > > > confidential > > > > > > > > > > > >>> and > > > > > > > > > > > >>>>> for > > > > > > > > > > > >>>>>>> the use of the addressee only, > > > unless > > > > > > > otherwise > > > > > > > > > > indicated. > > > > > > > > > > > If you > > > > > > > > > > > >>> are > > > > > > > > > > > >>>>> not > > > > > > > > > > > >>>>>>> the intended recipient, please > do > > > not > > > > > > > read, copy, > > > > > > > > > > use or > > > > > > > > > > > disclose > > > > > > > > > > > >>> to > > > > > > > > > > > >>>>>> others > > > > > > > > > > > >>>>>>> this message or any attachment. > > > > Please > > > > > > also > > > > > > > > notify > > > > > > > > > > the > > > > > > > > > > > sender by > > > > > > > > > > > >>>>> replying > > > > > > > > > > > >>>>>>> to this email or by telephone > > > > (+44(020 > > > > > > > 7896 0011) > > > > > > > > > > and then > > > > > > > > > > > delete > > > > > > > > > > > >>> the > > > > > > > > > > > >>>>>> email > > > > > > > > > > > >>>>>>> and any copies of it. Opinions, > > > > > > conclusion > > > > > > > (etc) > > > > > > > > > > that do > > > > > > > > > > > not > > > > > > > > > > > >> relate > > > > > > > > > > > >>>> to > > > > > > > > > > > >>>>>> the > > > > > > > > > > > >>>>>>> official business of this > company > > > > shall > > > > > > be > > > > > > > > > > understood as > > > > > > > > > > > neither > > > > > > > > > > > >>>> given > > > > > > > > > > > >>>>>> nor > > > > > > > > > > > >>>>>>> endorsed by it. IG is a trading > > > name > > > > of > > > > > > IG > > > > > > > > Markets > > > > > > > > > > Limited > > > > > > > > > > > (a > > > > > > > > > > > >>> company > > > > > > > > > > > >>>>>>> registered in England and > Wales, > > > > > company > > > > > > > number > > > > > > > > > > 04008957) > > > > > > > > > > > and IG > > > > > > > > > > > >>>> Index > > > > > > > > > > > >>>>>>> Limited (a company registered > in > > > > > England > > > > > > > and > > > > > > > > Wales, > > > > > > > > > > company > > > > > > > > > > > >> number > > > > > > > > > > > >>>>>>> 01190902). Registered address > at > > > > Cannon > > > > > > > Bridge > > > > > > > > > > House, 25 > > > > > > > > > > > Dowgate > > > > > > > > > > > >>>> Hill, > > > > > > > > > > > >>>>>>> London EC4R 2YA. Both IG > Markets > > > > > Limited > > > > > > > > (register > > > > > > > > > > number > > > > > > > > > > > 195355) > > > > > > > > > > > >>> and > > > > > > > > > > > >>>>> IG > > > > > > > > > > > >>>>>>> Index Limited (register number > > > > 114059) > > > > > > are > > > > > > > > > > authorised and > > > > > > > > > > > >> regulated > > > > > > > > > > > >>>> by > > > > > > > > > > > >>>>>> the > > > > > > > > > > > >>>>>>> Financial Conduct Authority. > > > > > > > > > > > >>>>>>> > > > > > > > > > > > >>>>>> > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>>> -- > > > > > > > > > > > >>>>> Nacho (Ignacio) Solis > > > > > > > > > > > >>>>> Kafka > > > > > > > > > > > >>>>> nso...@linkedin.com > > > > > > > > > > > >>>>> > > > > > > > > > > > >>>> > > > > > > > > > > > >>> > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > -Regards, > > > > > > > > > > > > Mayuresh R. Gharat > > > > > > > > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The information contained in this email is > > > > strictly > > > > > > > > > confidential > > > > > > > > > > and > > > > > > > > > > > for the use of the addressee only, unless > > otherwise > > > > > > > indicated. If > > > > > > > > > > you are > > > > > > > > > > > not the intended recipient, please do not read, > > > copy, > > > > > use > > > > > > > or > > > > > > > > > > disclose to > > > > > > > > > > > others this message or any attachment. Please > > also > > > > > notify > > > > > > > the > > > > > > > > > sender > > > > > > > > > > by > > > > > > > > > > > replying to this email or by telephone (+44(020 > > > 7896 > > > > > > 0011) > > > > > > > and > > > > > > > > then > > > > > > > > > > delete > > > > > > > > > > > the email and any copies of it. Opinions, > > > conclusion > > > > > > (etc) > > > > > > > that > > > > > > > > do > > > > > > > > > > not > > > > > > > > > > > relate to the official business of this company > > > shall > > > > > be > > > > > > > > understood > > > > > > > > > > as > > > > > > > > > > > neither given nor endorsed by it. IG is a > trading > > > > name > > > > > of > > > > > > > IG > > > > > > > > > Markets > > > > > > > > > > > Limited (a company registered in England and > > Wales, > > > > > > company > > > > > > > > number > > > > > > > > > > > 04008957) and IG Index Limited (a company > > > registered > > > > in > > > > > > > England > > > > > > > > and > > > > > > > > > > Wales, > > > > > > > > > > > company number 01190902). Registered address at > > > > Cannon > > > > > > > Bridge > > > > > > > > > House, > > > > > > > > > > 25 > > > > > > > > > > > Dowgate Hill, London EC4R 2YA. Both IG Markets > > > > Limited > > > > > > > (register > > > > > > > > > > number > > > > > > > > > > > 195355) and IG Index Limited (register number > > > 114059) > > > > > are > > > > > > > > > authorised > > > > > > > > > > and > > > > > > > > > > > regulated by the Financial Conduct Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The information contained in this email is strictly > > > > > > confidential > > > > > > > and > > > > > > > > for > > > > > > > > > > the use of the addressee only, unless otherwise > > > indicated. > > > > If > > > > > > > you are > > > > > > > > not > > > > > > > > > > the intended recipient, please do not read, copy, use > > or > > > > > > > disclose to > > > > > > > > > others > > > > > > > > > > this message or any attachment. Please also notify > the > > > > sender > > > > > > by > > > > > > > > replying > > > > > > > > > > to this email or by telephone (+44(020 7896 0011) and > > > then > > > > > > > delete the > > > > > > > > > email > > > > > > > > > > and any copies of it. Opinions, conclusion (etc) that > > do > > > > not > > > > > > > relate to > > > > > > > > > the > > > > > > > > > > official business of this company shall be understood > > as > > > > > > neither > > > > > > > given > > > > > > > > > nor > > > > > > > > > > endorsed by it. IG is a trading name of IG Markets > > > Limited > > > > (a > > > > > > > company > > > > > > > > > > registered in England and Wales, company number > > 04008957) > > > > and > > > > > > IG > > > > > > > Index > > > > > > > > > > Limited (a company registered in England and Wales, > > > company > > > > > > > number > > > > > > > > > > 01190902). Registered address at Cannon Bridge House, > > 25 > > > > > > Dowgate > > > > > > > Hill, > > > > > > > > > > London EC4R 2YA. Both IG Markets Limited (register > > number > > > > > > > 195355) and > > > > > > > > IG > > > > > > > > > > Index Limited (register number 114059) are authorised > > and > > > > > > > regulated by > > > > > > > > > the > > > > > > > > > > Financial Conduct Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > The information contained in this email is strictly > > > > confidential > > > > > > and > > > > > > > for > > > > > > > > the use of the addressee only, unless otherwise > indicated. > > If > > > > you > > > > > > > are not > > > > > > > > the intended recipient, please do not read, copy, use or > > > > disclose > > > > > > to > > > > > > > others > > > > > > > > this message or any attachment. Please also notify the > > sender > > > > by > > > > > > > replying > > > > > > > > to this email or by telephone (+44(020 7896 0011) and > then > > > > delete > > > > > > > the email > > > > > > > > and any copies of it. Opinions, conclusion (etc) that do > > not > > > > > relate > > > > > > > to the > > > > > > > > official business of this company shall be understood as > > > > neither > > > > > > > given nor > > > > > > > > endorsed by it. IG is a trading name of IG Markets > Limited > > (a > > > > > > company > > > > > > > > registered in England and Wales, company number 04008957) > > and > > > > IG > > > > > > > Index > > > > > > > > Limited (a company registered in England and Wales, > company > > > > > number > > > > > > > > 01190902). Registered address at Cannon Bridge House, 25 > > > > Dowgate > > > > > > > Hill, > > > > > > > > London EC4R 2YA. Both IG Markets Limited (register number > > > > 195355) > > > > > > > and IG > > > > > > > > Index Limited (register number 114059) are authorised and > > > > > regulated > > > > > > > by the > > > > > > > > Financial Conduct Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > -Regards, > > > > > > > Mayuresh R. Gharat > > > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > The information contained in this email is strictly > confidential > > > and > > > > > for > > > > > > > the use of the addressee only, unless otherwise indicated. If > you > > > are > > > > > not > > > > > > > the intended recipient, please do not read, copy, use or > disclose > > > to > > > > > > others > > > > > > > this message or any attachment. Please also notify the sender > by > > > > > replying > > > > > > > to this email or by telephone (+44(020 7896 0011) and then > delete > > > the > > > > > > email > > > > > > > and any copies of it. Opinions, conclusion (etc) that do not > > relate > > > > to > > > > > > the > > > > > > > official business of this company shall be understood as > neither > > > > given > > > > > > nor > > > > > > > endorsed by it. IG is a trading name of IG Markets Limited (a > > > company > > > > > > > registered in England and Wales, company number 04008957) and > IG > > > > Index > > > > > > > Limited (a company registered in England and Wales, company > > number > > > > > > > 01190902). Registered address at Cannon Bridge House, 25 > Dowgate > > > > Hill, > > > > > > > London EC4R 2YA. Both IG Markets Limited (register number > 195355) > > > and > > > > > IG > > > > > > > Index Limited (register number 114059) are authorised and > > regulated > > > > by > > > > > > the > > > > > > > Financial Conduct Authority. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > -Regards, > > > > > > Mayuresh R. Gharat > > > > > > (862) 250-7125 > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > -Regards, > > > > Mayuresh R. Gharat > > > > (862) 250-7125 > > > > > > > > > > > > > > > -- > > Nacho - Ignacio Solis - iso...@igso.net > > > -- -Regards, Mayuresh R. Gharat (862) 250-7125
