headers dont "leak" into application code. they are useful to application code as well. IIUC samze currently has headers "in-V" and would just switch over to kafka headers if they exist. im sure plenty of other users of kafka would have a use for headers. im pretty sure use cases exist around shuffling data into/out-of kafka (kafka connect or equivalent) where metadata from one end could copied over to the other (S3, for example uses http headers for user-accessible metadata). it will be kafka client code getting/setting those headers. not an interceptor.
On Fri, Feb 17, 2017 at 1:41 PM, Michael Pearce <michael.pea...@ig.com> wrote: > For APM single event tracing, need access to the header at the point of > processing on the processing thread. > > As such interceptors will not work/be suitable for these, due to the fact > they act on the ConsumerRecords as a batch, before the handling thread can > split out and process per message which is the point these tools will need > to continue to transaction tracing. > > Like wise tools and infra pieces will need access to the message outside > the interceptor realm. > > > > On 17/02/2017, 21:26, "Jason Gustafson" <ja...@confluent.io> wrote: > > > > > That’s exactly what we’re doing the headers are a slice of bytes, > which > > then gets parsed later if needed, or can be parsed right away, the > headers > > is part of the protocol, so can still be validated if wanted. > > If you had a header count then you would have to go through each > header > > key and value length value to work out how much to skip to get to > say the > > value or any future component in the message after the headers. > Having it > > as a byte[] with length value makes this a lot easier to skip. > > > So the broker will parse the headers and validate them. Good. The only > reason remaining that I can see to leave the headers as a byte array > is to > make it easier for the client to skip past them. Are we sure this is > not > premature optimization? Are there any performance results which show > that > this is worthwhile? > > What’s the issue with exposing a method getHeaders on the > producer/consumer > > record? It doesn’t break anything. We don’t need any special version. > > > See my previous explanation. What I am trying to resist is the headers > becoming a general application-level facility. The primary use case as > far > as I can tell is middleware, which is the use case that the > interceptors > are providing. > > Current batch consumer model and consumer interceptors don’t work where > > headers need to be acted on at per message level at time of > processing, > > very case is APM (the core one), where the header value is used to > continue > > tracing. > > > I still don't understand the point about batching. The consumer > records are > exposed as a batch in the consumer interceptor, but you can still > iterate > them individually. It is no different for the consumer API itself. > > -Jason > > On Fri, Feb 17, 2017 at 12:48 PM, Michael Pearce < > michael.pea...@ig.com> > wrote: > > > Re: > > > > “ The point about creation of maps seems orthogonal. We can still > > represent > > the headers as a slice of bytes until the time it is accessed.” > > > > That’s exactly what we’re doing the headers are a slice of bytes, > which > > then gets parsed later if needed, or can be parsed right away, the > headers > > is part of the protocol, so can still be validated if wanted. > > > > If you had a header count then you would have to go through each > header > > key and value length value to work out how much to skip to get to > say the > > value or any future component in the message after the headers. > Having it > > as a byte[] with length value makes this a lot easier to skip. > > > > > > On 17/02/2017, 20:37, "Michael Pearce" <michael.pea...@ig.com> > wrote: > > > > What’s the issue with exposing a method getHeaders on the > > producer/consumer record? It doesn’t break anything. We don’t need > any > > special version. > > > > Current batch consumer model and consumer interceptors don’t work > > where headers need to be acted on at per message level at time of > > processing, very case is APM (the core one), where the header value > is used > > to continue tracing. JMS/HTTP etc all expose these, without issues. > I would > > NOT want to lock this down to only be usable accessible via > interceptors, > > as we’d fail on one of the main goals. > > > > Regards > > Mike > > > > > > > > > > On 17/02/2017, 20:21, "Jason Gustafson" <ja...@confluent.io> > wrote: > > > > The point about creation of maps seems orthogonal. We can > still > > represent > > the headers as a slice of bytes until the time it is > accessed. > > > > > > > Yes exactly we have access to the records thus why the > header > > should be > > > accessible via it and not hidden for only interceptors to > access. > > > > > > As explained above, the point is to make the intended usage > clear. > > Applications should continue to rely on the key/value fields > to > > serialize > > their own headers, and it would be more ideal if we can avoid > > leaking > > third-party headers into applications. This is difficult to > do > > with the > > current interceptors because they share the record objects > with > > the common > > API. What I had in mind is something like an extension of the > > current > > interceptors which exposed a different object (e.g. > > `RecordAndHeaders`). > > The challenge is for MM-like use cases. Let me see if I can > come > > up with a > > concrete proposal for that problem. > > > > -Jason > > > > > > > > On Fri, Feb 17, 2017 at 11:55 AM, Michael Pearce < > > michael.pea...@ig.com> > > wrote: > > > > > I am happy to move the definition of the header into the > message > > body, but > > > would cause us not to lazy initialise/parse the headers, as > > obviously, we > > > would have to traverse these reading the message. > > > > > > This was actually one of Jay’s requests: > > > > > > “ 2. I think we should think about creating the Map > lazily to > > avoid > > > parsing out all the headers into little objects. > HashMaps > > themselves > > > are > > > kind of expensive and the consumer is very perf > sensitive so > > and making > > > gazillions of hashmaps that may or may not get used is > > probably a bad > > > idea.” > > > > > > > > > > > > > > > > > > On 17/02/2017, 19:44, "Michael Pearce" < > michael.pea...@ig.com> > > wrote: > > > > > > Yes exactly we have access to the records thus why the > > header should > > > be accessible via it and not hidden for only interceptors > to > > access. > > > > > > Sent using OWA for iPhone > > > ________________________________________ > > > From: Magnus Edenhill <mag...@edenhill.se> > > > Sent: Friday, February 17, 2017 7:34:49 PM > > > To: dev@kafka.apache.org > > > Subject: Re: [DISCUSS] KIP-82 - Add Record Headers > > > > > > Big +1 on VarInts. > > > CPUs are fast, memory is slow. > > > > > > I agree with Jason that we'll want to continue > verifying > > messages, > > > including their headers, so while I appreciate the > idea of > > the opaque > > > header blob it won't be useful in practice. > > > > > > /Magnus > > > > > > 2017-02-17 10:41 GMT-08:00 Jason Gustafson < > > ja...@confluent.io>: > > > > > > > Sorry, my mistake. The consumer interceptor is per > batch, > > though I'm > > > not > > > > sure that's an actual limitation since you still have > > access to the > > > > individual records. > > > > > > > > -Jason > > > > > > > > On Fri, Feb 17, 2017 at 10:39 AM, Jason Gustafson < > > > ja...@confluent.io> > > > > wrote: > > > > > > > > > Re headers as byte array and future use by broker. > This > > doesn't > > > take away > > > > >> from that at all. Nor makes it difficult at all > in my > > opinion. > > > > > > > > > > > > > > > Yeah, I didn't say it was difficult, only awkward. > You > > wouldn't > > > write the > > > > > schema that way if you were planning to use it on > the > > brokers from > > > the > > > > > beginning. Note also that one of the benefits of > letting > > the broker > > > > > understand headers is that it can validate that > they are > > properly > > > > > formatted. If cost is the only concern, we should > > confirm its > > > impact > > > > > through performance testing. > > > > > > > > > > One of the key use cases requires access on > consume at > > per > > > event/message > > > > >> level at the point that message is being > processed, as > > such the > > > batch > > > > >> interceptors and batch consume api isn't > suitable. It > > needs to be > > > at the > > > > >> record level. > > > > > > > > > > > > > > > I'm not sure I understand the point about batching. > > Interceptors > > > are > > > > > applied per-message, right? > > > > > > > > > > My intent on interceptors is to keep the usage of > headers > > > well-defined so > > > > > that they don't start leaking unnecessarily into > > applications. My > > > guess > > > > is > > > > > that it's probably inevitable, but isolating it in > the > > > interceptors would > > > > > at least give people a second thought before > deciding to > > use it. > > > The main > > > > > challenge in my mind is figuring out how an MM use > case > > would > > > work. It > > > > > would be more cumbersome to replicate headers > through an > > > interceptor, > > > > > though arguably MM should be working at a lower > level > > anyway. > > > > > > > > > > -Jason > > > > > > > > > > On Fri, Feb 17, 2017 at 10:16 AM, Michael Pearce < > > > michael.pea...@ig.com> > > > > > wrote: > > > > > > > > > >> Re headers available on the record va > interceptors only > > > > >> > > > > >> One of the key use cases requires access on > consume at > > per > > > event/message > > > > >> level at the point that message is being > processed, as > > such the > > > batch > > > > >> interceptors and batch consume api isn't > suitable. It > > needs to be > > > at the > > > > >> record level. > > > > >> > > > > >> This anyhow is similar to jms/http/amqp where > headers > > are > > > available to > > > > >> consuming applications. > > > > >> > > > > >> Re headers as byte array and future use by > broker. This > > doesn't > > > take > > > > away > > > > >> from that at all. Nor makes it difficult at all > in my > > opinion. > > > > >> > > > > >> > > > > >> > > > > >> Sent using OWA for iPhone > > > > >> ________________________________________ > > > > >> From: Jason Gustafson <ja...@confluent.io> > > > > >> Sent: Friday, February 17, 2017 5:55:42 PM > > > > >> To: dev@kafka.apache.org > > > > >> Subject: Re: [DISCUSS] KIP-82 - Add Record Headers > > > > >> > > > > >> > > > > > >> > Would you be proposing in KIP-98 to convert the > other > > message > > > int’s > > > > (key > > > > >> > length, value length) also to varint to keep it > > uniform. > > > > >> > Also I assume there will be a static or helper > method > > made to > > > > write/read > > > > >> > these in the client and server. > > > > >> > > > > >> > > > > >> Yes, that is what we are proposing, so using > varints > > for headers > > > would > > > > be > > > > >> consistent with the rest of the message. We have > used > > static > > > helper > > > > >> methods > > > > >> in our prototype implementation. > > > > >> > > > > >> The cost of parsing, we want to parse/interpret > the > > headers > > > lazily (this > > > > >> is > > > > >> > a key point brought up earlier in discussions) > > > > >> > > > > >> > > > > >> I'm a bit skeptical of this. Has anyone done the > > performance > > > testing? I > > > > >> can > > > > >> probably implement it and test it if no one else > has. I > > was also > > > under > > > > the > > > > >> impression that there may be use cases down the > road > > where the > > > broker > > > > >> would > > > > >> need to interpret headers. That wouldn't be off > the > > table in the > > > future > > > > if > > > > >> it's represented as bytes, but it would be quite > a bit > > more > > > awkward, > > > > >> right? > > > > >> > > > > >> By the way, one question I have been wondering > about. My > > > understanding > > > > is > > > > >> that headers are primarily for use cases where a > > third-party > > > components > > > > >> wants to enrich messages without needing to > understand > > or modify > > > the > > > > >> schema > > > > >> of the message key and value. For the applications > > which directly > > > > produce > > > > >> and consume the messages and control the key/value > > schema > > > directly, it > > > > >> seems we would rather have them implement headers > > directly in > > > their own > > > > >> schema. Supposing for the sake of argument that > it was > > possible, > > > my > > > > >> question is whether it be sufficient to expose the > > headers in the > > > > >> interceptor API and not in the common API? > > > > >> > > > > >> -Jason > > > > >> > > > > >> On Fri, Feb 17, 2017 at 3:26 AM, Michael Pearce < > > > michael.pea...@ig.com> > > > > >> wrote: > > > > >> > > > > >> > On the point of varInts > > > > >> > > > > > >> > Would you be proposing in KIP-98 to convert the > other > > message > > > int’s > > > > (key > > > > >> > length, value length) also to varint to keep it > > uniform. > > > > >> > Also I assume there will be a static or helper > method > > made to > > > > write/read > > > > >> > these in the client and server. > > > > >> > > > > > >> > Cheers > > > > >> > Mike > > > > >> > > > > > >> > > > > > >> > > > > > >> > On 17/02/2017, 11:22, "Michael Pearce" < > > michael.pea...@ig.com> > > > wrote: > > > > >> > > > > > >> > On the point re: headers in the message > protocol > > being a > > > byte > > > > array > > > > >> > and not a count of elements followed by the > elements. > > Again > > > this was > > > > >> > discussed/argued previously. > > > > >> > > > > > >> > It was agreed on for a few reasons some of > which > > you have > > > > obviously > > > > >> > picked up on: > > > > >> > > > > > >> > Broker is able to pass it through opaquely > > > > >> > The cost of parsing, we want to > parse/interpret > > the headers > > > lazily > > > > >> > (this is a key point brought up earlier in > > discussions) > > > > >> > Headers can be copied from consumer record > to > > producer > > > record (aka > > > > >> > mirror makers etc) without parsing if no > changes are > > being made > > > or > > > > being > > > > >> > looked at. > > > > >> > Keeps the broker agnostic to the format > > > > >> > You need an int32 either for the byte size > of the > > headers, > > > or for > > > > >> the > > > > >> > count of elements, so overheads are the same, > but > > with going > > > with an > > > > >> opaque > > > > >> > byte array has the above advantages. > > > > >> > > > > > >> > Cheers > > > > >> > Mike > > > > >> > > > > > >> > > > > > >> > On 17/02/2017, 02:50, "Jason Gustafson" < > > ja...@confluent.io > > > > > > > > wrote: > > > > >> > > > > > >> > Sorry, should have noted that the > performance > > testing > > > was done > > > > >> > using the > > > > >> > producer performance tool shipped with > Kafka. > > > > >> > > > > > >> > -Jason > > > > >> > > > > > >> > On Thu, Feb 16, 2017 at 6:44 PM, Jason > > Gustafson < > > > > >> > ja...@confluent.io> wrote: > > > > >> > > > > > >> > > Hey Nacho, > > > > >> > > > > > > >> > > I've compared performance of our > KIP-98 > > > implementation with > > > > >> and > > > > >> > without > > > > >> > > varints. For messages around 128 > bytes, we > > see an > > > increase > > > > in > > > > >> > throughput of > > > > >> > > about 30% using the default > configuration > > settings. > > > At 256 > > > > >> > bytes, the > > > > >> > > increase is around 16%. Obviously the > > performance > > > converge > > > > as > > > > >> > messages get > > > > >> > > larger, but it seems well worth the > cost. > > Note that > > > we are > > > > >> also > > > > >> > seeing a > > > > >> > > substantial performance increase > against > > trunk > > > primarily > > > > >> because > > > > >> > of the > > > > >> > > much more efficient packing that > varints > > provide us. > > > > Anything > > > > >> > adding to > > > > >> > > message overhead, such as record > headers, > > would only > > > > increase > > > > >> > the relative > > > > >> > > difference. (Of course take these > numbers > > with a > > > grain of > > > > salt > > > > >> > since I have > > > > >> > > only used the default settings with > both > > the producer > > > and > > > > >> broker > > > > >> > on my > > > > >> > > local machine. We intend to provide > more > > extensive > > > > performance > > > > >> > details as > > > > >> > > part of the work for KIP-98.) > > > > >> > > > > > > >> > > The implementation we are using is > from > > protobuf ( > > > > >> > > https://developers.google.com/ > > > > protocol-buffers/docs/encoding > > > > >> ), > > > > >> > which is > > > > >> > > also used in HBase. It is trivial to > > implement and as > > > far > > > > as I > > > > >> > know doesn't > > > > >> > > suffer from the aliasing problem you > are > > describing. I > > > > checked > > > > >> > with Magnus > > > > >> > > (the author of librdkafka) and he > agreed > > that the > > > savings > > > > >> seemed > > > > >> > worth the > > > > >> > > cost of implementation. > > > > >> > > > > > > >> > > -Jason > > > > >> > > > > > > >> > > On Thu, Feb 16, 2017 at 4:32 PM, > Ignacio > > Solis < > > > > >> iso...@igso.net> > > > > >> > wrote: > > > > >> > > > > > > >> > >> -VarInts > > > > >> > >> > > > > >> > >> I'm one of the people (if not the > most) > > opposed to > > > VarInts. > > > > >> > VarInts > > > > >> > >> have a place, but this is not it. > (We > > had a large > > > > >> discussion > > > > >> > about > > > > >> > >> them at the beginning of KIP-82 time) > > > > >> > >> > > > > >> > >> If anybody has real life performance > > numbers of > > > VarInts > > > > >> > improving > > > > >> > >> things or significantly reducing > resources > > I would > > > like to > > > > >> know > > > > >> > what > > > > >> > >> that case may be. Yes, you can save > some > > bytes here > > > and > > > > >> there, > > > > >> > but > > > > >> > >> this is probably insignificant to the > > overall system > > > > behavior > > > > >> > and > > > > >> > >> storage requirements. -- I say this > with > > respect to > > > using > > > > >> > VarInts in > > > > >> > >> the protocol itself, not as part of > the > > data. > > > > >> > >> > > > > >> > >> VarInts require you to parse the Int > > before using it > > > and > > > > >> > depending on > > > > >> > >> the encoding they can suffer from > aliasing > > (multiple > > > > >> > representations > > > > >> > >> for the same value). > > > > >> > >> > > > > >> > >> Why add complexity? > > > > >> > >> > > > > >> > >> Nacho > > > > >> > >> > > > > >> > >> > > > > >> > >> On Thu, Feb 16, 2017 at 10:29 AM, > Colin > > McCabe < > > > > >> > cmcc...@apache.org> > > > > >> > >> wrote: > > > > >> > >> > +1 for varints here-- it would save > > quite a bit of > > > space. > > > > >> > They are > > > > >> > >> > pretty quick to implement as well. > > > > >> > >> > > > > > >> > >> > I think it makes sense for values > to be > > byte > > > arrays. > > > > Users > > > > >> > might want > > > > >> > >> > to attach arbitrary payloads; they > > shouldn't be > > > forced to > > > > >> > serialize > > > > >> > >> > everything to Java strings. > > > > >> > >> > > > > > >> > >> > best, > > > > >> > >> > Colin > > > > >> > >> > > > > > >> > >> > > > > > >> > >> > On Thu, Feb 16, 2017, at 09:52, > Jason > > Gustafson > > > wrote: > > > > >> > >> >> Hey Michael, > > > > >> > >> >> > > > > >> > >> >> Hmm, I guess the point of > representing > > it as > > > bytes is to > > > > >> > allow the > > > > >> > >> broker > > > > >> > >> >> to pass it through opaquely? Is > the > > cost of > > > parsing > > > > them a > > > > >> > concern, or > > > > >> > >> >> are > > > > >> > >> >> we simply trying to ensure that > the > > broker stays > > > > agnostic > > > > >> to > > > > >> > the > > > > >> > >> format? > > > > >> > >> >> > > > > >> > >> >> On varints, I think adding > support for > > them makes > > > less > > > > >> sense > > > > >> > for an > > > > >> > >> >> isolated use case, but as part of > a > > more holistic > > > change > > > > >> > (such as what > > > > >> > >> we > > > > >> > >> >> have proposed in KIP-98), I think > they > > are > > > justifiable. > > > > If > > > > >> > we add them, > > > > >> > >> >> then the need to use attributes > becomes > > quite a > > > bit > > > > >> weaker, > > > > >> > right? The > > > > >> > >> >> other thing I find slightly odd > is the > > fact that > > > null > > > > >> > headers has no > > > > >> > >> >> actual > > > > >> > >> >> semantic meaning for the message > > (unlike null > > > keys and > > > > >> > values). It is > > > > >> > >> >> just > > > > >> > >> >> a space optimization. It seems a > bit > > better to > > > always > > > > use > > > > >> > size 0 to > > > > >> > >> >> indicate having no headers. > > > > >> > >> >> > > > > >> > >> >> Overall, the main point is > ensuring > > that the > > > message > > > > >> schema > > > > >> > remains > > > > >> > >> >> consistent, either within the > larger > > protocol, or > > > at a > > > > >> > minimum within > > > > >> > >> the > > > > >> > >> >> message itself. > > > > >> > >> >> > > > > >> > >> >> -Jason > > > > >> > >> >> > > > > >> > >> >> On Thu, Feb 16, 2017 at 6:39 AM, > > Michael Pearce < > > > > >> > michael.pea...@ig.com > > > > >> > >> > > > > > >> > >> >> wrote: > > > > >> > >> >> > > > > >> > >> >> > Hi Jason, > > > > >> > >> >> > > > > > >> > >> >> > On point 1) in the message > protocol > > the headers > > > are > > > > >> simply > > > > >> > a byte > > > > >> > >> array, > > > > >> > >> >> > as like the key or value, this > is to > > clearly > > > demarcate > > > > >> the > > > > >> > header in > > > > >> > >> the > > > > >> > >> >> > core message. Then the header > byte > > array in the > > > core > > > > >> > message is an > > > > >> > >> array of > > > > >> > >> >> > key, value pairs. This is what > it is > > denoting. > > > > >> > >> >> > > > > > >> > >> >> > Then this would be I guess in > the > > given > > > notation: > > > > >> > >> >> > > > > > >> > >> >> > Headers => [KeyLength, Key, > > ValueLength, Value] > > > > >> > >> >> > KeyLength => int32 > > <-----------------NEW > > > size of > > > > the > > > > >> > byte[] of > > > > >> > >> the > > > > >> > >> >> > serialised key value > > > > >> > >> >> > Key => bytes > > <---------------------- NEW > > > > serialised > > > > >> > string (UTF8) > > > > >> > >> >> > bytes of the header key > > > > >> > >> >> > ValueLength => int32 > > <-------------- NEW > > > size of > > > > the > > > > >> > byte[] of > > > > >> > >> the > > > > >> > >> >> > serialised header value > > > > >> > >> >> > Value => bytes > > <-------------------- NEW > > > > serialised > > > > >> > form of the > > > > >> > >> header > > > > >> > >> >> > value > > > > >> > >> >> > > > > > >> > >> >> > The key length and value length > is > > matching the > > > way > > > > the > > > > >> > protocol is > > > > >> > >> >> > defined in the core message > currently. > > > > >> > >> >> > > > > > >> > >> >> > > > > > >> > >> >> > > > > > >> > >> >> > > > > > >> > >> >> > On point 2) > > > > >> > >> >> > Var sized ints, this was > discussed > > much earlier > > > on, in > > > > >> > fact I had > > > > >> > >> >> > suggested it myself (with Hadoop > > references), > > > the > > > > >> > complexity of this > > > > >> > >> >> > compared to having a simpler > protocol > > was > > > argued and > > > > >> > agreed it > > > > >> > >> wasn’t worth > > > > >> > >> >> > the complexity as all other > clients > > in other > > > languages > > > > >> > would need to > > > > >> > >> ensure > > > > >> > >> >> > theyre using the right var size > > algorithm, as > > > there > > > > is a > > > > >> > few. > > > > >> > >> >> > > > > > >> > >> >> > On point 3) > > > > >> > >> >> > We did the attributes, optional > > approach as > > > originally > > > > >> > there was > > > > >> > >> marked > > > > >> > >> >> > concern that headers would > cause a > > message size > > > > overhead > > > > >> > for others, > > > > >> > >> who > > > > >> > >> >> > don’t want them. As such this > is the > > clean > > > solution to > > > > >> > achieve that. > > > > >> > >> If > > > > >> > >> >> > that no longer holds, and we > don’t > > care that we > > > add > > > > >> 4bytes > > > > >> > overhead, > > > > >> > >> then > > > > >> > >> >> > im happy to remove. > > > > >> > >> >> > > > > > >> > >> >> > I’m personally in favour of > keeping > > the message > > > as > > > > small > > > > >> > as possible > > > > >> > >> so > > > > >> > >> >> > people don’t get shocks in perf > and > > throughputs > > > dues > > > > to > > > > >> > message size, > > > > >> > >> >> > unless they actively use the > feature, > > as such I > > > do > > > > >> prefer > > > > >> > the > > > > >> > >> attribute bit > > > > >> > >> >> > wise feature flag approach > myself. > > > > >> > >> >> > > > > > >> > >> >> > > > > > >> > >> >> > > > > > >> > >> >> > > > > > >> > >> >> > On 16/02/2017, 05:40, "Jason > > Gustafson" < > > > > >> > ja...@confluent.io> wrote: > > > > >> > >> >> > > > > > >> > >> >> > We have proposed a few > > significant changes > > > to the > > > > >> > message format > > > > >> > >> in > > > > >> > >> >> > KIP-98 > > > > >> > >> >> > which now seems likely to > pass > > (perhaps > > > with some > > > > >> > iterations on > > > > >> > >> >> > implementation details). It > would > > be good > > > to try > > > > and > > > > >> > coordinate > > > > >> > >> the > > > > >> > >> >> > changes > > > > >> > >> >> > in both of the proposals to > make > > sure they > > > are > > > > >> > consistent and > > > > >> > >> >> > compatible. > > > > >> > >> >> > > > > > >> > >> >> > I think using the > attributes to > > indicate > > > null > > > > >> headers > > > > >> > is a > > > > >> > >> reasonable > > > > >> > >> >> > approach. We have proposed > to do > > the same > > > thing > > > > for > > > > >> > the message > > > > >> > >> key and > > > > >> > >> >> > value. That said, I > sympathize > > with Jay's > > > > argument. > > > > >> > Having > > > > >> > >> multiple > > > > >> > >> >> > ways to > > > > >> > >> >> > specify a null value > increases > > the overall > > > > >> complexity > > > > >> > of the > > > > >> > >> protocol. > > > > >> > >> >> > You > > > > >> > >> >> > can see this just from the > fact > > that you > > > need the > > > > >> > extra verbiage > > > > >> > >> in the > > > > >> > >> >> > protocol specification in > this > > KIP and in > > > KIP-98 > > > > to > > > > >> > describe the > > > > >> > >> >> > dependence > > > > >> > >> >> > between the fields and the > > attributes. It > > > seems > > > > >> like a > > > > >> > slippery > > > > >> > >> slope > > > > >> > >> >> > if > > > > >> > >> >> > you start allowing different > > request types > > > to > > > > >> > implement the > > > > >> > >> protocol > > > > >> > >> >> > specification differently. > > > > >> > >> >> > > > > > >> > >> >> > You can also argue that the > > messages > > > already are > > > > and > > > > >> > are likely > > > > >> > >> to > > > > >> > >> >> > remain a > > > > >> > >> >> > special case. For example, > there > > is > > > currently no > > > > >> > generality in > > > > >> > >> how > > > > >> > >> >> > compressed message sets are > > represented > > > that would > > > > >> be > > > > >> > applicable > > > > >> > >> for > > > > >> > >> >> > other > > > > >> > >> >> > request types. Some might > see this > > > divergence as > > > > an > > > > >> > unfortunate > > > > >> > >> >> > protocol > > > > >> > >> >> > deficiency which should be > fixed; > > others > > > might see > > > > >> it > > > > >> > as sort of > > > > >> > >> the > > > > >> > >> >> > inevitability of needing to > > optimize where > > > it > > > > counts > > > > >> > most. I'm > > > > >> > >> probably > > > > >> > >> >> > somewhere in between, but I > think > > we > > > probably all > > > > >> > share the > > > > >> > >> intuition > > > > >> > >> >> > that > > > > >> > >> >> > the protocol should be kept > as > > consistent as > > > > >> possible. > > > > >> > With that > > > > >> > >> in > > > > >> > >> >> > mind, > > > > >> > >> >> > here are a few comments: > > > > >> > >> >> > > > > > >> > >> >> > 1. One thing I found a > little odd > > when > > > reading the > > > > >> > current > > > > >> > >> proposal is > > > > >> > >> >> > that > > > > >> > >> >> > the headers are both > represented > > as an > > > array of > > > > >> bytes > > > > >> > and as an > > > > >> > >> array > > > > >> > >> >> > of > > > > >> > >> >> > key/value pairs. I'd > probably > > suggest > > > something > > > > like > > > > >> > this: > > > > >> > >> >> > > > > > >> > >> >> > Headers => [HeaderKey > HeaderValue] > > > > >> > >> >> > HeaderKey => String > > > > >> > >> >> > HeaderValue => Bytes > > > > >> > >> >> > > > > > >> > >> >> > An array in the Kafka > protocol is > > > represented as a > > > > >> > 4-byte integer > > > > >> > >> >> > indicating the number of > elements > > in the > > > array > > > > >> > followed by the > > > > >> > >> >> > serialization of the > elements. > > Unless I'm > > > > >> > misunderstanding, what > > > > >> > >> you > > > > >> > >> >> > have > > > > >> > >> >> > instead is the total size > of the > > headers in > > > bytes > > > > >> > followed by the > > > > >> > >> >> > elements. > > > > >> > >> >> > I'm not sure I see any > reason for > > this > > > > >> inconsistency. > > > > >> > >> >> > > > > > >> > >> >> > 2. In KIP-98, we've > introduced > > > variable-length > > > > >> integer > > > > >> > fields. > > > > >> > >> >> > Effectively, > > > > >> > >> >> > we've enriched (or > "complicated" > > as Jay > > > might say > > > > ;) > > > > >> > the protocol > > > > >> > >> >> > specification to include the > > following > > > types: > > > > >> VarInt, > > > > >> > VarLong, > > > > >> > >> >> > UnsignedVarInt and > > UnsignedVarLong. > > > > >> > >> >> > > > > > >> > >> >> > Along with these > primitives, we > > could > > > introduce > > > > the > > > > >> > following > > > > >> > >> types: > > > > >> > >> >> > > > > > >> > >> >> > VarSizeArray => > NumberOfItems > > Item1 Item2 > > > .. ItemN > > > > >> > >> >> > NumberOfItems => > UnsignedVarInt > > > > >> > >> >> > > > > > >> > >> >> > VarSizeNullableArray => > > NumberOfItemsOrNull > > > Item1 > > > > >> > Item2 .. ItemN > > > > >> > >> >> > NumberOfItemsOrNull => > VarInt > > (-1 means > > > null) > > > > >> > >> >> > > > > > >> > >> >> > And similarly for the > `String` > > and `Bytes` > > > types. > > > > >> > These types > > > > >> > >> can save > > > > >> > >> >> > a > > > > >> > >> >> > considerable amount of > space in > > this > > > proposal > > > > >> because > > > > >> > they can > > > > >> > >> be used > > > > >> > >> >> > for > > > > >> > >> >> > both the number of headers > > included in the > > > message > > > > >> and > > > > >> > the > > > > >> > >> lengths of > > > > >> > >> >> > the > > > > >> > >> >> > header keys and values. We > could > > do this > > > instead: > > > > >> > >> >> > > > > > >> > >> >> > Headers => > VarSizeArray[HeaderKey > > > HeaderValue] > > > > >> > >> >> > HeaderKey => VarSizeString > > > > >> > >> >> > HeaderValue => > VarSizeBytes > > > > >> > >> >> > > > > > >> > >> >> > Combining the savings from > the > > use of > > > variable > > > > >> length > > > > >> > fields, the > > > > >> > >> >> > benefit > > > > >> > >> >> > of using the attributes to > > represent null > > > seems > > > > >> pretty > > > > >> > small. > > > > >> > >> >> > > > > > >> > >> >> > 3. Whichever way we go > (whether > > we use the > > > > >> attributes > > > > >> > or not), we > > > > >> > >> >> > should at > > > > >> > >> >> > least be consistent between > this > > KIP and > > > KIP-98. > > > > It > > > > >> > would be very > > > > >> > >> >> > strange > > > > >> > >> >> > to have two ways to > represent > > null values > > > in the > > > > >> same > > > > >> > schema. > > > > >> > >> Either > > > > >> > >> >> > way is > > > > >> > >> >> > OK with me. I think some > > message-level > > > > optimizations > > > > >> > are > > > > >> > >> justifiable, > > > > >> > >> >> > but > > > > >> > >> >> > the savings here seem > minimal (a > > few bytes > > > per > > > > >> > message), so > > > > >> > >> maybe it's > > > > >> > >> >> > not > > > > >> > >> >> > worth the cost of letting > the > > message > > > diverge even > > > > >> > further from > > > > >> > >> the > > > > >> > >> >> > rest of > > > > >> > >> >> > the protocol. > > > > >> > >> >> > > > > > >> > >> >> > -Jason > > > > >> > >> >> > > > > > >> > >> >> > > > > > >> > >> >> > On Wed, Feb 15, 2017 at > 8:52 AM, > > radai < > > > > >> > >> radai.rosenbl...@gmail.com> > > > > >> > >> >> > wrote: > > > > >> > >> >> > > > > > >> > >> >> > > I've trimmed the inline > > contents as this > > > mail is > > > > >> > getting too > > > > >> > >> big for > > > > >> > >> >> > the > > > > >> > >> >> > > apache mailing list > software to > > deliver > > > :-( > > > > >> > >> >> > > > > > > >> > >> >> > > 1. the important thing for > > > interoperability is > > > > for > > > > >> > different > > > > >> > >> >> > "interested > > > > >> > >> >> > > parties" (plugins, infra > > layers/wrappers, > > > > >> user-code) > > > > >> > to be > > > > >> > >> able to > > > > >> > >> >> > stick > > > > >> > >> >> > > pieces of metadata onto > msgs > > without > > > getting in > > > > >> each > > > > >> > other's > > > > >> > >> way. a > > > > >> > >> >> > common > > > > >> > >> >> > > key scheme (Strings, as > of the > > time of > > > this > > > > >> > writing?) is all > > > > >> > >> thats > > > > >> > >> >> > required > > > > >> > >> >> > > for that. it is assumed > that > > the other end > > > > >> > interested in any > > > > >> > >> such > > > > >> > >> >> > piece of > > > > >> > >> >> > > metadata knows the > encoding, > > and byte[] > > > provides > > > > >> for > > > > >> > the most > > > > >> > >> >> > flexibility. > > > > >> > >> >> > > i believe this is the same > > logic behind > > > core > > > > kafka > > > > >> > being > > > > >> > >> >> > byte[]/byte[] - > > > > >> > >> >> > > Strings are more "usable" > but > > bytes are > > > flexible > > > > >> and > > > > >> > so were > > > > >> > >> chosen. > > > > >> > >> >> > > Also - core kafka doesnt > even > > do that > > > good of a > > > > >> job > > > > >> > on > > > > >> > >> usability of > > > > >> > >> >> > the > > > > >> > >> >> > > payload (example - i have > to > > specify the > > > nop > > > > >> byte[] > > > > >> > "decoders" > > > > >> > >> >> > explicitly > > > > >> > >> >> > > in conf), and again > sacrificies > > usability > > > for > > > > the > > > > >> > sake of > > > > >> > >> >> > performance (no > > > > >> > >> >> > > convenient single-record > > processing as > > > poll is a > > > > >> > batch, lots of > > > > >> > >> >> > obscure > > > > >> > >> >> > > little config details > exposing > > internals > > > of the > > > > >> > batching > > > > >> > >> mechanism, > > > > >> > >> >> > etc) > > > > >> > >> >> > > > > > > >> > >> >> > > this is also why i really > > dislike the > > > idea of a > > > > >> > "type system" > > > > >> > >> for > > > > >> > >> >> > header > > > > >> > >> >> > > values, it further > degrades the > > > usability, adds > > > > >> > complexity and > > > > >> > >> will > > > > >> > >> >> > > eventually get in > people's way, > > also, it > > > would > > > > be > > > > >> > the 2nd/3rd > > > > >> > >> >> > home-group > > > > >> > >> >> > > serialization mechanism > in core > > kafka > > > (counting > > > > 2 > > > > >> > iterations > > > > >> > >> of the > > > > >> > >> >> > "type > > > > >> > >> >> > > definition DSL") > > > > >> > >> >> > > > > > > >> > >> >> > > 2. this is an > implementation > > detail, and > > > not > > > > even > > > > >> a > > > > >> > very "user > > > > >> > >> >> > facing" one? > > > > >> > >> >> > > to the best of my > understanding > > the vote > > > process > > > > >> is > > > > >> > on proposed > > > > >> > >> >> > > API/behaviour. also - > since > > we're willing > > > to go > > > > >> with > > > > >> > strings > > > > >> > >> just > > > > >> > >> >> > serialize > > > > >> > >> >> > > a 0-sized header blob and > IIUC > > you dont > > > need any > > > > >> > optionals > > > > >> > >> anymore. > > > > >> > >> >> > > > > > > >> > >> >> > > 3. yes, we can :-) > > > > >> > >> >> > > > > > > >> > >> >> > > On Tue, Feb 14, 2017 at > 11:56 > > PM, Michael > > > > Pearce < > > > > >> > >> >> > michael.pea...@ig.com> > > > > >> > >> >> > > wrote: > > > > >> > >> >> > > > > > > >> > >> >> > > > Hi Jay, > > > > >> > >> >> > > > > > > > >> > >> >> > > > 1) There was some > initial > > debate on the > > > value > > > > >> > part, as youll > > > > >> > >> note > > > > >> > >> >> > String, > > > > >> > >> >> > > > String headers were > > discounted early > > > on. The > > > > >> > reason for this > > > > >> > >> is > > > > >> > >> >> > > flexibility > > > > >> > >> >> > > > and keeping in line > with the > > > flexibility of > > > > key, > > > > >> > value of the > > > > >> > >> >> > message > > > > >> > >> >> > > > object itself. I don’t > think > > it takes > > > away > > > > from > > > > >> an > > > > >> > ecosystem > > > > >> > >> as > > > > >> > >> >> > each > > > > >> > >> >> > > plugin > > > > >> > >> >> > > > will care for their own > key, > > this way > > > ints, > > > > >> > booleans , exotic > > > > >> > >> >> > custom > > > > >> > >> >> > > binary > > > > >> > >> >> > > > can all be catered for=. > > > > >> > >> >> > > > a. If you really wanted > to > > push for a > > > typed > > > > >> value > > > > >> > interface, > > > > >> > >> I > > > > >> > >> >> > wouldn’t > > > > >> > >> >> > > > want just String values > > supported, but > > > the the > > > > >> > primatives > > > > >> > >> plus > > > > >> > >> >> > string and > > > > >> > >> >> > > > also still keeping the > > ability to have a > > > > binary > > > > >> > for custom > > > > >> > >> >> > binaries that > > > > >> > >> >> > > > some organisations may > have. > > > > >> > >> >> > > > i. I have written this > slight > > > alternative > > > > here, > > > > >> > >> >> > > https://cwiki.apache.org/ > > > > >> > >> >> > > > > confluence/display/KAFKA/KIP- > > > > >> > 82+-+Add+Record+Headers+-+Typed > > > > >> > >> >> > > > ii. Essentially the > value > > bytes, has a > > > leading > > > > >> > byte overhead. > > > > >> > >> >> > > > 1. This tells you what > type > > the value > > > is, > > > > >> before > > > > >> > reading > > > > >> > >> the rest > > > > >> > >> >> > of the > > > > >> > >> >> > > > bytes, allowing > > > serialisation/deserialization > > > > to > > > > >> > and from the > > > > >> > >> >> > primitives, > > > > >> > >> >> > > > string and byte[]. This > is > > akin to some > > > other > > > > >> > messaging > > > > >> > >> systems. > > > > >> > >> >> > > > 2) We are making it > optional, > > so that > > > for > > > > those > > > > >> > not wanting > > > > >> > >> >> > headers have > > > > >> > >> >> > > 0 > > > > >> > >> >> > > > bytes overhead (think > of it > > as a feature > > > > flag), > > > > >> I > > > > >> > don’t > > > > >> > >> think this > > > > >> > >> >> > is > > > > >> > >> >> > > > complex, especially if > > comparing to > > > changes > > > > >> > proposed in > > > > >> > >> other kips > > > > >> > >> >> > like > > > > >> > >> >> > > > kip-98. > > > > >> > >> >> > > > a. If you really really > don’t > > like > > > this, we > > > > can > > > > >> > drop it, but > > > > >> > >> it > > > > >> > >> >> > would > > > > >> > >> >> > > mean > > > > >> > >> >> > > > buying into 4 bytes > extra > > overhead for > > > users > > > > who > > > > >> > do not want > > > > >> > >> to use > > > > >> > >> >> > > headers. > > > > >> > >> >> > > > 3) In the summary yes, > it is > > at a higher > > > > level, > > > > >> > but I think > > > > >> > >> this > > > > >> > >> >> > is well > > > > >> > >> >> > > > documented in the > proposed > > changes > > > section. > > > > >> > >> >> > > > a. Added getHeaders > method to > > > > Producer/Consumer > > > > >> > record (that > > > > >> > >> is it) > > > > >> > >> >> > > > b. We’ve also detailed > the > > new Headers > > > class > > > > >> that > > > > >> > this method > > > > >> > >> >> > returns > > > > >> > >> >> > > that > > > > >> > >> >> > > > encapsulates the headers > > protocol and > > > logic. > > > > >> > >> >> > > > > > > > >> > >> >> > > > Best, > > > > >> > >> >> > > > Mike > > > > >> > >> >> > > > > > > > >> > >> >> > > > ==Original questions > from the > > vote > > > thread from > > > > >> > Jay.== > > > > >> > >> >> > > > > > > > >> > >> >> > > > Couple of things I > think we > > still need > > > to work > > > > >> out: > > > > >> > >> >> > > > > > > > >> > >> >> > > > 1. I think we agree > about > > the key, > > > but I > > > > >> think > > > > >> > we haven't > > > > >> > >> >> > talked about > > > > >> > >> >> > > > the value yet. I > think if > > our goal > > > is an > > > > open > > > > >> > ecosystem > > > > >> > >> of these > > > > >> > >> >> > > header > > > > >> > >> >> > > > spread across many > plugins > > from many > > > > systems > > > > >> we > > > > >> > should > > > > >> > >> consider > > > > >> > >> >> > making > > > > >> > >> >> > > > this > > > > >> > >> >> > > > a string as well so > it can > > be > > > printed, set > > > > >> via > > > > >> > a UI, set > > > > >> > >> in > > > > >> > >> >> > config, > > > > >> > >> >> > > etc. > > > > >> > >> >> > > > Basically encouraging > > pluggable > > > > serialization > > > > >> > formats > > > > >> > >> here will > > > > >> > >> >> > lead > > > > >> > >> >> > > to > > > > >> > >> >> > > > a > > > > >> > >> >> > > > bit of a tower of > babel. > > > > >> > >> >> > > > 2. This proposal > still > > includes a > > > pretty > > > > big > > > > >> > change to our > > > > >> > >> >> > > serialization > > > > >> > >> >> > > > and protocol > definition > > layer. > > > Essentially > > > > >> it is > > > > >> > >> introducing an > > > > >> > >> >> > > optional > > > > >> > >> >> > > > type, where the > format is > > data > > > dependent. I > > > > >> > think this is > > > > >> > >> >> > actually a > > > > >> > >> >> > > big > > > > >> > >> >> > > > change though it > doesn't > > seem like > > > it. It > > > > >> means > > > > >> > you can no > > > > >> > >> >> > longer > > > > >> > >> >> > > > specify > > > > >> > >> >> > > > this type with our > type > > definition > > > DSL, and > > > > >> > likewise it > > > > >> > >> requires > > > > >> > >> >> > > custom > > > > >> > >> >> > > > handling in client > libs. > > This isn't > > > a huge > > > > >> > thing, since > > > > >> > >> the > > > > >> > >> >> > Record > > > > >> > >> >> > > > definition is custom > > anyway, but I > > > think > > > > this > > > > >> > kind of > > > > >> > >> protocol > > > > >> > >> >> > > > inconsistency is very > > non-desirable > > > and > > > > ties > > > > >> > you to > > > > >> > >> hand-coding > > > > >> > >> >> > > things. > > > > >> > >> >> > > > I > > > > >> > >> >> > > > think the type should > > instead by [Key > > > > Value] > > > > >> in > > > > >> > our BNF, > > > > >> > >> where > > > > >> > >> >> > key and > > > > >> > >> >> > > > value are both short > > strings as used > > > > >> elsewhere. > > > > >> > This > > > > >> > >> brings it > > > > >> > >> >> > in line > > > > >> > >> >> > > > with > > > > >> > >> >> > > > the rest of the > protocol. > > > > >> > >> >> > > > 3. Could we get more > > specific about > > > the > > > > exact > > > > >> > Java API > > > > >> > >> change to > > > > >> > >> >> > > > ProducerRecord, > > ConsumerRecord, > > > Record, > > > > etc? > > > > >> > >> >> > > > > > > > >> > >> >> > > > -Jay > > > > >> > >> >> > > > > > > > >> > >> >> > > > > > > >> > >> >> > > > > > >> > >> >> > > > > > >> > >> >> > The information contained in > this > > email is > > > strictly > > > > >> > confidential and > > > > >> > >> for > > > > >> > >> >> > the use of the addressee only, > unless > > otherwise > > > > >> indicated. > > > > >> > If you > > > > >> > >> are not > > > > >> > >> >> > the intended recipient, please > do not > > read, > > > copy, use > > > > or > > > > >> > disclose to > > > > >> > >> others > > > > >> > >> >> > this message or any attachment. > > Please also > > > notify the > > > > >> > sender by > > > > >> > >> replying > > > > >> > >> >> > to this email or by telephone > > (+44(020 7896 > > > 0011) and > > > > >> then > > > > >> > delete > > > > >> > >> the email > > > > >> > >> >> > and any copies of it. Opinions, > > conclusion > > > (etc) that > > > > do > > > > >> > not relate > > > > >> > >> to the > > > > >> > >> >> > official business of this > company > > shall be > > > understood > > > > as > > > > >> > neither > > > > >> > >> given nor > > > > >> > >> >> > endorsed by it. IG is a trading > name > > of IG > > > Markets > > > > >> Limited > > > > >> > (a company > > > > >> > >> >> > registered in England and Wales, > > company number > > > > >> 04008957) > > > > >> > and IG > > > > >> > >> Index > > > > >> > >> >> > Limited (a company registered in > > England and > > > Wales, > > > > >> > company number > > > > >> > >> >> > 01190902). Registered address at > > Cannon Bridge > > > House, > > > > 25 > > > > >> > Dowgate > > > > >> > >> Hill, > > > > >> > >> >> > London EC4R 2YA. Both IG Markets > > Limited > > > (register > > > > >> number > > > > >> > 195355) > > > > >> > >> and IG > > > > >> > >> >> > Index Limited (register number > > 114059) are > > > authorised > > > > >> and > > > > >> > regulated > > > > >> > >> by the > > > > >> > >> >> > Financial Conduct Authority. > > > > >> > >> >> > > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> -- > > > > >> > >> Nacho - Ignacio Solis - > iso...@igso.net > > > > >> > >> > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > The information contained in this email is > strictly > > > confidential and > > > > for > > > > >> > the use of the addressee only, unless otherwise > > indicated. If > > > you are > > > > >> not > > > > >> > the intended recipient, please do not read, > copy, use > > or > > > disclose to > > > > >> others > > > > >> > this message or any attachment. Please also > notify > > the sender by > > > > >> replying > > > > >> > to this email or by telephone (+44(020 7896 > 0011) and > > then > > > delete the > > > > >> email > > > > >> > and any copies of it. Opinions, conclusion > (etc) that > > do not > > > relate to > > > > >> the > > > > >> > official business of this company shall be > understood > > as > > > neither given > > > > >> nor > > > > >> > endorsed by it. IG is a trading name of IG > Markets > > Limited (a > > > company > > > > >> > registered in England and Wales, company number > > 04008957) and > > > IG Index > > > > >> > Limited (a company registered in England and > Wales, > > company > > > number > > > > >> > 01190902). Registered address at Cannon Bridge > House, > > 25 > > > Dowgate Hill, > > > > >> > London EC4R 2YA. Both IG Markets Limited > (register > > number > > > 195355) and > > > > IG > > > > >> > Index Limited (register number 114059) are > authorised > > and > > > regulated by > > > > >> the > > > > >> > Financial Conduct Authority. > > > > >> > > > > > >> The information contained in this email is > strictly > > confidential > > > and for > > > > >> the use of the addressee only, unless otherwise > > indicated. If you > > > are > > > > not > > > > >> the intended recipient, please do not read, copy, > use > > or disclose > > > to > > > > others > > > > >> this message or any attachment. Please also > notify the > > sender by > > > > replying > > > > >> to this email or by telephone (+44(020 7896 0011) > and > > then delete > > > the > > > > email > > > > >> and any copies of it. Opinions, conclusion (etc) > that > > do not > > > relate to > > > > the > > > > >> official business of this company shall be > understood > > as neither > > > given > > > > nor > > > > >> endorsed by it. IG is a trading name of IG Markets > > Limited (a > > > company > > > > >> registered in England and Wales, company number > > 04008957) and IG > > > Index > > > > >> Limited (a company registered in England and > Wales, > > company number > > > > >> 01190902). Registered address at Cannon Bridge > House, > > 25 Dowgate > > > Hill, > > > > >> London EC4R 2YA. Both IG Markets Limited (register > > number 195355) > > > and IG > > > > >> Index Limited (register number 114059) are > authorised > > and > > > regulated by > > > > the > > > > >> Financial Conduct Authority. > > > > >> > > > > > > > > > > > > > > > > > The information contained in this email is strictly > > confidential and > > > for the use of the addressee only, unless otherwise > indicated. > > If you are > > > not the intended recipient, please do not read, copy, use > or > > disclose to > > > others this message or any attachment. Please also notify > the > > sender by > > > replying to this email or by telephone (+44(020 7896 0011) > and > > then delete > > > the email and any copies of it. Opinions, conclusion (etc) > that > > do not > > > relate to the official business of this company shall be > > understood as > > > neither given nor endorsed by it. IG is a trading name of > IG > > Markets > > > Limited (a company registered in England and Wales, company > > number > > > 04008957) and IG Index Limited (a company registered in > England > > and Wales, > > > company number 01190902). Registered address at Cannon > Bridge > > House, 25 > > > Dowgate Hill, London EC4R 2YA. Both IG Markets Limited > (register > > number > > > 195355) and IG Index Limited (register number 114059) are > > authorised and > > > regulated by the Financial Conduct Authority. > > > > > > > > > > > > > > > > > > The information contained in this email is strictly confidential > and > > for the use of the addressee only, unless otherwise indicated. If > you are > > not the intended recipient, please do not read, copy, use or > disclose to > > others this message or any attachment. Please also notify the sender > by > > replying to this email or by telephone (+44(020 7896 0011) and then > delete > > the email and any copies of it. Opinions, conclusion (etc) that do > not > > relate to the official business of this company shall be understood > as > > neither given nor endorsed by it. IG is a trading name of IG Markets > > Limited (a company registered in England and Wales, company number > > 04008957) and IG Index Limited (a company registered in England and > Wales, > > company number 01190902). Registered address at Cannon Bridge House, > 25 > > Dowgate Hill, London EC4R 2YA. Both IG Markets Limited (register > number > > 195355) and IG Index Limited (register number 114059) are authorised > and > > regulated by the Financial Conduct Authority. > > > > > > > > > The information contained in this email is strictly confidential and for > the use of the addressee only, unless otherwise indicated. If you are not > the intended recipient, please do not read, copy, use or disclose to others > this message or any attachment. Please also notify the sender by replying > to this email or by telephone (+44(020 7896 0011) and then delete the email > and any copies of it. Opinions, conclusion (etc) that do not relate to the > official business of this company shall be understood as neither given nor > endorsed by it. IG is a trading name of IG Markets Limited (a company > registered in England and Wales, company number 04008957) and IG Index > Limited (a company registered in England and Wales, company number > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, > London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG > Index Limited (register number 114059) are authorised and regulated by the > Financial Conduct Authority. >
