another potential benefit of headers is it would reduce the number of API changes required to support future features (as they could be implemented as plugins). that would greatly accelerate the rate with which kafka can be extended.
On Mon, Oct 3, 2016 at 12:46 PM, Michael Pearce <michael.pea...@ig.com> wrote: > Opposite way around v4 instead of v3 ;) > ________________________________________ > From: Michael Pearce > Sent: Monday, October 3, 2016 8:45 PM > To: dev@kafka.apache.org > Subject: Re: [DISCUSS] KIP-82 - Add Record Headers > > Thanks guys for spotting this, i have updated KIP-82 to state v3 instead > of v4. > > Mike. > ________________________________________ > From: Becket Qin <becket....@gmail.com> > Sent: Monday, October 3, 2016 6:18 PM > To: dev@kafka.apache.org > Subject: Re: [DISCUSS] KIP-82 - Add Record Headers > > Yes, KIP-74 has already been checked in. The new FetchRequest/Response > version should be V4. :) > > On Mon, Oct 3, 2016 at 10:14 AM, Sean McCauliff < > smccaul...@linkedin.com.invalid> wrote: > > > Change to public interfaces: > > > > "Add ProduceRequest/ProduceResponse V3 which uses the new message format. > > Add FetchRequest/FetchResponse V3 which uses the new message format." > > > > When I look at org.apache.kafka.common.requests.FetchResponse on > > master I see that there is already a version 3. Seems like this is > > from a recent commit about implementing KIP-74. Do we need to > > coordinate these changes with KIP-74? > > > > > > "The serialisation of the [int, bye[]] header set will on the wire > > using a strict format" bye[] -> byte[] > > > > Sean > > -- > > Sean McCauliff > > Staff Software Engineer > > Kafka > > > > smccaul...@linkedin.com > > linkedin.com/in/sean-mccauliff-b563192 > > > > > > On Fri, Sep 30, 2016 at 3:43 PM, radai <radai.rosenbl...@gmail.com> > wrote: > > > I think headers are a great idea. > > > > > > Right now, people who are trying to implement any sort of org-wide > > > functionality like monitoring, tracing, profiling etc pretty much have > to > > > define their own wrapper layers, which probably leads to everyone > > > implementing their own variants of the same underlying functionality. > > > > > > I think a common base for headers would allow implementing a lot of > this > > > functionality only one in a way that different header-based > capabilities > > > could be shared and composed and open the door the a wide range of > > possible > > > Kafka middleware that's simply impossible to write against the current > > API. > > > > > > Here's a list of things that could be implemented as "plugins" on top > of > > a > > > header mechanism (full list here - > > > https://cwiki.apache.org/confluence/display/KAFKA/A+ > > Case+for+Kafka+Headers). > > > > > > A lot of these already exist within LinkedIn and could for example be > > open > > > sourced if Kafka had headers. I'm fairly certain the same is true in > > other > > > organizations using Kafka > > > > > > > > > > > > On Thu, Sep 22, 2016 at 12:31 PM, Michael Pearce < > michael.pea...@ig.com> > > > wrote: > > > > > >> Hi All, > > >> > > >> > > >> I would like to discuss the following KIP proposal: > > >> > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > >> 82+-+Add+Record+Headers > > >> > > >> > > >> > > >> I have some initial ?drafts of roughly the changes that would be > needed. > > >> This is no where finalized and look forward to the discussion > > especially as > > >> some bits I'm personally in two minds about. > > >> > > >> https://github.com/michaelandrepearce/kafka/tree/ > > kafka-headers-properties > > >> > > >> > > >> > > >> Here is a link to a alternative option mentioned in the kip but one i > > >> would personally would discard (disadvantages mentioned in kip) > > >> > > >> https://github.com/michaelandrepearce/kafka/tree/kafka-headers-full? > > >> > > >> > > >> Thanks > > >> > > >> Mike > > >> > > >> > > >> > > >> > > >> > > >> The information contained in this email is strictly confidential and > for > > >> the use of the addressee only, unless otherwise indicated. If you are > > not > > >> the intended recipient, please do not read, copy, use or disclose to > > others > > >> this message or any attachment. Please also notify the sender by > > replying > > >> to this email or by telephone (+44(020 7896 0011) and then delete the > > email > > >> and any copies of it. Opinions, conclusion (etc) that do not relate to > > the > > >> official business of this company shall be understood as neither given > > nor > > >> endorsed by it. IG is a trading name of IG Markets Limited (a company > > >> registered in England and Wales, company number 04008957) and IG Index > > >> Limited (a company registered in England and Wales, company number > > >> 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, > > >> London EC4R 2YA. Both IG Markets Limited (register number 195355) and > IG > > >> Index Limited (register number 114059) are authorised and regulated by > > the > > >> Financial Conduct Authority. > > >> > > > The information contained in this email is strictly confidential and for > the use of the addressee only, unless otherwise indicated. If you are not > the intended recipient, please do not read, copy, use or disclose to others > this message or any attachment. Please also notify the sender by replying > to this email or by telephone (+44(020 7896 0011) and then delete the email > and any copies of it. Opinions, conclusion (etc) that do not relate to the > official business of this company shall be understood as neither given nor > endorsed by it. IG is a trading name of IG Markets Limited (a company > registered in England and Wales, company number 04008957) and IG Index > Limited (a company registered in England and Wales, company number > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, > London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG > Index Limited (register number 114059) are authorised and regulated by the > Financial Conduct Authority. >
