I think headers are a great idea. Right now, people who are trying to implement any sort of org-wide functionality like monitoring, tracing, profiling etc pretty much have to define their own wrapper layers, which probably leads to everyone implementing their own variants of the same underlying functionality.
I think a common base for headers would allow implementing a lot of this functionality only one in a way that different header-based capabilities could be shared and composed and open the door the a wide range of possible Kafka middleware that's simply impossible to write against the current API. Here's a list of things that could be implemented as "plugins" on top of a header mechanism (full list here - https://cwiki.apache.org/confluence/display/KAFKA/A+Case+for+Kafka+Headers). A lot of these already exist within LinkedIn and could for example be open sourced if Kafka had headers. I'm fairly certain the same is true in other organizations using Kafka On Thu, Sep 22, 2016 at 12:31 PM, Michael Pearce <michael.pea...@ig.com> wrote: > Hi All, > > > I would like to discuss the following KIP proposal: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 82+-+Add+Record+Headers > > > > I have some initial ?drafts of roughly the changes that would be needed. > This is no where finalized and look forward to the discussion especially as > some bits I'm personally in two minds about. > > https://github.com/michaelandrepearce/kafka/tree/kafka-headers-properties > > > > Here is a link to a alternative option mentioned in the kip but one i > would personally would discard (disadvantages mentioned in kip) > > https://github.com/michaelandrepearce/kafka/tree/kafka-headers-full? > > > Thanks > > Mike > > > > > > The information contained in this email is strictly confidential and for > the use of the addressee only, unless otherwise indicated. If you are not > the intended recipient, please do not read, copy, use or disclose to others > this message or any attachment. Please also notify the sender by replying > to this email or by telephone (+44(020 7896 0011) and then delete the email > and any copies of it. Opinions, conclusion (etc) that do not relate to the > official business of this company shall be understood as neither given nor > endorsed by it. IG is a trading name of IG Markets Limited (a company > registered in England and Wales, company number 04008957) and IG Index > Limited (a company registered in England and Wales, company number > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, > London EC4R 2YA. Both IG Markets Limited (register number 195355) and IG > Index Limited (register number 114059) are authorised and regulated by the > Financial Conduct Authority. >
