2016-10-25 21:36 GMT+02:00 Nacho Solis <nso...@linkedin.com.invalid>:
> I think you probably require a MagicByte bump if you expect correct > behavior of the system as a whole. > > From a client perspective you want to make sure that when you deliver a > message that the broker supports the feature you're expecting > (compaction). So, depending on the behavior of the broker on encountering > a previously undefined bit flag I would suggest making some change to make > certain that flag-based compaction is supported. I'm going to guess that > the MagicByte would do this. > I dont believe this is needed since it is already attributed through the request's API version. Producer: * if a client sends ProduceRequest V4 then attributes.bit5 indicates a tombstone * if a clients sends ProduceRequest <V4 then attributes.bit5 is ignored and value==null indicates a tombstone * in both cases the on-disk messages are stored with attributes.bit5 (I assume?) Consumer: * if a clients sends FetchRequest V4 messages are sendfile():ed directly from disk (with attributes.bit5) * if a client sends FetchRequest <V4 messages are slowpathed and translated from attributes.bit5 to value=null as required. That's my understanding anyway, please correct me if I'm wrong. /Magnus > On Tue, Oct 25, 2016 at 10:17 AM, Magnus Edenhill <mag...@edenhill.se> > wrote: > > > It is safe to assume that a previously undefined attributes bit will be > > unset in protocol requests from existing clients, if not, such a client > is > > already violating the protocol and needs to be fixed. > > > > So I dont see a need for a MagicByte bump, both broker and client has the > > information it needs to construct or parse the message according to > request > > version. > > > > > > 2016-10-25 18:48 GMT+02:00 Michael Pearce <michael.pea...@ig.com>: > > > > > Hi Magnus, > > > > > > I was wondering if I even needed to change those also, as technically > > > we’re just making use of a non used attribute bit, but im not 100% that > > it > > > be always false currently. > > > > > > If someone can say 100% it will already be set false with current and > > > historic bit wise masking techniques used over the time, we could do > away > > > with both, and simply just start to use it. Unfortunately I don’t have > > that > > > historic knowledge so was hoping it would be flagged up in this > > discussion > > > thread ☺ > > > > > > Cheers > > > Mike > > > > > > On 10/25/16, 5:36 PM, "Magnus Edenhill" <mag...@edenhill.se> wrote: > > > > > > Hi Michael, > > > > > > With the version bumps for Produce and Fetch requests, do you > really > > > need > > > to bump MagicByte too? > > > > > > Regards, > > > Magnus > > > > > > > > > 2016-10-25 18:09 GMT+02:00 Michael Pearce <michael.pea...@ig.com>: > > > > > > > Hi All, > > > > > > > > I would like to discuss the following KIP proposal: > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > > > 87+-+Add+Compaction+Tombstone+Flag > > > > > > > > This is off the back of the discussion on KIP-82 / KIP meeting > > > where it > > > > was agreed to separate this issue and feature. See: > > > > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610. > > > > mbox/%3cCAJS3ho8OcR==EcxsJ8OP99pD2hz=iiGecWsv- > > > > EZsBsNyDcKr=g...@mail.gmail.com%3e > > > > > > > > Thanks > > > > Mike > > > > > > > > The information contained in this email is strictly confidential > > and > > > for > > > > the use of the addressee only, unless otherwise indicated. If you > > > are not > > > > the intended recipient, please do not read, copy, use or disclose > > to > > > others > > > > this message or any attachment. Please also notify the sender by > > > replying > > > > to this email or by telephone (+44(020 7896 0011) and then delete > > > the email > > > > and any copies of it. Opinions, conclusion (etc) that do not > relate > > > to the > > > > official business of this company shall be understood as neither > > > given nor > > > > endorsed by it. IG is a trading name of IG Markets Limited (a > > company > > > > registered in England and Wales, company number 04008957) and IG > > > Index > > > > Limited (a company registered in England and Wales, company > number > > > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate > > > Hill, > > > > London EC4R 2YA. Both IG Markets Limited (register number 195355) > > > and IG > > > > Index Limited (register number 114059) are authorised and > regulated > > > by the > > > > Financial Conduct Authority. > > > > > > > > > > > > > The information contained in this email is strictly confidential and > for > > > the use of the addressee only, unless otherwise indicated. If you are > not > > > the intended recipient, please do not read, copy, use or disclose to > > others > > > this message or any attachment. Please also notify the sender by > replying > > > to this email or by telephone (+44(020 7896 0011) and then delete the > > email > > > and any copies of it. Opinions, conclusion (etc) that do not relate to > > the > > > official business of this company shall be understood as neither given > > nor > > > endorsed by it. IG is a trading name of IG Markets Limited (a company > > > registered in England and Wales, company number 04008957) and IG Index > > > Limited (a company registered in England and Wales, company number > > > 01190902). Registered address at Cannon Bridge House, 25 Dowgate Hill, > > > London EC4R 2YA. Both IG Markets Limited (register number 195355) and > IG > > > Index Limited (register number 114059) are authorised and regulated by > > the > > > Financial Conduct Authority. > > > > > > > > > -- > Nacho (Ignacio) Solis > Kafka > nso...@linkedin.com >
