Two questions: 1. My understanding based on KIP-35 is that this won't be a problem for clients that want to support older broker versions since they will use v0 produce requests with broken checksum to send to those, and any broker advertising support for v1 produce requests will also support valid checksums? In other words, the KIP is structured in terms of Java client versions, but I'd like to make sure we have the compatibility path for non-Java clients cleanly mapped out. (And think we do, especially given that Dana is proposing, but still would like an ack on that.) 2. We're completely disabling checksumming of the compressed payload on consumption. Normally you'd want to validate each level of framing for correct end-to-end validation. You could still do this (albeit more weakly) by validating the checksum is one of the two potentially valid values (correct checksum or old, incorrect checksum). This obviously has computational cost. Are we sure the tradeoff we're going with makes sense?
-Ewen On Mon, Apr 25, 2016 at 2:01 PM, Gwen Shapira <g...@confluent.io> wrote: > Hi Dana, > > Thank you for proposing this fix. It looks great to me (and LinkedIn, > who are running trunk confirmed that they did not use LZ4 yet). > Since for 0.10.0 time is of essence, do you mind starting a voting > thread in parallel? > > Gwen > > On Mon, Apr 25, 2016 at 9:30 AM, Dana Powers <dana.pow...@gmail.com> > wrote: > > Hi all, > > > > I've written up a new KIP based on KAFKA-3160 / fixing LZ4 framing. > > The write-up is here: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing > > > > Please take a look if you are using LZ4 compression or are interested > > in framing specs. > > > > One question: does anyone out there fall into this category: (1) > > deploy from trunk, (2) upgraded to v1 message format, (3) using > > LZ4-compressed messages ? > > > > > > Feedback welcome, > > > > -Dana > -- Thanks, Ewen
