Re: [VOTE] KIP-57: Interoperable LZ4 Framing

I have verified this functionality on kafka trunk using librdkafka lz4 branch and it works as intended. 2016-05-07 18:07 GMT+02:00 Dana Powers : > Vote Passed! I will update the wiki. > > -Dana > On May 7, 2016 3:48 AM, "Ismael Juma" wrote: > > > Dana, a long time has passed since the vote sta

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

Vote Passed! I will update the wiki. -Dana On May 7, 2016 3:48 AM, "Ismael Juma" wrote: > Dana, a long time has passed since the vote started and there are enough > binding votes, so maybe it's time to declare that the vote has passed? > Please mark the KIP as adopted in the KIP page and move it

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

Dana, a long time has passed since the vote started and there are enough binding votes, so maybe it's time to declare that the vote has passed? Please mark the KIP as adopted in the KIP page and move it to the adopted table in the KIPs page once you do that. Ismael On 6 May 2016 22:16, "Ismael Jum

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

+1 (assuming the changes I mentioned in the discuss thread are incorporated) Ismael On Thu, May 5, 2016 at 1:13 AM, Jun Rao wrote: > Thanks for the response. +1 on the KIP. > > Jun > > On Thu, Apr 28, 2016 at 9:01 AM, Dana Powers > wrote: > > > Sure thing. Yes, the substantive change is fixing

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

Thanks for the response. +1 on the KIP. Jun On Thu, Apr 28, 2016 at 9:01 AM, Dana Powers wrote: > Sure thing. Yes, the substantive change is fixing the HC checksum. > > But to further improve interoperability, the kafka LZ4 class would no > longer reject messages that have these optional header

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

Good work on this Dana. I'll test it with librdkafka (which uses the official liblz4) and report back. 2016-05-03 20:02 GMT+02:00 Dana Powers : > Yes, great point. The intent of adding "naive" support for the > remaining LZ4 header flags (contentsize and contentchecksum) is to > avoid rejecting

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

Yes, great point. The intent of adding "naive" support for the remaining LZ4 header flags (contentsize and contentchecksum) is to avoid rejecting any lz4 messages framed according to the interoperable spec (v1.5.1). If we can accept all such messages (which this KIP does), we should never have to m

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

+1 One caveat on the vote though -- I don't know the details of LZ4 (format, libraries, etc) well enough to have a handle on whether the changes under "KafkaLZ4* code" are going to be sufficient to get broad support from other LZ4 libraries. Are we going to have multiple implementations we can tes

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

Sure thing. Yes, the substantive change is fixing the HC checksum. But to further improve interoperability, the kafka LZ4 class would no longer reject messages that have these optional header flags set. The flags might get set if the client/user chooses to use a non-java lz4 compression library th

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

Hi, Dana, Could you explain the following from the KIP a bit more? The KIP is intended to just fix the HC checksum, but the following seems to suggest there are other format changes? KafkaLZ4* code: - add naive support for optional header flags (ContentSize, ContentChecksum) to enable inte

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

+1 On Mon, Apr 25, 2016 at 2:26 PM, Dana Powers wrote: > Hi all, > > Initiating a vote thread because the KIP-57 proposal is specific to > the 0.10 release. > > KIP-57 can be accessed here: > . > > The related J

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

+1 (non-binding) 2016-04-25 14:26 GMT-07:00 Dana Powers : > Hi all, > > Initiating a vote thread because the KIP-57 proposal is specific to > the 0.10 release. > > KIP-57 can be accessed here: > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing > >. > > The

[VOTE] KIP-57: Interoperable LZ4 Framing

Hi all, Initiating a vote thread because the KIP-57 proposal is specific to the 0.10 release. KIP-57 can be accessed here: . The related JIRA is https://issues.apache.org/jira/browse/KAFKA-3160 and working gith