Re: KIP-57: Interoperable LZ4 Framing

2016-05-09 Thread Gwen Shapira
Agree. On Fri, May 6, 2016 at 3:14 PM, Ismael Juma wrote: > On Fri, May 6, 2016 at 11:07 PM, Dana Powers wrote: > >> Updated: >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing >> >> Should I restart the vote? >> > &g

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

2016-05-09 Thread Magnus Edenhill
t; > > > > KafkaLZ4* code: > > > > > > > > > >- add naive support for optional header flags (ContentSize, > > > > >ContentChecksum) to enable interoperability with off-the-shelf > lz4 > > > > libraries > > > &g

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

2016-05-07 Thread Dana Powers
; >ContentChecksum) to enable interoperability with off-the-shelf lz4 > > > libraries > > > >- the only flag left unsupported is dependent-block compression, > > which > > > >our implementation does not currently support. > > > &

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

2016-05-07 Thread Ismael Juma
-shelf lz4 > > libraries > > >- the only flag left unsupported is dependent-block compression, > which > > >our implementation does not currently support. > > > > > > > > > Thanks, > > > > > > Jun > > > > > > O

Re: KIP-57: Interoperable LZ4 Framing

2016-05-06 Thread Ismael Juma
On Fri, May 6, 2016 at 11:07 PM, Dana Powers wrote: > Updated: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing > > Should I restart the vote? > I think the update is small enough that we don't need to restart the vote. Ismael

Re: KIP-57: Interoperable LZ4 Framing

2016-05-06 Thread Dana Powers
Ok -- removed Public Interfaces discussion. It should be up to date w/ PR review comments now. -Dana On Fri, May 6, 2016 at 2:15 PM, Ismael Juma wrote: > One more suggestion Dana, I would remove the "Public interfaces" section as > those classes are not actually public (only the classes with Jav

Re: KIP-57: Interoperable LZ4 Framing

2016-05-06 Thread Dana Powers
Updated: https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing Should I restart the vote? -Dana On Fri, May 6, 2016 at 2:12 PM, Ismael Juma wrote: > On Sun, May 1, 2016 at 3:57 AM, Dana Powers wrote: >> >> > 2. We're completely disabl

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

2016-05-06 Thread Ismael Juma
t;- the only flag left unsupported is dependent-block compression, > which > > >our implementation does not currently support. > > > > > > > > > Thanks, > > > > > > Jun > > > > > > On Mon, Apr 25, 2016 at 2:26 PM, Dan

Re: KIP-57: Interoperable LZ4 Framing

2016-05-06 Thread Ismael Juma
One more suggestion Dana, I would remove the "Public interfaces" section as those classes are not actually public (only the classes with Javadoc are public: https://kafka.apache.org/090/javadoc/index.html) and the information in the KIP is a bit stale when compared to the PR. Ismael On Fri, May 6

Re: KIP-57: Interoperable LZ4 Framing

2016-05-06 Thread Ismael Juma
On Sun, May 1, 2016 at 3:57 AM, Dana Powers wrote: > > > 2. We're completely disabling checksumming of the compressed payload on > > consumption. Normally you'd want to validate each level of framing for > > correct end-to-end validation. You could still do this (albeit more > weakly) > > by valid

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

2016-05-04 Thread Jun Rao
dent-block compression, which > >our implementation does not currently support. > > > > > > Thanks, > > > > Jun > > > > On Mon, Apr 25, 2016 at 2:26 PM, Dana Powers > wrote: > > > >> Hi all, > >> > >> Initiating a

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

2016-05-03 Thread Magnus Edenhill
g seems to > suggest > >> > there are other format changes? > >> > > >> > KafkaLZ4* code: > >> > > >> >- add naive support for optional header flags (ContentSize, > >> >ContentChecksum) to enable interoperability w

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

2016-05-03 Thread Dana Powers
e-shelf lz4 >> libraries >> >- the only flag left unsupported is dependent-block compression, which >> >our implementation does not currently support. >> > >> > >> > Thanks, >> > >> > Jun >> > >> > On Mon

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

2016-05-03 Thread Ewen Cheslack-Postava
- the only flag left unsupported is dependent-block compression, which > >our implementation does not currently support. > > > > > > Thanks, > > > > Jun > > > > On Mon, Apr 25, 2016 at 2:26 PM, Dana Powers > wrote: > > > >> Hi a

Re: KIP-57: Interoperable LZ4 Framing

2016-04-30 Thread Dana Powers
On Fri, Apr 29, 2016 at 6:29 PM, Ewen Cheslack-Postava wrote: > Two questions: > > 1. My understanding based on KIP-35 is that this won't be a problem for > clients that want to support older broker versions since they will use v0 > produce requests with broken checksum to send to those, and any b

Re: KIP-57: Interoperable LZ4 Framing

2016-04-29 Thread Ewen Cheslack-Postava
n up a new KIP based on KAFKA-3160 / fixing LZ4 framing. > > The write-up is here: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing > > > > Please take a look if you are using LZ4 compression or are interested > &

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

2016-04-28 Thread Dana Powers
KIP-57 proposal is specific to >> the 0.10 release. >> >> KIP-57 can be accessed here: >> < >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing >> >. >> >> The related JIRA is https://issues.apache.org/jira/browse/KAFKA-3160 >> and working github PR at https://github.com/apache/kafka/pull/1212 >> >> The vote will run for 72 hours. >> >> +1 (non-binding) >>

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

2016-04-28 Thread Jun Rao
he KIP-57 proposal is specific to > the 0.10 release. > > KIP-57 can be accessed here: > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing > >. > > The related JIRA is https://issues.apache.org/jira/browse/KAFKA-3160 > and work

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

2016-04-25 Thread Gwen Shapira
+1 On Mon, Apr 25, 2016 at 2:26 PM, Dana Powers wrote: > Hi all, > > Initiating a vote thread because the KIP-57 proposal is specific to > the 0.10 release. > > KIP-57 can be accessed here: > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+F

Re: [VOTE] KIP-57: Interoperable LZ4 Framing

2016-04-25 Thread Magnus Edenhill
+1 (non-binding) 2016-04-25 14:26 GMT-07:00 Dana Powers : > Hi all, > > Initiating a vote thread because the KIP-57 proposal is specific to > the 0.10 release. > > KIP-57 can be accessed here: > < > https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperab

[VOTE] KIP-57: Interoperable LZ4 Framing

2016-04-25 Thread Dana Powers
Hi all, Initiating a vote thread because the KIP-57 proposal is specific to the 0.10 release. KIP-57 can be accessed here: <https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing>. The related JIRA is https://issues.apache.org/jira/browse/KAFKA-3160 and w

Re: KIP-57: Interoperable LZ4 Framing

2016-04-25 Thread Gwen Shapira
Hi all, > > I've written up a new KIP based on KAFKA-3160 / fixing LZ4 framing. > The write-up is here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing > > Please take a look if you are using LZ4 compression or are interested > in

Re: KIP-57: Interoperable LZ4 Framing

2016-04-25 Thread Ismael Juma
ve written up a new KIP based on KAFKA-3160 / fixing LZ4 framing. > The write-up is here: > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing > > Please take a look if you are using LZ4 compression or are interested > in framing specs. >

KIP-57: Interoperable LZ4 Framing

2016-04-25 Thread Dana Powers
Hi all, I've written up a new KIP based on KAFKA-3160 / fixing LZ4 framing. The write-up is here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-57+-+Interoperable+LZ4+Framing Please take a look if you are using LZ4 compression or are interested in framing specs. One question: