Well, the files that I am sending are themselves not compressed, but I do use
gzip compression when
sending data. kafka client is configured with option
compression.codec=1
which implies gzip compression. Setting that value to 2 means "Snappy
compression". What compression
would you recommend? Is "Snappy compression" any good?
--
Dmitri Priimak
On 11/30/2012 01:17 PM, Paul Sutter wrote:
> Any chance the stream was created by the gzip command? If so, that just
> happens sometime. The only
> way we found is to confirm that the error happened at the expected end of the
> file and not a
> too-short file (we learned this dealing with petabytes of gziped logfiles)
>
> Apologies if thats irrelevant.
>
> Begin forwarded message:
>
>> *From:* Dmitri Priimak <prii...@highwire.stanford.edu
>> <mailto:prii...@highwire.stanford.edu>>
>> *Date:* November 30, 2012 12:26:10 PM PST
>> *To:* users@kafka.apache.org <mailto:users@kafka.apache.org>
>> *Subject:* *Re: Unexpected end of ZLIB input stream*
>> *Reply-To:* users@kafka.apache.org <mailto:users@kafka.apache.org>
>>
>> Well, it does not look like I can reproduce it anymore. While I was able to
>> consistently crash it
>> yesterday, today kafka broker runs without any problems.
>>
>> --
>> Dmitri Priimak
>>
>> On 11/29/2012 02:10 PM, Dmitri Priimak wrote:
>>> On 11/29/2012 02:06 PM, Neha Narkhede wrote:
>>>> That just meant that we couldn't reproduce it during our testing, but
>>>> we occasionally do see it in production, which is much harder to
>>>> reproduce.
>>>> It will be great if you can reproduce the issue and attach the test case.
>>> I see. I will try to isolate this problem. I am hoping that I can locate
>>> exact sequence of messages
>>> that trigger this problem, but at the moment I cannot easily do that.
>>>
>>> --
>>> Dmitri Priimak
