I enabled gzip compression. Each topic has 10 partitions and each partition
is handled by 1 simple consumer thread. All consumers stop to iterate after
iterate first several responses. The responses still return with bytes, but
cannot iterate.
On Thu, Oct 31, 2013 at 9:59 PM, Lu Xuechao <lux...@gmail.com> wrote:
> No. The simple consumer does receive some responses and can iterate the
> loop:
>
>
> for (MessageAndOffset messageAndOffset : fetchResponse.messageSet(m_topic,
> m_partition)) {
> //handle messages
> }
>
> but after that, the response still returns will byte[], I can see the
> content, but the iterator cannot iterate:
>
>
> Iterator<MessageAndOffset> itr = fetchResponse.messageSet(m_topic,
> m_partition).iterator();
> itr.hasNext() is FALSE.
>
> No error messages found.
>
>
>
> On Thu, Oct 31, 2013 at 9:33 PM, Jun Rao <jun...@gmail.com> wrote:
>
>> Is that related to
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whybrokersdonotreceiveproducersentmessages%3F
>> ?
>>
>> Thanks,
>>
>> Jun
>>
>>
>> On Thu, Oct 31, 2013 at 2:23 PM, Lu Xuechao <lux...@gmail.com> wrote:
>>
>> > It seems the reason is I enabled gzip compression.
>> >
>> > what the code would like to consume compressed messages?
>> >
>> > thanks.
>> >
>> >
>> > On Thu, Oct 31, 2013 at 11:26 AM, Lu Xuechao <lux...@gmail.com> wrote:
>> >
>> > > Hi,
>> > >
>> > > I am following the
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example
>> > >
>> > > When I send KeyedMessage<String, String> with StringEncoder, I can get
>> > the
>> > > messages sent:
>> > >
>> > > for (MessageAndOffset messageAndOffset :
>> > fetchResponse.messageSet(m_topic,
>> > > m_partition)) {
>> > > //handle messages
>> > > }
>> > >
>> > > But when I send KeyedMessage<byte[], byte[]> with DefaultEncoder, I
>> > cannot
>> > > get the messages:
>> > >
>> > > Iterator<MessageAndOffset> itr = fetchResponse.messageSet(m_topic,
>> > > m_partition).iterator();
>> > > itr.hasNext() is FALSE.
>> > >
>> > > the test code is the same, what is causing this? What change needs to
>> be
>> > > made?
>> > >
>> > > thanks.
>> > >
>> >
>>
>
>
