Re: Problems consuming snappy compressed messages via SimpleConsumer

2014-02-27 Thread Neha Narkhede
I was actually referring to kafka-simple-consumer-shell.sh. Please make sure that you pass the same fetch size that you configured your high level consumer with, to the kafka-simple-consumer-shell with the --fetchsize option. Thanks, Neha On Wed, Feb 26, 2014 at 6:26 PM, Dan Hoffman wrote: > T

Re: Problems consuming snappy compressed messages via SimpleConsumer

2014-02-26 Thread Dan Hoffman
The kafka-console-consumer is fine. On Wednesday, February 26, 2014, Neha Narkhede wrote: > Actually I meant the simple consumer shell that ships with kafka in the bin > directory. > > > On Wed, Feb 26, 2014 at 6:17 PM, Dan Hoffman wrote: > > > I haven't tried that yet. But since the high level

Re: Problems consuming snappy compressed messages via SimpleConsumer

2014-02-26 Thread Neha Narkhede
Actually I meant the simple consumer shell that ships with kafka in the bin directory. On Wed, Feb 26, 2014 at 6:17 PM, Dan Hoffman wrote: > I haven't tried that yet. But since the high level consumer can consume it, > should it matter who published it? > > On Wednesday, February 26, 2014, Neha

Re: Problems consuming snappy compressed messages via SimpleConsumer

2014-02-26 Thread Dan Hoffman
I haven't tried that yet. But since the high level consumer can consume it, should it matter who published it? On Wednesday, February 26, 2014, Neha Narkhede wrote: > Do you see the same issue if you send snappy data using the console > producer instead of librdkafka? > > > On Wed, Feb 26, 2014

Re: Problems consuming snappy compressed messages via SimpleConsumer

2014-02-26 Thread Neha Narkhede
Do you see the same issue if you send snappy data using the console producer instead of librdkafka? On Wed, Feb 26, 2014 at 5:58 PM, Dan Hoffman wrote: > Publisher (using librdkafka C api) has sent both gzip and snappy compressed > messages. I find that the java Simple Consumer ( > > https://c

Problems consuming snappy compressed messages via SimpleConsumer

2014-02-26 Thread Dan Hoffman
Publisher (using librdkafka C api) has sent both gzip and snappy compressed messages. I find that the java Simple Consumer ( https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example#) is unable to read the snappy ones, while the High Level one is. Is this expected? Is ther