Some notes on this: - remember that Messages are wrapped in a compressed MessageSet that is wrapped in an outer Message which is wrapped in an outer MessageSet (the mind boggles). - the Kafka snappy compression has a non-standard framing - snappy-java (prefix "\x82SNAPPY\x0") - this framing is specific to the java implementation and is not in the official snappy spec or implementation.
You can use the rdkafka_example client from librdkafka to send snappy-compressed messages: ./rdkafka_example -P -t topic -p 0 -b mybroker -z snappy 2014-03-27 19:21 GMT+07:00 小宇 <mocking...@gmail.com>: > Yes,maybe I missed something, and how can I send message in snappy > compression with Kafka-console-producer? > Thanks. > > Neha Narkhede <neha.narkh...@gmail.com>于2014年3月27日星期四写道： > > > Have you followed the wire protocol documented here - > > > > > https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol > > > > > > > > > > On Thu, Mar 27, 2014 at 3:22 AM, 小宇 <mocking...@gmail.com<javascript:;>> > > wrote: > > > > > Hi, all > > > I'm working on snappy compression in our Node.js client ( > > > https://github.com/SOHU-Co/kafka-node),but I found the offset of > message > > > always 0 after snappy compression, for example: <Buffer 00 00 00 00 00 > 00 > > > 00 00 00 00 00 19 3e 69 5f c8 00 00 00 00 00 00 00 00 00 0b 7b 22 6f 6e > > 65 > > > 22 3a 22 31 22 7d>, I try to test it in kafka-cosole-producer.sh, but > it > > > seems not support snappy. So I can't figure out what's wrong with my > > > client. Did anyone encounter the same problem? > > > Thanks. > > > > > >
