Because the producer-property option is used to set other properties that are not compression type. //[email protected] -------- Original message --------From: ZHU Hua B <[email protected]> Date: 10/16/16 11:20 PM (GMT-08:00) To: Radoslaw Gruchalski <[email protected]>, [email protected] Subject: RE: A question about kafka Hi,
Thanks for your reply! If console producer only allows for compression codec argument, why we could found option —producer-property defined in ConsoleProducer.scala? And we could find the usage also if we running console producer? The version we used is Kafka 0.10.0.0. Thanks! # ./kafka-console-producer.sh Read data from standard input and publish it to Kafka. Option Description ------ ----------- --compression-codec [compression-codec] The compression codec: either 'none', 'gzip', 'snappy', or 'lz4'.If specified without value, then it defaults to 'gzip' --producer-property <producer_prop> A mechanism to pass user-defined properties in the form key=value to the producer. --producer.config <config file> Producer config properties file. Note that [producer-property] takes precedence over this config. --property <prop> A mechanism to pass user-defined properties in the form key=value to the message reader. This allows custom configuration for a user- defined message reader. Best Regards Johnny From: Radoslaw Gruchalski [mailto:[email protected]] Sent: 2016年10月17日 14:02 To: ZHU Hua B; [email protected] Subject: RE: A question about kafka Hi, I believe the answer is in the code. This is where the --compression-codec is processed: https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/tools/ConsoleProducer.scala#L143 and this is —producer-property: https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/tools/ConsoleProducer.scala#L234 The usage is here: https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/tools/ConsoleProducer.scala#L114 The answer is: The console producer allows for compression codec only with —compression-codec argument. – Best regards, Radek Gruchalski [email protected]<mailto:[email protected]> On October 17, 2016 at 7:46:41 AM, ZHU Hua B ([email protected]<mailto:[email protected]>) wrote: Hi, Anybody could help to answer this question? Thanks! Best Regards Johnny -----Original Message----- From: ZHU Hua B Sent: 2016年10月14日 16:41 To: [email protected]<mailto:[email protected]> Subject: [COMMERCIAL] A question about kafka Hi, I have a question about kafka, could you please help to have a look? I want to send a message from producer with snappy compression codec. So I run the command "bin/kafka-console-producer.sh --compression-codec snappy --broker-list localhost:9092 --topic test", after that I checked the data log, compresscodec is SnappyCompressionCodec as expectation. Then I tried another command "bin/kafka-console-producer.sh --producer-property compression.type=snappy --broker-list localhost:9092 --topic test" to send a message from producer with snappy also, but this time I found compresscodec is NoCompressionCodec in data log. I double checked the option of "kafka-console-producer.sh" as below, seems both "--compression-codec" and "--producer-property" could be used to set up compression codec, why the second way failed? The "key=value" I found in config/producer.properties, so I think it is regular. Thanks! # bin/kafka-console-producer.sh --help --compression-codec [compression-codec] The compression codec: either 'none','gzip', 'snappy', or 'lz4'.If specified without value, then it defaults to 'gzip'. --producer-property <producer_prop> A mechanism to pass user-defined properties in the form key=value to the producer. --producer.config <config file> Producer config properties file. Note that [producer-property] takes precedence over this config. Best Regards Johnny
