Getting maximum message size

2016-08-09 Thread Sergio Gonzalez
Hi users, How i should setting the broker (replica.fetch.max.bytes, message.max.bytes) and the consumer (fetch.message.max.bytes) to transmit the MAXIMUM message size in kafka_2.10-0.8.2.0 version? Thanks, ​Sergio GQ​

Compression + Batch problem?

2016-08-09 Thread Felipe Santos
I was testing kafka 0.10 with and without compression, when I actived compression I got a message that max bytes exceeded with same batch size. the command was this with compression kafka-producer-perf-test --topic test-rep-one4 --num-records 10 --record-size 35000 --throughput -1 --producer-

Re: Log compaction leaves empty files?

2016-08-09 Thread Dustin Cote
I would say the zero-byte files should be cleaned up by the LogCleaner, but on Windows, I'm not sure how any of this works at this point. I'll let someone with more experience in that area chime in. On Mon, Aug 8, 2016 at 8:29 AM, Harald Kirsch wrote: > Hi Dustin, > > thanks for the reply. to b

Re: Not able to create test topic from quick guide

2016-08-09 Thread Pankaj Agrawal
Thanks for the reply Siva. Issue has been resolved. It was a local environmental issue. Rgds P.K On Tue, Aug 9, 2016 at 8:41 PM, Sivakumaran S wrote: > Have you started Zookeeper and the Kafka Cluster? > >- Start the Zookeeper Server by the command > - zookeeper-server-start.sh $KAFK

Re: Not able to create test topic from quick guide

2016-08-09 Thread Sivakumaran S
Have you started Zookeeper and the Kafka Cluster? Start the Zookeeper Server by the command zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties Start the Kafka cluster by the command kafka-server-start.sh $KAFKA_HOME/config/server.properties Regards, Sivakumaran S > On 09-Aug-2016,

Not able to create test topic from quick guide

2016-08-09 Thread Pankaj Agrawal
Hi I’m not able to create test topic following steps given in quick guide. kafka and zookeeper starts but following cos yield no output/result: bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test bin/kafka-topics.sh --list --zookeeper loc

Mirror maker committed offset is ahead of log end offset

2016-08-09 Thread tao xiao
Hi team, I was seeing an issue where a mirror maker attempted to commit an offset for a partition that was ahead of log end offset and in the mean time the leader of the partition was being restarted. In theory only committed messages can be consumed by consumer which means the messages received b