Re: 答复: kafka performance question

Do you read from the file in the callback from kafka? I just implemented c++ bindings and in one of the tests i did I got the following results: 1000 messages per batch (fairly small messages ~150 bytes) and then wait for the network layer to ack the send (not server ack)'s before putting another

Mbean "kafka.server":type="BrokerTopicMetrics",name="AllTopicsMessagesInPerSec" returning 0 always

Hi, We have a Kafka cluster which contains 3 kafka brokers. Data is pushed into kafka queue continously. To monitor the kafka brokers, we use the JMX metrics given by kafka. One thing i observed is there is one broker which is returning*"0"* all the time for the bean *"kafka.server":type="Brok

Re: Mbean "kafka.server":type="BrokerTopicMetrics",name="AllTopicsMessagesInPerSec" returning 0 always

Hi Arjun , What is the configured number of logical partitions per topic per server . With the partition size as 2 , Only two partitions would be created distributed over 2 brokers with one partition each , you can confirm the same by checking the log file directory for each broker (default - /tm

multi-broker topic rebalance

Hello! How can I rebalance topic within multiple brokers? For example: 1.I have about 30k consumers... 2.I need to use 3 broker (10k consumer per broker) 3. I use multiple (around 10) producers with default balance strategy Sometimes one broker have empty topic (consumer

multi-broker topic rebalance

Hello! How can I rebalance topic within multiple brokers? For example: 1.I have about 30k consumers... 2.I need to use 3 broker (10k consumer per broker) 3. I use multiple (around 10) producers with default balance strategy Sometimes one broker have empty topic (consumer

Re: Unable to scale kafka cluster

To add partitions, you need to use the --alter option in kafka-topics.sh. Thanks, Jun On Sat, May 24, 2014 at 3:52 AM, Ertuğrul Yılmaz < ertugrul.yil...@convertale.com> wrote: > Hi, > > I know replica partition tools. I try to add new partition and assign to a > new broker during producer test

Re: how many instances of the broker is suggested in one server.

The number of instances required will depend on the load. However, keep in mind that all instances on a server will compete for memory (e.g., for pagecache, jvm heap, etc), storage (log space) and CPU. Thanks, Jun On Sun, May 25, 2014 at 11:10 PM, Zhujie (zhujie, Smartcare) < first.zhu...@huawe

Re: multi-broker topic rebalance

Which version of Kafka are you using? How many partitions do you have in that topic? Thanks, Jun On Mon, May 26, 2014 at 7:14 AM, Рябков Алексей Николаевич < a.ryab...@ntc-vulkan.ru> wrote: > > Hello! > > How can I rebalance topic within multiple brokers? > For example: > > 1.I have ab

Re: kafka broker failed to recovery from ZK failure

well. I just run the kafka-run-class.sh script with DeleteTopicCommand class directly. I noticed the warning on documentation page. but I thought it is only about not complete cleanup. I didn't know it can affect cluster health. maybe it's worthwhile to point it out the

HA: multi-broker topic rebalance

Hello! >>Which version of Kafka are you using? 1. I am using 0.8.1 >>How many partitions do you have in that topic? 2. In docs I found that number of partition must be equal or greater than number of consumers ... so I try 2 case: - 10k partition per broker - 20k partition per broker And

答复: how many instances of the broker is suggested in one server.

Is there any principles? -邮件原件- 发件人: Jun Rao [mailto:jun...@gmail.com] 发送时间: 2014年5月27日 11:25 收件人: users@kafka.apache.org 主题: Re: how many instances of the broker is suggested in one server. The number of instances required will depend on the load. However, keep in mind that all instan

java.net.SocketTimeoutException in Broker

Hi All, This is what we are running on the Broker: /Users/worun/kafkabuild/kafka-0.8.1-src/bin/kafka-producer-perf-test.sh --broker-list vp21q12ic-hpaj020921:9092 --messages 1000 --topic imessage --threads 10 --message-size 420 --batch-size 1000 --compression-codec 1 We are getting the b