Re: Kafka Producer Errors

your `jvm.log` and update the `request.timeout.ms`. The same property is applicable to producer, consumer and broker. Increase the config only for the KafkaProducer. -- Kamal On Sat, Aug 12, 2017 at 12:19 AM, Saladi Naidu < naidusp2...@yahoo.com.invalid> wrote: > Looks like some of

Re: Kafka Producer Errors

and batch.size - < 0 Any clue on why linger.ms 0 or btach.size 0 is not taken into consideration ???   Naidu Saladi On Thursday, August 10, 2017 1:57 PM, Saladi Naidu wrote: We have recently upgraded our Kafka cluster from 0.8.2 to 0.10.1. As part of the upgrade our clients upgrad

Kafka Producer Errors

We have recently upgraded our Kafka cluster from 0.8.2 to 0.10.1. As part of the upgrade our clients upgraded the client libraries as well. After the upgrade we are facing following issue org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for OM-LOCAL-DC3-REQUISITION-SUBMISSIO

FetchRequestPurgatory

We have Kafka clusters in three datacenters with Mirror Maker replicating across three data centers. We are monitoring JMX metrics of our clusters. I have question regarding FetchRequestPurgatory (kafka.server:type=FetchRequestPurgatory,name=PurgatorySize), description on Wiki says "Number of re

Re: Optimal number of partitions for topic

In general partitions are to improve throughput by parallelism. From your explanation below yes partitions are written to different physical locations but still append only. With write ahead buffering and append only writes, having partitions still will increase throughput. Below is an excellent

Kafka Monitoring using JMX

We are planning to monitor health of our Kafka environment using JMX. I have looked at below links to find what is available via https://cwiki.apache.org/confluence/display/KAFKA/Available+Metrics  https://cwiki.apache.org/confluence/display/KAFKA/JMX+Reporters why some of the kafka objects ("ka

No broker partitions consumed by consumer thread

We are running 0.8.1.1, am experiencing following issue, any clue I have 7 topics in my Kafka cluster of 5 nodes. I pushed  1 million messages to each topic using perf producer tool (no compression, no acknowledgement) . I saw old messages about a bug in 0.8 Consumer code and it is fixed but I am

Fw: How to measure performance of Mirror Maker

Any suggestions on how to measure throughput of the Mirror Maker Naidu Saladi - Forwarded Message - From: Saladi Naidu To: "users@kafka.apache.org" Sent: Monday, March 16, 2015 10:31 PM Subject: How to measure performance of Mirror Maker We have three Kafk

How to measure performance of Mirror Maker

We have three Kafka clusters deployed in 3 DC's, each one having their own topics. We are using Mirror Maker to keep all the three clusters up to date with continuous  replication using Mirror Maker. We used Perf producer and Perf consumer to conduct basic testing, all seems to work great. Our 2

Out of Disk Space - Infinite loop

We have 3 DC's and created 5 node Kafka cluster in each DC, connected these 3 DC's using Mirror Maker for replication. We were conducting performance testing using Kafka Producer Performance tool to load 100 million rows into 7 topics. We expected that data will be loaded evenly across 7 topics