I already have Prometheus/Grafana for monitoring for most of the metrics. But I
was looking for metrics which shows disk usage per broker/topic.
-Original Message-
From: George
Sent: Thursday, January 9, 2020 10:49 PM
To: users@kafka.apache.org
Subject: Re: JMX Metrics to display disk
I pulled the 2.4 branch from github—version 2.4.1-snapshot of pom.xml. When I
packaged it and ran it, I still found the same problem. But the data and offset
could be synchronized to the target cluster.
| |
xinzhuxiansheng
|
|
xinzhuxiansh...@126.com
|
签名由网易邮箱大师定制
On 1/7/2020 04:26，Ryanne Dolan
Why not rather use Prometheus with node-app module deployed on all involved
hosts, Producers, consumers, brokers etc,
You can also use prometheus to integrate with Kafka via JMX to expose it's
kafka metrics,
Display all via a Grafana dashboard
G
On Fri, Jan 10, 2020 at 6:21 AM Peter Bukowinski
Kafka does not report collect/report on topic data filesystem usage. I used
this collectd project to help me collect the topic usage data and export it to
graphite:
https://github.com/HubSpot/collectd-kafka-disk/blob/master/README.md
The plugin collects the size of each topic-partition director
Is there any metrics available to view disk usage in each broker/topic via JMX
exporter? If yes, could you please send me details? Thanks
+1, it is preferable to run MM2 at the target/destination cluster.
Basically, producer lag is more problematic than consumer lag, so you want
the producer as close to the target cluster as possible.
Also take a look at the "--clusters" option of the connect-mirror-maker.sh
command, which lets you
> Does the producer have a ( limit or time value ) for when to drop messages
IIUC:
https://kafka.apache.org/documentation/#producerconfigs
mostly buffer.memory and request.timeout.ms
> Also, can the producer indicate to its source this event is happing.
Someone more familiar with MM2 would have to
Hello
Does the producer have a ( limit or time value ) for when to drop messages,
when the QOS is low.
Also, can the producer indicate to its source this event is happing.
-Original Message-
From: Andrew Otto
Sent: Thursday, January 9, 2020 8:32 AM
To: users@kafka.apache.org
Subject: R
Hi Peter,
My understanding here comes from MirrorMaker 1, but I believe it holds for
MM2 (someone correct me if I am wrong!)
For the most part, if you have no latency or connectivity issues, running
MM at the source will be fine. However, the failure scenario is different
if something goes wrong.
Hey,
I am thinking about where (well in which AWS region) should I run MM2.
I might be wrong, but as I know it is better to run it close to the
destination cluster.
But for other reasons, it would be much easier for me to run it at the
source.
So is it still advised to run MM2 at the destination?
Peter, that's right. So long as ReplicationPolicy is implemented with
proper semantics (i.e. the methods do what they say they should do) any
naming convention will work. You can also use something like double
underscore "__" as a separator with DefaultReplicationPolicy -- it doesn't
need to be a s
Hi Ryanne,
Am I right that as far as I implement ReplicationPolicy properly, those
features you just mentioned will work fine?
Asking because we already use dot(.) underscore(_) and even hyphen(-)
characters in not replicated topics :D , so it seems to be that we will
need a more advanced renamin
Hey,
yes, at first it specified client.id in the config (I was not sure if which
one is needed):
source->destination.producer.client.id = "mm2"
source->destination.consumer.client.id = "mm2"
source.producer.client.id = "mm2"
source.consumer.client.id = "mm2"
destination.producer.client.id = "
13 matches
Mail list logo
