awesome, thanks Gabriele
On Wed, Feb 26, 2020 at 1:24 PM Gabriele Paggi wrote:
>
> Hi Richard,
>
> Yes, it's the size in bytes for all log segments for a given
> topic/partition on a given broker, without the index files:
>
> [gpaggi@kafkalog001 ~]$ kafka-log-dirs.sh --bootstrap-server
> $(hostna
Hi Richard,
Yes, it's the size in bytes for all log segments for a given
topic/partition on a given broker, without the index files:
[gpaggi@kafkalog001 ~]$ kafka-log-dirs.sh --bootstrap-server
$(hostname -f):9092 --describe --broker-list 1 --topic-list
access_logs | tail -n+3 | jq '.brokers[].l
Thanks Gabriele, it turned out I didn't have that pattern deployed (facepalm).
After deploying it , worked right away.
Now I'm struggling with understanding the size metric. Do you know if
it's reporting the size (in bytes) of all segments for that
broker/topic/partition?
I'm trying to compare tho
Hi Richard,
The beans path is:
kafka.log:name=Size,partition=,topic=,type=Log
I don't have a jmx_exporter at hand to test it at the moment but I
don't see anything obviously wrong in your config, other than type:
GAUGE missing.
Did you try browsing the beans with jmxterm before configuring the exp
Hi Gabriele,
I'm using Kafka 5.3.1, which is apache kafka 2.3 and I'm using JMX to
retrieve metrics from brokers.
The only metric I saw from "kafka.log" is LogFlushStats, but nothing
about Log.partition
This is the pattern I have deployed for kafka.log metrics, maybe I
need a different for partiti
Hi Richard,
If you are running Kafka > 1.0.0 the information you are looking for
is exposed by the Admin API, describeLogDirs method, or via CLI with
kafka-log-dirs.sh.
The metrics are also exposed by the dropwizard metrics reporter, e.g.
for Graphite:
kafkalog001.kafka.log.Log.partition.49.topic.
Hi List,
I'm trying to find the way to keep track of topics' size (or
partitions) across brokers, but
I didn't found any Kafka metrics for that. I start wonder if that is possible.
I have ideas for a work around (querying log files) but I wonder what
will be the right way or how do you keep monit
Apparently it's documented in the FAQ - but I ignored it since it said
"0.8.0" and I was using 0.8.2.1. After reading all the lengthy forum post
dating to 2013: The problematic code there is in DefaultEventHandler.scala,
but if I'm only using KafkaProducer.java - the java flavor - I won't be
expose
Since the image is now shown, here's a direct link to it:
https://s32.postimg.org/xoet3vu2t/image.png
On Tue, Jul 5, 2016 at 7:01 AM Asaf Mesika wrote:
> As we continue to track down the cause I'm trying to ping back here in
> case someone new might have an answer to the question below?
>
>
> On
As we continue to track down the cause I'm trying to ping back here in case
someone new might have an answer to the question below?
On Thu, Jun 16, 2016 at 12:39 PM Asaf Mesika wrote:
> Hi,
>
> We've noticed that we have some partitions receiving more messages than
> others. What I've done to le
Hi,
We've noticed that we have some partitions receiving more messages than
others. What I've done to learn that is:
* In Kafka Manager, per a given topic, the list of Partition Information is
displayed.
* For each partition there's a column called Latest Offset - which I assume
is the producer of
Was reported by one of the users, so I looked at the code as that was the only
thing I could go by. Trying to get more details
Sent from my iPhone
> On Apr 2, 2016, at 01:07, Ewen Cheslack-Postava wrote:
>
> Oleg,
>
> Normally the number of partitions doesn't change (or infrequently, at
> l
Oleg,
Normally the number of partitions doesn't change (or infrequently, at
least) so regardless of how you got the number of partitions there
shouldn't be an inconsistency. Are you actually seeing an inconsistency
causing this exception? And is the number of partitions not changing? Is it
possibl
Hi
It seems there are several ways to get to the same number in Kafka API.
In Kafka Partitioner which is invoked by KafkaProducer we have this
public int partition(ProducerRecord record, Cluster cluster) {
List partitions =
cluster.partitionsForTopic(record.topic());
int numParti
Yes it is based on the machines capacity.
In practice we try to limit partitions to about 50GB to ensure the data is
evenly spread across machines and that the recovery time of a failure is
minimized.
-Clark
Sent from my iPhone
> On Apr 3, 2015, at 6:17 AM, Nirmal ram wrote:
>
> Hi,
>
> Wh
Hi,
What is the maximum size a partition can have?
Does it depends on the machines storage capacity?
16 matches
Mail list logo
