Hi Gyula,
I did not meant to say the MetricsReporter code should stay with the metric
storage, e.g. Kafka. I am trying to argue that metric reporter is a plugin
less related to Flink, but more related to a specific user environment. If
we include such plugins into Flink, as time goes, the user env
So this probably doesn't belong in this thread, but here goes:
When you think of the metric system as source and reporters and sinks,
one has to consider what he source emits:
Either:
a) events for added/removed metrics
b) periodically emit the values of all metrics, with the plethora of
addi
@Becket , Yun:
Regarding the core/ecosystem project:
I don't completely agree with your arguments regarding why this should be
an external ecosystem project instead of part of the Flink repo.
A metric connector is relevant for the Flink users, not the metric store.
Metric storage systems don't car
@Bowen I can see where you're coming from, but I don't think this would
work too well. Your "stream" would have to contain events for
added/removed metrics, but metrics are inherently not Serializable. I
think this would end up being a weird special case.
(Periodically emitting the values of a
Hi,
What still unclear to me so far is - As I don't see any yet., what would be
the fundamental differences between this Kafka reporter and Flink’s
existing Kafka producer?
I’ve been thinking of Flink metrics for a while, and the “metric reporter”
feels a bit redundant to me. As you may already k
Hi all
Glad to see this topic in community.
We at Alibaba also implemented a kafka metrics reporter and extend it to other
message queues like Alibaba cloud log service [1] half a year ago. The reason
why we not launch a similar discussion is that we previously thought we only
provide a way to
Hi Gyula,
Thanks for bringing this up. It is a useful addition to have a Kafka
metrics reporter. I understand that we already have Prometheus and DataDog
reporters in the Flink main repo. However, personally speaking, I would
slightly prefer to have the Kafka metrics reporter as an ecosystem proje
Hi Gyula,
thank you for proposing this. +1 for adding a KafkaMetricsReporter. In
terms of the dependency we could go a similar route as for the "universal"
Flink Kafka Connector which to my knowledge always tracks the latest Kafka
version as of the Flink release and relies on compatibility of the
