Hi,
I think it's okay to remove the realm-ID from the metric tags and leave
it in traces. So +1 from me on doing this.
High cardinality values are not good for metrics (or metrics systems)
and can easily cause a lot of "interesting situations" in production
systems - things that are hard to figure out (slow downs, heap issues,
...). It's hard to say when a value has a "high enough cardinality" that
it becomes a problem, so I think it's better to remove the realm-ID
value entirely from metrics.
Metrics are great to monitor the overall behavior of a system and a good
start to further investigate potential issues. The actual investigation
however requires way more details, which is where traces come in.
Robert
On 15.05.25 21:30, Alex Dutra wrote:
Hi all,
I would like to suggest removing the "realm_id" metric tag entirely.
My concern is that this tag has the potential for high cardinality, which
is generally considered a bad practice when dealing with metrics. High
cardinality can lead to performance issues and increased memory usage.
Granted, the default realm resolver in Polaris is tailored for just a
handful of realms, but nothing prevents users from declaring hundreds of
realms.
I believe we can still effectively monitor Polaris servers without this
specific tag, since the realm ID is also propagated in traces emitted by
Polaris. Tracing is a much better fit for high-cardinality domains.
I'm open to discussing this further; a potential alternative would be to
introduce a flag to disable this specific metric tag, but I feel like
removing it would be a much cleaner approach.
Let me know your thoughts.
Thanks,
Alex
--
Robert Stupp
@snazy
