Federation is a bit of a neglected feature. The Thanos project is rather more popular as a way to aggregate data from multiple Prometheus.
(Other projects also exist which can let you centralise metrics storage) Bryan On Thursday, 16 November 2023 at 15:51:12 UTC [email protected] wrote: > I take the silence as a "no, it is not possible to connect recording rules > with federation requests (or vice versa)" ...? > > On Tuesday, October 31, 2023 at 5:54:05 PM UTC+1 [email protected] > wrote: > >> *Is it possible to include an aggregation formula in a federation query?* >> This would avoid creating and storing aggregated metrics on the >> lower-level Prometheus, and also remove the delay between aggregation and >> federation. >> >> *Or, can a recording rule fetch metrics directly from another Prometheus >> server?* >> This would avoid the delay between federation and execution of the >> recording rule. >> >> We've got hundreds of pods, delivering millions of metrics. We plan to >> partition our pods and deploy one Prometheus per partition. A top-level >> Prometheus will then offer globally aggregated metrics. >> >> How should we set this up? My current assumption is based on aggregation >> recording rules within each partition, then again at the top-level to get >> the global aggregation. This seems both complicated and a waste of >> resources, plus also introduces delays since recording rules and federation >> cannot be synced to each other. >> >> To minimize delay, each partition-level Prometheus needs to aggregate as >> often as possible to offer "fresh" metrics to federation requests. Then the >> top level Prometheus also needs to federate these aggregated metrics as >> often as possible to offer "fresh" values at the global level. Doing things >> as often as possible "just in case" seems wasteful, which is why I am >> asking if this is the right approach for us. >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/6ec7e309-1e6f-4700-97d8-496283d74fb9n%40googlegroups.com.
