Hi All,
We are using prometheus version 2.48.0 and enabled out_of_order_time_window
to 1d.
Promethues A and Prometheus B are in same version
Prometheus A normally remote writes to Prometheus B. Prometheus A receives
data from a fleet of Prometheus Agents .
Prometheus A was done for 45 minutes and once recovered it received all the
data from Prometheus Agent but those metrics didn't reach our Prometheus B
which is our long term storage.
We generated the backfill metrics in Openmetrics format from Prometheus A
for the missing time tried to push it to Prometheus B using the remote
write endpoint using promtool.
It failed with below error.
HTTP 500 - Too old sample on Prometheus A
On Prometheus B we can see the below error.
{"caller":"write_handler.go:76","component":"web","err":"too
oldsample","level":"error","msg":"Error appending remote write"}
Can get any assistance on why this is happening ?
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/8722f180-ef60-4c4f-b3df-afa1fbf14b72n%40googlegroups.com.
