Well that's odd. Let's go through some things: * The two metrics have gaps at different points in time, so it can't be an issue with Prometheus not being able to scrape the Pushgateway itself * Other than that, series appearing/disappearing from a scrape endpoint can trigger these kinds of gaps due to staleness handling, but if you're using Add(), no series should disappear, only existing ones should get replaced with new values.
Or are you using Push() somewhere else in your app, or clearing out an entire metrics group in the Pushgateway once in a while? On Tue, Jan 10, 2023 at 7:43 PM 'Achintya Kumar' via Prometheus Users < [email protected]> wrote: > Understood. I used Gatherer() in my code now. But seeing some black spaces > in graph. What can be the issue here? These both are counter metrics. > > [image: Screenshot 2023-01-11 at 12.09.37 AM.png] > [image: Screenshot 2023-01-11 at 12.09.54 AM.png] > On Tuesday, January 10, 2023 at 8:28:35 PM UTC+5:30 [email protected] > wrote: > >> You *will* need to use the Gatherer() method to register some metrics >> with your Pusher, otherwise that explains why nothing is being pushed. See >> this example: >> https://pkg.go.dev/github.com/prometheus/client_golang/prometheus/push#example-Pusher.Add >> >> On Tue, Jan 10, 2023 at 3:38 PM 'Achintya Kumar' via Prometheus Users < >> [email protected]> wrote: >> >>> No, not using Gatherer() method. Here's my pastebin link >>> <https://pastebin.com/qnSmhCBa> for the code. >>> Also, I've created different maps to store the previously generated >>> collectors because earlier I was getting the "Duplicate metric collector >>> registration attempted". This was solved using maps. >>> >>> On Tuesday, January 10, 2023 at 7:51:56 PM UTC+5:30 >>> [email protected] wrote: >>> >>>> Are you adding any metrics registry to the Pusher, using the Gatherer() >>>> method? >>>> https://pkg.go.dev/github.com/prometheus/client_golang/prometheus/push#Pusher.Gatherer >>>> >>>> On Tue, Jan 10, 2023 at 12:06 PM 'Achintya Kumar' via Prometheus Users < >>>> [email protected]> wrote: >>>> >>>>> Hey Julius, >>>>> Thank you for your reply. Understood the difference between Add and >>>>> Push. >>>>> Regarding the goroutine thing, I was trying something like this >>>>> >>>>> pushGateWayClient := push.New(endpoint, jobName) >>>>> ticker := time.NewTicker(10 * time.Second) >>>>> go func() { >>>>> for { >>>>> select { >>>>> case <-ticker.C: >>>>> if err := pushGateWayClient.Add(); err != nil { >>>>> log.Error("Could not push to push gateway ", >>>>> zap.Error(err)) >>>>> } >>>>> } >>>>> } >>>>> }() >>>>> >>>>> But this isn't working. I can not see the metrics when verifying on >>>>> grafana. Can you help here? >>>>> >>>>> On Tuesday, January 10, 2023 at 3:21:42 PM UTC+5:30 >>>>> [email protected] wrote: >>>>> >>>>>> Hi Achintya, >>>>>> >>>>>> On Tue, Jan 10, 2023 at 9:29 AM 'Achintya Kumar' via Prometheus Users >>>>>> <[email protected]> wrote: >>>>>> >>>>>>> Hey Guys, >>>>>>> I am using github.com/prometheus/client_golang/prometheus/push in >>>>>>> my Golang code to push metrics to a push gateway. >>>>>>> But I wanted to understand the core difference between Add and Push. >>>>>>> I am currently using Add to push the metrics but still have some >>>>>>> confusions >>>>>>> regarding which previous metrics will it >>>>>> >>>>>> >>>>>> Add() will replace only any metrics with the same metric name and >>>>>> labels (so the same time series identity) within a pushed group, if >>>>>> metrics >>>>>> with the same name and labels have been pushed before. In contrast, >>>>>> Push() >>>>>> completely removes all previous metrics in a pushed group and then only >>>>>> adds the newly pushed ones. So you can use Add() instead if you are not >>>>>> sending *all* possible metrics to a group every time, and you don't >>>>>> want the old ones to disappear. >>>>>> >>>>>> >>>>>>> replace? Also, I want to use the pusher.Add function to be run after >>>>>>> every 10 seconds rather than pushing metrics right away. How can I >>>>>>> achieve >>>>>>> that? The prometheus/push package doesn't have this kind of >>>>>>> functionality. >>>>>>> >>>>>> >>>>>> If you really want that you'll have to implement that yourself by >>>>>> just having a goroutine running in the background that pushes your >>>>>> metrics >>>>>> every 10 seconds. Just be aware of the general limitations of what the >>>>>> Pushgateway should be used for at >>>>>> https://prometheus.io/docs/practices/pushing/. Most of the time it's >>>>>> just used to push metrics once, at the end of a batch job run, but in >>>>>> some >>>>>> cases it can also make sense to push metrics on a more ongoing basis, >>>>>> like >>>>>> if you want to report on the progress of an ongoing batch job while it's >>>>>> running. >>>>>> >>>>>> Julius >>>>>> >>>>>> >>>>>>> Thank you. >>>>>>> This message and its attachments are confidential (or legally >>>>>>> privileged) information and are meant solely for the addressee of such >>>>>>> message. Any unauthorized use of the message / its attachments is >>>>>>> strictly >>>>>>> prohibited. >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Prometheus Users" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/prometheus-users/a1df943c-1146-4c39-9983-f077952d1f28n%40googlegroups.com >>>>>>> <https://groups.google.com/d/msgid/prometheus-users/a1df943c-1146-4c39-9983-f077952d1f28n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Julius Volz >>>>>> PromLabs - promlabs.com >>>>>> >>>>> >>>>> This message and its attachments are confidential (or legally >>>>> privileged) information and are meant solely for the addressee of such >>>>> message. Any unauthorized use of the message / its attachments is strictly >>>>> prohibited. >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Prometheus Users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/prometheus-users/d86405bd-2fa0-4582-b1e2-7da4ab47bb3bn%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/prometheus-users/d86405bd-2fa0-4582-b1e2-7da4ab47bb3bn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> >>>> >>>> -- >>>> Julius Volz >>>> PromLabs - promlabs.com >>>> >>> >>> This message and its attachments are confidential (or legally >>> privileged) information and are meant solely for the addressee of such >>> message. Any unauthorized use of the message / its attachments is strictly >>> prohibited. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Prometheus Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/prometheus-users/cef48dee-05c4-45d5-91fb-921e51c631b9n%40googlegroups.com >>> <https://groups.google.com/d/msgid/prometheus-users/cef48dee-05c4-45d5-91fb-921e51c631b9n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> >> -- >> Julius Volz >> PromLabs - promlabs.com >> > > This message and its attachments are confidential (or legally privileged) > information and are meant solely for the addressee of such message. Any > unauthorized use of the message / its attachments is strictly prohibited. > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/e918fe7b-c698-4233-9eab-43fc55b524f8n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/e918fe7b-c698-4233-9eab-43fc55b524f8n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- Julius Volz PromLabs - promlabs.com -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAObpH5yWNT9rJR2gz66q6unbpMaoMBJg0%3DqH1TW%3Di6LDKE%2BiTA%40mail.gmail.com.
