Hi Brian, Thanks for commands for troubleshooting. Finally, single quotes are not added in some of the targets. I have added it and works fine.
These commands will be helpful in troubleshooting in future. Regards, Venkatraman N On Mon, Jan 29, 2024 at 7:52 PM 'Brian Candler' via Prometheus Users < [email protected]> wrote: > I can't see the image link you posted: it gives a 404. > > There's probably some config error in your prometheus.yml or your targets > file, but seeing only snippets means I can't spot them. > > Try these commands (adjust paths as necessary): > > /path/to/promtool check service-discovery /path/to/prometheus.yml > blackbox-fundconnect_retail > /path/to/promtool check config /path/to/prometheus.yml > > And what does the query > up{job="blackbox-fundconnect_retail"} > show? > > Apart from that, I don't think there's much more I can do to help you. > There's a problem with the configs on your system that you'll need to > diagnose locally. > > On Monday 29 January 2024 at 13:57:19 UTC Venkatraman Natarajan wrote: > >> Hi Brian, >> >> Thanks for the response. >> >> *Are you saying that job blackbox-fundconnect_retail does not appear >> there, but blackbox-fundconnect_retail_03 does? * >> >> Yes, Correct. We have 100 jobs in prometheus. Now we have added 5 jobs >> additionally, in that it is showing only 4 jobs. >> >> blackbox-fundconnect_retail_03, >> blackbox-fundconnect_retail_04 >> blackbox-fundconnect_retail_05 >> blackbox-fundconnect_retail_06 >> blackbox-fundconnect_retail - This one is not showing in prometheus UI. >> >> *Is it possible that blackbox-fundconnect_retail has zero targets >> configured?* >> >> No, It has 48 targets configured. >> >> Note: We need to configure different scrape intervals for those jobs >> that's why configuring different jobs instead of a single job. >> >> [image: image.png] >> >> >> Thanks, >> Venkatraman N >> >> On Thursday, January 25, 2024 at 7:13:23 PM UTC+5:30 Brian Candler wrote: >> >>> Sorry, I don't know what you mean by "not showing all the jobs". You >>> have only shown a small portion of the targets page. Are you saying that >>> job blackbox-fundconnect_retail does not appear there, >>> but blackbox-fundconnect_retail_03 does? Is it possible >>> that blackbox-fundconnect_retail has zero targets configured? >>> >>> The PromQL query "up" will show you all the targets. "count by (job) >>> (up)" will show you the jobs, with the number of targets for each. Those >>> won't show a job with zero targets though. >>> >>> Note that in the above case, both scrape jobs look to be identical, in >>> which case you could have one job: >>> >>> file_sd_configs: >>> - files: >>> - './dynamic/blackbox/blackbox_retail_FundConnect_targets.yml' >>> - './dynamic/blackbox/blackbox_retail_FundConnectHealth-03_targets.yml' >>> >>> (You can use target labels to distinguish them, if you wish) >>> >>> > Do we have limitations in prometheus jobs.? >>> >>> No. >>> >>> > Prometheus version: prometheus:v2.27.1 >>> >>> That's pretty old (May 2021). Current LTS version is v2.45.2 >>> >>> On Thursday 25 January 2024 at 13:11:27 UTC Venkatraman Natarajan wrote: >>> >>>> Hi Team, >>>> >>>> We have 100+ prometheus jobs in prometheus UI but not showing all the >>>> jobs. >>>> >>>> The below are sample 2 jobs. In this, first job is not showing in UI >>>> but second job metrics showing fine. >>>> - job_name: 'blackbox-fundconnect_retail' >>>> scrape_interval: 1020s >>>> metrics_path: /probe >>>> honor_timestamps: true >>>> params: >>>> module: [http_2xx] >>>> file_sd_configs: >>>> - files: >>>> ['./dynamic/blackbox/blackbox_retail_FundConnect_targets.yml'] >>>> relabel_configs: >>>> - source_labels: [__address__] >>>> separator: ; >>>> regex: (.*) >>>> target_label: __param_target >>>> replacement: $1 >>>> action: replace >>>> - source_labels: [__param_target] >>>> separator: ; >>>> regex: (.*) >>>> target_label: instance >>>> replacement: $1 >>>> action: replace >>>> - separator: ; >>>> regex: (.*) >>>> target_label: __address__ >>>> replacement: {{ XXX }}:9122 >>>> action: replace >>>> >>>> - job_name: 'blackbox-fundconnect_retail_03' >>>> scrape_interval: 1020s >>>> metrics_path: /probe >>>> honor_timestamps: true >>>> params: >>>> module: [http_2xx] >>>> file_sd_configs: >>>> - files: >>>> ['./dynamic/blackbox/blackbox_retail_FundConnectHealth-03_targets.yml'] >>>> relabel_configs: >>>> - source_labels: [__address__] >>>> separator: ; >>>> regex: (.*) >>>> target_label: __param_target >>>> replacement: $1 >>>> action: replace >>>> - source_labels: [__param_target] >>>> separator: ; >>>> regex: (.*) >>>> target_label: instance >>>> replacement: $1 >>>> action: replace >>>> - separator: ; >>>> regex: (.*) >>>> target_label: __address__ >>>> replacement: {{ XXXXX}}:9122 >>>> action: replace >>>> >>>> Do we have limitations in prometheus jobs.? >>>> >>>> I would like to add more prometheus jobs to scrape the metrics. >>>> >>>> Please find attached screenshot which shows jobs in prometheus UI >>>> >>>> Prometheus version: prometheus:v2.27.1 >>>> >>>> Could you please help me on this? >>>> >>>> Thanks, >>>> Venkatraman N >>>> >>> -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/c5aa0bb1-e86e-4842-90d2-e1aab4b76d18n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/c5aa0bb1-e86e-4842-90d2-e1aab4b76d18n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CANSgTEZUc9Yb8N93%2Bj1hKiVFc9iJ1njqbLGO5LQFWG5wBfRHAg%40mail.gmail.com.
