Hi Brian,
Thanks for the response.
I have configured static and sd both as different job and when I am hitting
SD job, I am getting targets as all the services inside the cluster but
they are throwing 400 or 404 error but when I check the logs, the status is
200.
I am attaching job for your reference. Let me know if I am missing
something. I also tried to check connection by going into blackbox-exporter
pod but it is still throwing same error. I have put all under same
namespace to avoid any communication failure.
- job_name: "blackbox-kubernetes-services"
metrics_path: /probe
params:
module: [http_2xx]
kubernetes_sd_configs:
- role: service
relabel_configs:
# Example relabel to probe only some services that have
"example.io/should_be_probed = true" annotation
# - source_labels:
[__meta_kubernetes_service_annotation_example_io_should_be_probed]
# action: keep
# regex: true
- source_labels: [__address__]
target_label: __param_target
- target_label: __address__
replacement: prometheus-blackbox-exporter:9115
- source_labels: [__param_target]
target_label: instance
- action: labelmap
regex: __meta_kubernetes_service_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
target_label: kubernetes_namespace
- source_labels: [__meta_kubernetes_service_name]
target_label: kubernetes_service_name
On Friday, 17 March 2023 at 21:04:29 UTC+5:30 Brian Candler wrote:
> > When I am using static config then it shows fine but when I am using
> service discovery for status of service or pod some are coming up some are
> down
>
> Then clearly you're not scraping/probing the same endpoints as you did
> with the static config.
>
> The easiest way to check is to go to the Prometheus web interface and look
> at the "Status > Targets" page and see what *actual* targets you're
> probing: first with the static config, and then secondly with the SD config
> (actually you can do both at the same time, but configure them as separate
> jobs so you can distinguish them by the "job" label). The first column
> shows the URL being scraped and the URL params being passed ("module" and
> "target" are the ones that need to be set for blackbox_exporter, of course).
>
> If you can show your static config, and the service discovery config, the
> problem might be clearer. My guess is the SD discovery isn't returning
> data in the form that you expect, or that your rewrite rules aren't working
> in the way that you expect.
>
> On Friday, 17 March 2023 at 15:23:28 UTC dikshant rai wrote:
>
>> Hi,
>> I tried to configure blackbox exporter for checking api status with
>> prometheus on kubernetes. When I am using static config then it shows fine
>> but when I am using service discovery for status of service or pod some are
>> coming up some are down. I used telnet to check connectivity as well that
>> is fine but still don't know where it is failing.
>> Please help.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/e362d6b2-27c5-43a9-a2b2-b7193b12d564n%40googlegroups.com.
