Hi This is my first time asking a question in the group, and I may not be very clear on some of the rules, but I will do my best to describe my question clearly. Below is the description of my issue:
- I deployed* Prometheus, Node Exporter, Grafana, Alertmanager, Blackbox Exporter, *and* Webhook-WeChat *on the server with IP *10.1.1.161* using Docker, and all the ports are set to their default values. These services are running normally. I want to use *Blackbox Exporter* to monitor whether the *Tableau *service installed on a Windows system with IP *10.101.1.112* returns a status code of 200. I am monitoring port 8850 (the Tableau port).I used* curl https://10.101.1.112:8850* on 10.1.1.161 and received the following response: ------------------------------------------------------------------------------------------------------------------------------------------ curl: (60) Peer's certificate issuer has been marked as not trusted by the user. More details here: http://curl.haxx.se/docs/sslcerts.html curl performs SSL certificate verification by default, using a "bundle" of Certificate Authority (CA) public keys (CA certs). If the default bundle file isn't adequate, you can specify an alternate file using the --cacert option. ------------------------------------------------------------------------------------------------------------------------------------------- - when I use "*curl -k https://10.101.1.112:8850*";, I can successfully retrieve the response body. Therefore, I added insecure_skip_verify: true in the *'config/blackbox.yml*' file to skip certificate verification. - I restarted Prometheus, Blackbox Exporter, and the Blackbox container, but after the restart, the Webhook-WeChat service triggered an alert: ' *https://10.101.1.112:8850* responded with code 0' instead of 200. However, when I access *https://10.101.1.112:8850* in the browser, it works, though it warns that the connection is not secure. At the same time, when I use the browser to access *http://10.1.1.161:9115/probemodule=http_2xx&target=https://10.101.1.112:8850*, it returns a status in a format like this, which seems to indicate that the request was unsuccessful(for example：probe_success 0,probe_http_status_code 0): ------------------------------------------------------------------------------------------------------------------------------------------- # HELP probe_dns_lookup_time_seconds Returns the time taken for probe dns lookup in seconds # TYPE probe_dns_lookup_time_seconds gauge probe_dns_lookup_time_seconds 1.6356e-05 # HELP probe_duration_seconds Returns how long the probe took to complete in seconds # TYPE probe_duration_seconds gauge probe_duration_seconds 0.022416452 # HELP probe_failed_due_to_regex Indicates if probe failed due to regex # TYPE probe_failed_due_to_regex gauge probe_failed_due_to_regex 0 # HELP probe_http_content_length Length of http content response # TYPE probe_http_content_length gauge probe_http_content_length 0 # HELP probe_http_duration_seconds Duration of http request by phase, summed over all redirects # TYPE probe_http_duration_seconds gauge probe_http_duration_seconds{phase="connect"} 0 probe_http_duration_seconds{phase="processing"} 0 probe_http_duration_seconds{phase="resolve"} 1.6356e-05 probe_http_duration_seconds{phase="tls"} 0 probe_http_duration_seconds{phase="transfer"} 0 # HELP probe_http_redirects The number of redirects # TYPE probe_http_redirects gauge probe_http_redirects 0 # HELP probe_http_ssl Indicates if SSL was used for the final redirect # TYPE probe_http_ssl gauge probe_http_ssl 0 # HELP probe_http_status_code Response HTTP status code # TYPE probe_http_status_code gauge probe_http_status_code 0 # HELP probe_http_uncompressed_body_length Length of uncompressed response body # TYPE probe_http_uncompressed_body_length gauge probe_http_uncompressed_body_length 0 # HELP probe_http_version Returns the version of HTTP of the probe response # TYPE probe_http_version gauge probe_http_version 0 # HELP probe_ip_addr_hash Specifies the hash of IP address. It's useful to detect if the IP address changes. # TYPE probe_ip_addr_hash gauge probe_ip_addr_hash 3.959139489e+09 # HELP probe_ip_protocol Specifies whether probe ip protocol is IP4 or IP6 # TYPE probe_ip_protocol gauge probe_ip_protocol 4 # HELP probe_success Displays whether or not the probe was a success # TYPE probe_success gauge probe_success 0 ------------------------------------------------------------------------------------------------------------------------------------------ Here is my Prometheus prometheus.yml configuration: --- global: scrape_interval: 15s evaluation_interval: 15s # config alertmanagers alerting: alertmanagers: - static_configs: - targets: - 10.1.1.161:9093 rule_files: - "/etc/prometheus/rules/*.yml" scrape_configs: # Prometheus port 9090 - job_name: 'prometheus' static_configs: - targets: ['10.1.1.161:9090'] labels: instance: 'prometheus' - job_name: 'tableau_blackbox' metrics_path: /probe params: module: [http_2xx] static_configs: - targets: - https://10.101.1.112:8850 relabel_configs: - source_labels: [__address__] target_label: __param_target - source_labels: [__param_target] target_label: instance - target_label: __address__ # This is Blackbox Exporter replacement: 10.1.1.161:9115 - source_labels: [__param_target] target_label: service replacement: 'new pro environment' # https://10.101.1.112:8850 regex: 'https://10\.101\.1\.112:8850' --- Here is the Prometheus rule configuration: --- groups: - name: tableau_alerts rules: # 1. HTTP/HTTPS - alert: Tableau service problem expr: probe_http_status_code{job="tableau_blackbox",service="new pro environment"} != 200 for: 10s labels: severity: critical annotations: summary: "tableau code exception" description: "Target service {{ $labels.instance }} response code {{ $value }}，should be 200!" --- Here is the Blackbox Exporter blackbox.yml configuration: --- modules: http_2xx: prober: http timeout: 5s http: valid_http_versions: ["HTTP/1.1", "HTTP/2.0"] method: GET preferred_ip_protocol: "ip4" ip_protocol_fallback: false no_follow_redirects: false fail_if_ssl: false fail_if_not_ssl: false tls_config: insecure_skip_verify: true --- Here is my Docker startup command: *Prometheus：* *docker run -d --name prometheus -p 9090:9090 -v /data/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml \ -v /data/prometheus/rules/:/etc/prometheus/rules/ prom/prometheus* *blackbox explorter:* *docker run -d -p 9115:9115 -v /data/blackbox_exporter/config/:/config/ --name blackbox_exporter quay.io/prometheus/blackbox-exporter * -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/prometheus-users/e64ca74d-169c-4454-bcd7-87b67a5f3272n%40googlegroups.com.
