Glad you got it working. For tidiness, you can avoid the DNS:localhost in
the certificate by changing your scrape job, either to
- job_name: prometheus
scheme: https
static_configs:
- targets: ['example.com:9090']
tls_config:
...
(assuming that example.com resolves to an IP address that prometheus can
use to connect to itself); or:
- job_name: prometheus
scheme: https
static_configs:
- targets: ['localhost:9090']
tls_config:
server_name: example.com
...
The latter case still connects to localhost (127.0.0.1), but verifies the
certificate subject is "example.com"
On Wednesday, 25 May 2022 at 04:20:26 UTC+1 [email protected] wrote:
> Thank you for the info, they are very helpful! I get it working basicaly
> following your advice one-by-one.
> I am testing to query from a Prometheus DB, so I am setting up a local
> test environment. I will switch to querying a remote Promethues DB in the
> future.
> For the record, This is what I have changed:
> 1. use https scheme in the config file
> 2. use example.com.crt as ca_file
> 3. removed insecure_skip_verify
> 4. regenerated the cert with SubjectAltName
> openssl req -x509 -newkey rsa:4096 -nodes -keyout example.com.key -out
> example.com.crt -subj /commonName=example.com/ -addext
> "subjectAltName=DNS:example.com, DNS:localhost"
>
> On Tuesday, May 24, 2022 at 7:10:01 PM UTC+8 Brian Candler wrote:
>
>> Please don't paste graphical screenshots: they are hard to read, and it's
>> impossible to copy-paste them to make corrections.
>>
>> First thing is, you're scraping port 9090 but you haven't told it to use
>> HTTPS. You need setting "scheme: https
>> <https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config>"
>>
>> in the scrape job.
>>
>> Secondly, you've set up TLS wrongly, although it may work given that you
>> have "insecure_skip_verify: true".
>>
>> - At the *server* side you need tls_server_config with cert_file and
>> key_file, which is as you have it.
>> - At the *client* side (which in this case is prometheus making an
>> outbound scrape connection to itself), you don't want cert_file or
>> key_file; you need ca_file. This points to the certificate file of the
>> certificate authority which signed the example.com.crt certificate. If
>> this is a self-signed certificate, then this is the same certificate, i.e.
>> "ca_file: example.com.crt"
>>
>> Thirdly, you're connecting to the host using name "localhost", but this
>> will only verify successfully if the certificate contains "localhost" as
>> one of its SubjectAltNames. You should connect using whatever name you
>> signed for the certificate. Or, you can use the "server_name: ..." setting
>> in tls_config to say what name to expect in the certificate presented by
>> the server. Again, "insecure_skip_verify" will probably skip this check.
>>
>> (But of course, really you don't want to use "insecure_skip_verify". Why
>> are you deploying TLS at all, if you're doing it in an insecure way?)
>>
>> Fourthly, you didn't show how you generated the certificates. With
>> modern versions of Go (and hence recent versions of Prometheus), the
>> certificate CommonName is ignored. The server *must* have a certificate
>> with at least one SubjectAltName. So if you followed an out-of-date how-to
>> for signing certificates, you probably made a bad certificate.
>>
>> This is what I use:
>>
>> mkdir /etc/prometheus/ssl
>> cd /etc/prometheus/ssl
>> openssl genpkey -genparam -algorithm EC -pkeyopt ec_paramgen_curve:P-256
>> -out p-256.param
>> openssl req -x509 -newkey ec:p-256.param -keyout prometheus_key.pem -out
>> prometheus_cert.pem -days 29220 -nodes -subj /commonName=*prometheus*/
>> -addext "subjectAltName=DNS:*prometheus*"
>>
>> In "/commonName=*prometheus*/" and "DNS:*prometheus*", replace
>> "prometheus" with the hostname you want in the certificate. "localhost"
>> would work, but apart from self-scraping, normally your clients are
>> connecting to the prometheus server using some real fully-qualified domain
>> name not "localhost", so you should use that FQDN.
>>
>> On Tuesday, 24 May 2022 at 11:33:43 UTC+1 [email protected] wrote:
>>
>>> Hi,
>>>
>>> I don't know how TLS certs work on Windows, but you should at least be
>>> able to see the exact scrape error on the /targets page of your Prometheus
>>> server - what does it say?
>>>
>>> Cheers,
>>> Julius
>>>
>>> On Tue, May 24, 2022 at 11:57 AM Hank Huang <[email protected]> wrote:
>>>
>>>> Hi all!
>>>>
>>>> So I setup Prometheus to monitor itself.
>>>> Now I want to test with https, so I followed the doc and generated
>>>> example.com.crt and example.com.key, and referenced them in the config
>>>> file
>>>> and web config file.
>>>> I also double clicked on the example.com.crt to install the cert onto
>>>> my machine.
>>>> Then I launch the Prometheus with the two config files：
>>>> .\prometheus.exe --config.file=prometheus.yml --web.config.file=web.yml
>>>>
>>>> When I query "up" from Prometheus, it's always 0, the response status
>>>> is 200 though. Also there's a "TLS handshake error" in the console.
>>>> I think maybe it's because I didn't install the cert correctly. Any
>>>> insight is appreciated.
>>>>
>>>> [image: Screenshot 2022-05-24 173210.png]
>>>>
>>>> [image: Screenshot 2022-05-24 173734.png]
>>>>
>>>>
>>>> config file (prometheus.yml):
>>>> [image: Screenshot 2022-05-24 161738.png]
>>>>
>>>> web config file (web.yml):
>>>> [image: Screenshot 2022-05-24 161825.png]
>>>>
>>>>
>>>> syntax wise looks fine:
>>>> [image: Screenshot 2022-05-24 162119.png]
>>>> [image: Screenshot 2022-05-24 162158.png]
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Prometheus Users" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/prometheus-users/51ceb9ba-58e0-4149-b199-e49f21661b1cn%40googlegroups.com
>>>>
>>>> <https://groups.google.com/d/msgid/prometheus-users/51ceb9ba-58e0-4149-b199-e49f21661b1cn%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>>
>>>
>>> --
>>> Julius Volz
>>> PromLabs - promlabs.com
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/1ffb412f-769e-4b6a-b2ce-7b6458468006n%40googlegroups.com.
