On Sat, May 28, 2022 at 11:48 AM Arrigo Marchiori <ard...@yahoo.it.invalid>
wrote:
> Hello Damjan,
>
> On Sat, May 28, 2022 at 11:12:45AM +0200, Damjan Jovanovic wrote:
>
> > On Sat, May 28, 2022 at 9:20 AM Arrigo Marchiori <ard...@yahoo.it.invalid
> >
> > wrote:
> >
> > > Hello Damjan, all,
> > >
> > > On Fri, May 27, 2022 at 11:27:11PM +0200, Arrigo Marchiori wrote:
> > >
> > > > Hello,
> > > >
> > > > On Fri, May 27, 2022 at 09:46:51PM +0200, Arrigo Marchiori wrote:
> > > >
> > > > > Hello Damjan,
> > > > >
> > > > > On Sun, May 22, 2022 at 06:10:46PM +0200, Damjan Jovanovic wrote:
> > > > >
> > > > > > On Sun, May 22, 2022 at 2:43 PM Arrigo Marchiori
> > > <ard...@yahoo.it.invalid>
> > > > > > wrote:
> > > > > >
> > > > > > > Hello Damjan, all,
> > > > > > >
> > > > > > > On Tue, Apr 26, 2022 at 07:56:22PM +0200, Damjan Jovanovic
> wrote:
> > > > > > >
> > > > > > > > On Mon, Nov 15, 2021 at 9:57 PM Jim Jagielski <
> j...@jagunet.com>
> > > wrote:
> > > > > > > >
> > > > > > > > > I'm gonna look into the serf->(lib)curl option... Since we
> > > don't use
> > > > > > > any
> > > > > > > > > of the fancy features of serf, I'm thinking that the easy
> > > option might
> > > > > > > be
> > > > > > > > > best
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hi
> > > > > > > >
> > > > > > > > I've ported our WebDAV content provider module from Serf to
> Curl.
> > > > > > >
> > > > > > > I just enhanced the error reporting a bit; I am finding a
> problem
> > > > > > > under Linux and I do not really know how to assess it.
> > > > > > >
> > > > > > > The problem: if we build AOO on CentOS (that is our reference
> > > > > > > platform) then Curl will look for CA certificates in
> > > > > > > /etc/pki/tls/certs/ca-bundle.crt
> > > > > > >
> > > > > > > This will fail on openSUSE and probably on Ubuntu as well.
> > > > > > >
> > > > > > > It seems that the above path is set at configure time and
> embedded
> > > > > > > into Curl's code as #define macros.
> > > > > > >
> > > > > > > Is there an ``official'' way to assess this? Like, can we
> depend on
> > > > > > > NSS' certificate store as you wrote (quoted below)?
> > > > > > >
> > > > > >
> > > > > > Curl/OpenSSL have an enormous number of options and I am pretty
> sure
> > > it can
> > > > > > be fixed, but first I need to understand where and how it's
> failing.
> > > > > >
> > > > > > We currently allow it to run with the default CA certificate
> path, do
> > > > > > pre-verification on the server's certificate using those CA
> > > certificates,
> > > > > > then call our SSL_VERIFY_PEER function where we override the
> > > verification
> > > > > > result with the certificates from NSS.
> > > > >
> > > > > Apparently, it is failing before calling our SSL_VERIFY_PEER
> function.
> > > > >
> > > > > > If it's failing before reaching our SSL_VERIFY_PEER function, we
> > > should be
> > > > > > able to use Curl's CURLOPT_CAINFO or CURLOPT_CAINFO_BLOB
> functions
> > > to set a
> > > > > > custom CA certificate path (or in-memory buffer), maybe even an
> empty
> > > > > > buffer, so that it proceeds further. ("man CURLOPT_CAINFO", "man
> > > > > > CURLOPT_CAINFO_BLOB", or "man curl_easy_setopt" and read under
> the
> > > "SSL and
> > > > > > SECURITY OPTIONS" section.)
> > > > >
> > > > > So we would need to hard-code and try all possible paths to the CA
> > > > > bundle on Unix systems?
> > > > >
> > > > > > With the CURLOPT_CAINFO_BLOB option it might even be possible to
> > > skip the
> > > > > > custom certificate verification we do later, and pre-populate
> > > Curl/OpenSSL
> > > > > > with NSS certificates from the beginning, I just don't know
> enough
> > > about
> > > > > > NSS to rely on that (eg. if you are using a cryptographic device
> or
> > > smart
> > > > > > card in NSS, how does that work?). If that option is ok, then we
> > > might not
> > > > > > even need the NSS libraries: recent versions of NSS store all the
> > > > > > certificates in an SQLite database, which can be accessed with
> > > SQLite APIs
> > > > > > directly, no need to build with or ship the NSS libraries at all.
> > > > >
> > > > > If I understood correctly [1], a NSS-linked Curl would query NSS by
> > > > > itself... are we not in this condition?
> > > [...]
> > > > 1: https://curl.se/libcurl/c/CURLOPT_CAINFO.html
> > >
> > > Apparently, we are not. If we force CURLOPT_CAINFO and CURLOPT_CAPATH
> > > to be NULL, we get a bit further but eventually Curl aborts because
> > > validateServerX509Certificate fails.
> > >
> > > Here is the log for checking AOO updates (please excuse the long
> > > lines). I added an extra bit of logging when entering and leaving
> > > OPENSSL_ValidateServerCertificate, that is ``our SSL_VERIFY_PEER
> > > function''.
> > >
> > >
> ----8<--------8<--------8<--------8<--------8<--------8<--------8<---------
> > >
> > > 1 9 2022-05-28 07:13:52.61 CurlSession::CurlSession
> with
> > > URL
> https://ooo-updates.apache.org/aoonext/check.Update?pkgfmt=installed
> > > 2 9 2022-05-28 07:13:52.61 Not using a proxy server
> > > 3 9 2022-05-28 07:13:52.61 GET line 1093
> > > 4 9 2022-05-28 07:13:52.61 [CurlINFO ] Trying
> > > 151.101.2.132:443...
> > > 5 9 2022-05-28 07:13:52.63 [CurlINFO ] Connected to
> > > ooo-updates.apache.org (151.101.2.132) port 443 (#0)
> > > 6 9 2022-05-28 07:13:52.63 [CurlINFO ] ALPN, offering
> > > http/1.1
> > > 7 9 2022-05-28 07:13:52.63 [CurlINFO ] Cipher
> selection:
> > > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> > > 8 9 2022-05-28 07:13:52.63 [CurlINFO ] TLSv1.2 (OUT),
> TLS
> > > handshake, Client hello (1):
> > > 9 9 2022-05-28 07:13:52.64 [CurlINFO ] TLSv1.2 (IN),
> TLS
> > > handshake, Server hello (2):
> > > 10 9 2022-05-28 07:13:52.64 [CurlINFO ] TLSv1.2 (IN),
> TLS
> > > handshake, Certificate (11):
> > > --here we are inside OPENSSL_ValidateServerCertificate
> > > 11 9 2022-05-28 07:13:52.64
> validateServerX509Certificate()
> > > returning 0 at depth 2
> > > --here we are leaving OPENSSL_ValidateServerCertificate
> > > 12 9 2022-05-28 07:13:52.64 [CurlINFO ] TLSv1.2 (OUT),
> TLS
> > > alert, unknown CA (560):
> > > 13 9 2022-05-28 07:13:52.64 [CurlINFO ] SSL certificate
> > > problem: unable to get local issuer certificate
> > > 14 9 2022-05-28 07:13:52.64 [CurlINFO ] Closing
> connection
> > > 0
> > > 15 9 2022-05-28 07:13:52.64 Curl request failed with
> > > CURLcode 60
> > > 16 9 2022-05-28 07:13:52.64 CurlSession::~CurlSession:
> > > closed curl session
> > >
> > >
> ----8<--------8<--------8<--------8<--------8<--------8<--------8<---------
> > >
> > > I hope this helps.
> > >
> > >
> > I think the problem there is NSS's validation.
> >
> > In CurlSession::validateServerX509Certificate(), try change the "if (
> depth
> > == 0 )" to "if (1)".
>
> After this edit, the connection is established. The log is below.
>
That's good :).
>
> But NSS was also used before, with Serf... it stopped working when the
> bundled CA certificates started to be ``too obsolete''. After we
> updated the bundled NSS, then https connections, such as the check for
> updates, started to work again.
>
We don't need NSS specifically. From Curl, we need OpenSSL, because without
it, we are unable to ask the user to confirm self-signed certificates
during TLS negotiations [1]. Upstream OpenSSL however doesn't ship any CA
certificates, so the WebDAV provider gets them from our
com.sun.star.security.CertificateContainer, which comes from WinCrypt on
Windows (also called "mscrypt" by main/xmlsecurity/source/xmlsec/mscrypt)
and NSS elsewhere.
Therefore that update check should not have been failing on Windows. Only
if your Windows is out of date, should WinCrypt CA certificates fail.
It is also particularly strange for NSS certificates to be out of date. The
NSS certificate database that our
com.sun.star.security.CertificateContainer uses comes from a profile
(getMozillaCurrentProfile() in
main/xmlsecurity/source/nss/nssinitializer.cxx) which (AFAIK) is in the
user's home directory, and shared with and updated by Firefox or
Thunderbird. Only if you don't update AOO, and don't run Firefox, and don't
run Thunderbird, can your NSS certificates get out of date.
>
> Please also bear in mind that I am now testing a build from a _CentOS
> Docker container_. My computer is not running CentOS, but rather
> OpenSUSE. The binary is run on ``native'' OpenSUSE. If I recall
> correctly, when I build on the same native OpenSUSE, the resulting AOO
> binary connects properly to https web sites.
>
> For this reason, I was wondering if this problem is not just about
> hardcoded paths to CA certificates.
>
Since Curl's OpenSSL CA path is hardcoded to a non-portable value at
compile time, my proposed approach here disables OpenSSL's certificates and
only uses the ones from our com.sun.star.security.CertificateContainer
instead. (And, who knows, it may be helpful to make another
com.sun.star.security.CertificateContainer that gets certificates from
guessed system paths for all of AOO, not just WebDAV.)
Is there something else you propose instead?
Regards
Damjan
[1] But is Curl's inability to use custom certificate verification
functions (which can ask the user to confirm problematic certificates) with
NSS a limitation of Curl, or a limitation of NSS? Firefox uses NSS, and it
certainly asks the user to confirm expired and self-signed certificates.
How does Firefox do that, by disconnecting, asking the user, and
reconnecting if confirmed, or the Curl/OpenSSL way, by keeping the TCP
connection open during the user interaction and then resuming TLS
negotiations if confirmed?
