Hi Damian
I use Webdav almost on a daily basis. Looking forward to testing a new build under Linux as soon as you push it to 42X Can you share your working binaries for Windows? Thanks! Pedro > On 04/26/2022 6:56 PM Damjan Jovanovic <dam...@apache.org> wrote: > > > > > > > On Mon, Nov 15, 2021 at 9:57 PM Jim Jagielski <j...@jagunet.com > <mailto:j...@jagunet.com>> wrote: > > I'm gonna look into the serf->(lib)curl option... Since we don't use any of > > the fancy features of serf, I'm thinking that the easy option might be best > > > > > > Hi > > > > I've ported our WebDAV content provider module from Serf to Curl. > > > > While it ended well, and several other bugs were found and fixed, it > > definitely wasn't the "easy option" Jim ;). Starting with conservative > > changes, it ended up needing total restructuring, and became more of a > > rewrite. The crashes were frequent and hung connections many, and I had to > > read up on the HTTP protocol, and read Curl and Serf's source code, but > > eventually I prevailed, and a clean elegant stable Curl WebDAV module > > emerged. > > > > The huge patch is attached for anyone wishing to review and test. Unless > > there are major objections, I'll push it in a couple of days. > > > > STATUS > > > > It builds and works well on FreeBSD and Windows. > > > > Most of the code was reused, and all the operations and semantics > > previously present with Serf, should have been preserved. > > > > Browsing WebDAV files and directories, loading files, overwriting them > > ("Save"), creating them ("Save As"), renaming and deleting them, all works. > > > > HTTP and HTTPS proxies work. Unlike Serf, Curl could also support SOCKS > > proxies (with minimal further changes), but AOO lacks that setting in the > > proxy UI and configuration. > > > > Authentication works, both to the destination server and to the proxy > > server. I've successfully tested Basic and Digest authentication. Curl > > supports every authentication method Serf does and more. > > > > HTTPS works, with a custom certificate verification function, using our own > > certificate store from NSS and its API (like the Serf code used). A bug was > > discovered (which is in the Serf implementation too) where self-signed > > certificates were being unconditionally rejected; apparently NSS wants to > > see that a copy of that certificate in its certificate chain parameter > > too. Now they work, and the user gets prompted to allow access. > > > > > > HTTPS and authentication can be used together on the same connection and > > work well, both bringing up their UI dialogs as needed. > > > > A bug was fixed where when username and password were both present in the > > URL (dav://user:pass@host/path), the code was trying to split them at the > > "@" instead of ":". > > > > Unnecessary base64 encoding and decoding was removed, when importing the > > TLS connection's certificates into our XSecurityEnvironment. They now come > > in directly as ASN.1 DER, and still work. > > > > The code was greatly restructured and cleaned up, as Curl's API is > > synchronous and blocking, with parameters set in advance instead of through > > many callbacks, which has allowed using short clear methods, and clean > > separation between the session and request classes. The WebDAV content > > provider has shrunk from 35 to 21 C++ files, 43 to 29 header files, and > > 19129 to 15991 lines of code. With WebDAV methods centralized and > > refactored into only 10-20 lines of code each, instead of scattered across > > 4 files, it is much more understandable and maintainable now. > > > > Curl is vastly more portable than Serf. We should build easily now even on > > OS/2. We can remain with existing build tools instead of needing scons or > > cmake just to build Serf. > > > > 3 now unused dependencies were removed: apr, apr-util, and serf. Serf isn't > > so bad. APR's pool idea is an ingenious way of doing resource management in > > C. However Curl has excellent documentation, guides, examples, and detailed > > explanations and even example code for each setting, while Serf has no > > documentation. Serf might be worth it in a project that already uses APR a > > lot, but we don't. > > > > Instead of the historical, crippled forms of logging like OSL_TRACE(), > > which don't appear in release builds, I've made it use the newer > > com.sun.star.logging UNO component (wrapped in comphelper::EventLogger), > > which was inspired by java.util.logging, with configurable verbosity > > levels, handlers (file and console) and output formats (plain, csv), and > > importantly, which produces output in release builds too. I've also made it > > so that on LogLevel::FINEST, Curl's verbose mode is enabled and Curl's > > debug output is also logged through us, with descriptions of what Curl is > > doing, and logs of all HTTP traffic including headers and bodies, before > > encryption and after decryption in the case of HTTPS, giving us tremendous > > detail that can be used for troubleshooting problems. > > > > CURL CHANGED TO USE OPENSSL AND ZLIB > > > > Curl only supports the custom TLS certificate verification function we use > > (the CURLOPT_SSL_CTX_FUNCTION option) when built with OpenSSL, wolfSSL or > > mbedTLS providers. We currently use schannel on Windows instead, which had > > to be changed. I also made it use zlib, which generally helps, and WebDAV > > uses XML which is very verbose and benefits from compression. On other OSes > > with system curl, it is now checked for its SSL provider, and configure > > fails if it isn't OpenSSL. > > > > The new WebDAV module successfully builds and runs with both OpenSSL 1.0.2 > > or 1.1.1. However 1 function was renamed between those versions, so the > > OpenSSL version at runtime probably has to match the one used at compile > > time (although building with 1.0.2 headers might allow running with 1.1.1 - > > not tested). > > > > (After completing development and testing, it dawned on me there is a > > completely different way to do the certification verification, which should > > allow other SSL providers to be used and might be better in various ways. > > See later.) > > > > We currently build zlib as a static library only, and on Windows its C > > runtime library is linked statically (_MT), which can't be mixed with > > curl's dynamic linking of the runtime library (_MD). Thus curl was made to > > link it statically too. Most if not all of our modules link the runtime > > library statically too (which begs the question, why do we ship the msvcr* > > redistributables to users at all then?). > > > > ISSUES > > > > The file open dialog (Ctrl+O) can hang for several minutes when first > > connecting to a server. This is not new - it happens with Serf as well. > > This appears to be caused by autocompletion in the file dialog. When typing > > in a URL like "davs://127.0.0.1 <http://127.0.0.1>", the WebDAV content > > provider is first called with a partial "https://1";, before the rest of the > > URL is entered. That "1" is (somehow) treated as IP address "0.0.0.1", and > > a TCP connection to 0.0.0.1 is started. Only after several minutes, when > > that connection times out and fails, does the content provider get another > > request with the complete URL, which succeeds. The distinctly unpleasant > > wait, is luckily only present that the first time that server is used, as > > caching remembers the URL even across AOO restarts, so the "1" is > > automatically expanded to the full URL and the content provider never sees > > "https://1"; again. > > > > You can only enter credentials for the HTTP(S) proxy or the destination, > > never both, as the credential manager caches credentials per URL, not per > > host/realm, so while you are prompted for credentials for both, and Curl is > > told about both, the destination credentials overwrite the proxy > > credentials in the cache, so one Curl request works but future Curl > > requests use the wrong credentials to the proxy. This doesn't matter, as > > AOO doesn't support passwords for proxy servers anyway, and Serf has the > > same issue. > > > > Damjan > > > > -- > > > > P.S. APACHE 2.4 SETUP FOR TESTING > > > > Put some files in /var/tmp/dav/files, then configure Apache HTTPD's > > httpd.conf with something along these lines: > > > > Listen 127.0.0.1:8080 <http://127.0.0.1:8080> > > LoadModule dav_module libexec/apache24/mod_dav.so > > LoadModule dav_fs_module libexec/apache24/mod_dav_fs.so > > > > <VirtualHost 127.0.0.1:8080 <http://127.0.0.1:8080>> > > DocumentRoot "/var/tmp/dav" > > ErrorLog "/var/tmp/dav/errors.txt" > > TransferLog "/var/tmp/dav/access.txt" > > DavLockDB "/var/tmp/dav/DavLock" > > > > # If using TLS, generate these self-signed certificates too: > > # SSLEngine on > > # SSLCertificateFile "/var/tmp/dav/cert.pem" > > # SSLCertificateKeyFile "/var/tmp/dav/key.pem" > > > > <Directory "/var/tmp/dav/files"> > > AllowOverride none > > Require valid-user > > # Require all granted > > > > AuthType Basic > > AuthName "WebDAV" > > AuthBasicProvider file > > AuthUserFile "/var/tmp/dav/passwords" > > > > Dav on > > </Directory> > > </VirtualHost> > > > > Enter your username and password with: > > htpasswd -c /var/tmp/dav/passwords username > > > > Then in the file open dialog, enter "dav://127.0.0.1:8080/files/ > > <http://127.0.0.1:8080/files/>" (or use scheme davs:// instead, if using > > HTTPS). > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@openoffice.apache.org > > For additional commands, e-mail: dev-h...@openoffice.apache.org
