Hi Vladimir, Personally, even testing "local" REST catalogs, I'm setting up SSL certificates with a local CA, etc. It's not very painful.
That said, I got your point, and I think we can update https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/HTTPClient.java to add a flag to disable SSL certificate checks. I would propose to add: private static final String REST_SSL_DISABLE_CERTIFICATE_CHECK = "rest.ssl.disable.cert.check"; and used this for HTTP5 client setup. Regards JB On Wed, Nov 13, 2024 at 1:53 PM Vladimir Ozerov <voze...@querifylabs.com> wrote: > > Hi, > > Currently, RESTCatalog can work either over HTTP or HTTPS. In the latter > case, Iceberg always performs a fully-fledged check of server certificate. > While it is expected in production environments, this is not necessarily ok > for nob-prod on-premises REST catalog deployments (DEV, QA, etc). > > Consider a data platform team that would like to evaluate a migration to a > REST catalog from, say, HMS. It might be very convenient to issue a > self-signed certificate and use it when communicating with the catalog. > However, the current implementation of RESTCatalog client doesn't allow this, > as there is no hook to override the behavior of instantiated HTTP5 client. > Note that the advice to switch to HTTP for non-prod deployment is not valid > in this case, because during testing teams would like to have their > deployments as close as possible to production, looking for some unexpected > issues (e.g., unexpected performance drop when enabling SSL, etc). > > Many vendors allow to disable SSL certificate checks in some cases. E.g., AWS > S3 APIs allows this, Trino allows, etc. It might be convenient to add a > similar capability to Iceberg's HTTPClient as well. E.g., this might be a > property "rest.client.insecure-ssl" passed to the client. > > What do you think about this? Apologize if it was already discussed > elsewhere, I couldn't find any relevant discussions. > > Regards, > -- > Vladimir Ozerov > Founder > querifylabs.com
