Hi Jean-Baptiste,

Thank you. I am currently prototyping this.
I'd like to ask for advice on how to properly test that feature.
Essentially, I need to set up a server, install a self-signed certificate
there, and test the behavior. Does a test similar to TestRESTCatalog with a
custom TLS-enabled Jetty server seem like the way to go? Or are there some
existing HTTPS tests?

Regards,
Vladimir

On Wed, Nov 13, 2024 at 4:11 PM Jean-Baptiste Onofré <j...@nanthrax.net>
wrote:

> Hi Vladimir,
>
> Personally, even testing "local" REST catalogs, I'm setting up SSL
> certificates with a local CA, etc. It's not very painful.
>
> That said, I got your point, and I think we can update
>
> https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/HTTPClient.java
> to add a flag to disable SSL certificate checks.
>
> I would propose to add:
>
> private static final String REST_SSL_DISABLE_CERTIFICATE_CHECK =
> "rest.ssl.disable.cert.check";
>
> and used this for HTTP5 client setup.
>
> Regards
> JB
>
> On Wed, Nov 13, 2024 at 1:53 PM Vladimir Ozerov <voze...@querifylabs.com>
> wrote:
> >
> > Hi,
> >
> > Currently, RESTCatalog can work either over HTTP or HTTPS. In the latter
> case, Iceberg always performs a fully-fledged check of server certificate.
> While it is expected in production environments, this is not necessarily ok
> for nob-prod on-premises REST catalog deployments (DEV, QA, etc).
> >
> > Consider a data platform team that would like to evaluate a migration to
> a REST catalog from, say, HMS. It might be very convenient to issue a
> self-signed certificate and use it when communicating with the catalog.
> However, the current implementation of RESTCatalog client doesn't allow
> this, as there is no hook to override the behavior of instantiated HTTP5
> client. Note that the advice to switch to HTTP for non-prod deployment is
> not valid in this case, because during testing teams would like to have
> their deployments as close as possible to production, looking for some
> unexpected issues (e.g., unexpected performance drop when enabling SSL,
> etc).
> >
> > Many vendors allow to disable SSL certificate checks in some cases.
> E.g., AWS S3 APIs allows this, Trino allows, etc. It might be convenient to
> add a similar capability to Iceberg's HTTPClient as well. E.g., this might
> be a property "rest.client.insecure-ssl" passed to the client.
> >
> > What do you think about this? Apologize if it was already discussed
> elsewhere, I couldn't find any relevant discussions.
> >
> > Regards,
> > --
> > Vladimir Ozerov
> > Founder
> > querifylabs.com
>


-- 
*Vladimir Ozerov*
Founder
querifylabs.com

Reply via email to