[
https://issues.apache.org/jira/browse/FLINK-38269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yaroslav updated FLINK-38269:
-----------------------------
Description:
Currently the address with which Job Manager's REST endpoint advertises itself
(to, for example, YARN Resource Manager) is defined in the following way (see
[RestServerEndpoint.java|https://github.com/apache/flink/blob/release-2.0.0/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java#L298-L304]):
{code:java}
final InetSocketAddress bindAddress = (InetSocketAddress)
serverChannel.localAddress();
final String advertisedAddress;
if (bindAddress.getAddress().isAnyLocalAddress()) {
advertisedAddress = this.restAddress;
} else {
advertisedAddress = bindAddress.getAddress().getHostAddress();
}
{code}
That is, {{rest.address}} value is used if {{rest.bind-address}} is {{0.0.0.0}}
wildcard, and {{{}rest.bind-address{}}}' IP address form is used otherwise.
This causes a problem when TLS is enabled and Job Manager's certificate for
external communication (REST) contains a hostname instead of an IP address. If
we bind to a specific IP address, the advertised address will be in a form of
IP address and a TLS client which only trusts URLs with a hostname would fail
to verify Job Manager's certificate. YARN Resource Manager Proxy is an example
of such a client, which fails with:
{code:java}
javax.net.ssl.SSLPeerUnverifiedException: Certificate for <192.168.33.11>
doesn't match any of the subject alternative names: []{code}
We can workaround it by setting {{rest.bind-address}} to {{0.0.0.0}} and
specifying a desired hostname to advertise in {{{}rest.address{}}}, however
this can have a negative impact on security in multi-homed environments due to
excessive network exposure.
In the related [mailing
list|https://lists.apache.org/thread/d8s0wx4z0pyo5gcyoxbh1d9rwd1mpozj] (for
some reason the link only shows 2 messages, in fact there are 10 now, please
try to find the whole thread in
[here|https://lists.apache.org/list.html?u...@flink.apache.org]), [~gsomogyi]
explained that using {{getHostName()}} instead of {{getHostAddress()}} in the
code block above is not an option because this way we'll introduce new security
& performance issues due to rDNS.
However, instead we could just add a new dedicated configuration option for Job
Manager's REST endpoint advertised address. The default behavior (when no value
specified) would remain unchanged, but if the option is configured we'll use
provided advertised address. This way:
# We can solve the TLS problem by setting this option to a hostname
# We don't bind to {{0.0.0.0}} and thus avoid excessive network exposure
# We also avoid rDNS-related issues
# We don't change the default behavior
This Jira is an improvement request to add such an option.
was:
Currently the address with which Job Manager's REST endpoint advertises itself
(to, for example, YARN Resource Manager) is defined in the following way (see
[RestServerEndpoint.java|https://github.com/apache/flink/blob/release-2.0.0/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java#L298-L304]):
{code:java}
final InetSocketAddress bindAddress = (InetSocketAddress)
serverChannel.localAddress();
final String advertisedAddress;
if (bindAddress.getAddress().isAnyLocalAddress()) {
advertisedAddress = this.restAddress;
} else {
advertisedAddress = bindAddress.getAddress().getHostAddress();
}
{code}
That is, {{rest.address}} value is used if {{rest.bind-address}} is {{0.0.0.0}}
wildcard, and {{{}rest.bind-address{}}}' IP address form is used otherwise.
This causes a problem when TLS is enabled and Job Manager's certificate for
external communication (REST) contains a hostname instead of an IP address. If
we bind to a specific IP address, the advertised address will be in a form of
IP address and a TLS client which only trusts URLs with a hostname would fail
to verify Job Manager's certificate. YARN Resource Manager Proxy is an example
of such a client, which fails with:
{code:java}
javax.net.ssl.SSLPeerUnverifiedException: Certificate for <192.168.33.11>
doesn't match any of the subject alternative names: []{code}
We can workaround it by setting {{rest.bind-address}} to {{0.0.0.0}} and
specifying a desired hostname to advertise in {{{}rest.address{}}}, however
this can have a negative impact on security in multi-homed environments due to
excessive network exposure.
In the related [mailing
list|https://lists.apache.org/thread/d8s0wx4z0pyo5gcyoxbh1d9rwd1mpozj] (for
some reason the link only shows 2 messages, in fact there are 10 now, please
try to find the whole thread in
[here|https://lists.apache.org/list.html?u...@flink.apache.org]), [~gsomogyi]
explained that using {{getHostName()}} instead of {{getHostAddress()}} in the
code block above is not an option because this way we'll introduce new security
& performance issues due to rDNS.
However, instead we could just add a new dedicated configuration option for Job
Manager's REST endpoint advertised address. The default behavior (when no value
specified) would remain unchanged, but if the option is configured we'll use
provided advertised address. This way:
# We can solve the TLS problem by setting this option to a hostname
# We don't bind to {{0.0.0.0 }}and thus avoid excessive network exposure
# We also avoid rDNS-related issues
# We don't change the default behavior
This Jira is an improvement request to add such an option.
> Make Job Manager's REST endpoint advertised address configurable
> ----------------------------------------------------------------
>
> Key: FLINK-38269
> URL: https://issues.apache.org/jira/browse/FLINK-38269
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Configuration, Runtime / REST, Runtime / Web
> Frontend
> Affects Versions: 2.0.0
> Reporter: Yaroslav
> Priority: Minor
>
> Currently the address with which Job Manager's REST endpoint advertises
> itself (to, for example, YARN Resource Manager) is defined in the following
> way (see
> [RestServerEndpoint.java|https://github.com/apache/flink/blob/release-2.0.0/flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestServerEndpoint.java#L298-L304]):
> {code:java}
> final InetSocketAddress bindAddress = (InetSocketAddress)
> serverChannel.localAddress();
> final String advertisedAddress;
> if (bindAddress.getAddress().isAnyLocalAddress()) {
> advertisedAddress = this.restAddress;
> } else {
> advertisedAddress = bindAddress.getAddress().getHostAddress();
> }
> {code}
> That is, {{rest.address}} value is used if {{rest.bind-address}} is
> {{0.0.0.0}} wildcard, and {{{}rest.bind-address{}}}' IP address form is used
> otherwise.
> This causes a problem when TLS is enabled and Job Manager's certificate for
> external communication (REST) contains a hostname instead of an IP address.
> If we bind to a specific IP address, the advertised address will be in a form
> of IP address and a TLS client which only trusts URLs with a hostname would
> fail to verify Job Manager's certificate. YARN Resource Manager Proxy is an
> example of such a client, which fails with:
> {code:java}
> javax.net.ssl.SSLPeerUnverifiedException: Certificate for <192.168.33.11>
> doesn't match any of the subject alternative names: []{code}
> We can workaround it by setting {{rest.bind-address}} to {{0.0.0.0}} and
> specifying a desired hostname to advertise in {{{}rest.address{}}}, however
> this can have a negative impact on security in multi-homed environments due
> to excessive network exposure.
> In the related [mailing
> list|https://lists.apache.org/thread/d8s0wx4z0pyo5gcyoxbh1d9rwd1mpozj] (for
> some reason the link only shows 2 messages, in fact there are 10 now, please
> try to find the whole thread in
> [here|https://lists.apache.org/list.html?u...@flink.apache.org]), [~gsomogyi]
> explained that using {{getHostName()}} instead of {{getHostAddress()}} in the
> code block above is not an option because this way we'll introduce new
> security & performance issues due to rDNS.
> However, instead we could just add a new dedicated configuration option for
> Job Manager's REST endpoint advertised address. The default behavior (when no
> value specified) would remain unchanged, but if the option is configured
> we'll use provided advertised address. This way:
> # We can solve the TLS problem by setting this option to a hostname
> # We don't bind to {{0.0.0.0}} and thus avoid excessive network exposure
> # We also avoid rDNS-related issues
> # We don't change the default behavior
> This Jira is an improvement request to add such an option.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
