Hi all, after some time we updated KIP-302 to reuse the config key introduced by KIP-235, with a different value to avoid conflicts between the two. Also clarified the use of multiple IPs only of the same type (IPv4/IPv6).
We'd appreciate a further review and discussion. Thanks! Edo On Fri, 25 May 2018 at 12:36, Edoardo Comar <edoco...@gmail.com> wrote: > Hi Jonathan, > I'm ok with an expandable enum for the config that could be extended > in the future. > It is marginally better than multiple potentially conflicting config > entries. > > Though as I think the change for KIP-302 is independent from KIP-235 > and they do not conflict, > when we'll look back at it post 2.0 we may see if it is more valuable > to shoehorn its config in an expanded enum or not > > thanks, > Edo > > On 24 May 2018 at 16:50, Skrzypek, Jonathan <jonathan.skrzy...@gs.com> > wrote: > > Hi, > > > > As Rajini suggested in the thread for KIP 235 (attached), we could try > to have an enum that would drive what does the client expands/resolves. > > > > I suggest a client config called client.dns.lookup with different values > possible : > > > > - no : no dns lookup > > - hostnames.only : perform dns lookup on both bootstrap.servers and > advertised listeners > > - canonical.hostnames.only : perform dns lookup on both > bootstrap.servers and advertised listeners > > - bootstrap.hostnames.only : perform dns lookup on bootstrap.servers > list and expand it > > - bootstrap.canonical.hostnames.only : perform dns lookup on > bootstrap.servers list and expand it > > - advertised.listeners.hostnames.only : perform dns lookup on advertised > listeners > > - advertised.listeners.canonical.hostnames.only : perform dns lookup on > advertised listeners > > > > I realize this is a bit heavy but this gives users the ability to pick > and choose. > > I didn't include a setting to mix hostnames and canonical hostnames as > I'm not sure there would be a valid use case. > > > > Alternatively, to have less possible values, we could have 2 parameters : > > > > - dns.lookup.type with values : hostname / canonical.host.name > > - dns.lookup.behaviour : bootstrap.servers, advertised.listeners, both > > > > Thoughts ? > > > > Jonathan Skrzypek > > > > > > -----Original Message----- > > From: Edoardo Comar [mailto:edoco...@gmail.com] > > Sent: 17 May 2018 23:50 > > To: dev@kafka.apache.org > > Subject: Re: [DISCUSS] KIP-302 - Enable Kafka clients to use all DNS > resolved IP addresses > > > > Hi Jonathan, > > > >> A solution might be to expose to users the choice of using hostname or > canonical host name on both sides. > >> Say having one setting that collapses functionalities from both KIPs > (bootstrap expansion + advertised lookup) > >> and an additional parameter that defines how the resolution is > performed, using getCanonicalHostName() or not. > > > > thanks sounds to me *less* simple than independent config options, sorry. > > > > I would like to say once again that by itself KIP-302 only speeds up > > the client behavior that can happen anyway when the client restarts > > multiple times, > > as every time there is no guarantee that - in presence of multiple A > > DNS records - the same IP is returned. Attempting to use additiona IPs > > if the first fail just makes client recovery faster. > > > > cheers > > Edo > > > > On 17 May 2018 at 12:12, Skrzypek, Jonathan <jonathan.skrzy...@gs.com> > wrote: > >> Yes, makes sense. > >> You mentioned multiple times you see no overlap and no issue with your > KIP, and that they solve different use cases. > >> > >> Appreciate you have an existing use case that would work, but we need > to make sure this isn't confusing to users and that any combination will > always work, across security protocols. > >> > >> A solution might be to expose to users the choice of using hostname or > canonical host name on both sides. > >> Say having one setting that collapses functionalities from both KIPs > (bootstrap expansion + advertised lookup) and an additional parameter that > defines how the resolution is performed, using getCanonicalHostName() or > not. > >> > >> Maybe that gives less flexibility as users wouldn't be able to decide > to only perform DNS lookup on bootstrap.servers or on advertised listeners. > >> But this would ensure consistency so that a user can decide to use > cnames or not (depending on their certificates and Kerberos principals in > their environment) and it would work. > >> > >> Jonathan Skrzypek > >> > >> -----Original Message----- > >> From: Edoardo Comar [mailto:edoco...@gmail.com] > >> Sent: 16 May 2018 21:59 > >> To: dev@kafka.apache.org > >> Subject: Re: [DISCUSS] KIP-302 - Enable Kafka clients to use all DNS > resolved IP addresses > >> > >> Hi Jonathan, > >> I am afraid that may not work for everybody. > >> > >> It would not work for us. > >> With our current DNS, my Kafka clients are perfectly happy to use any > IPs - > >> DNS has multiple A records for the 'myhostname.mydomain' used for > >> bootstrap and advertised listeners. > >> The hosts all serve TLS certificates that include > >> 'myhostname.mydomain' and the clients are happy. > >> > >> However, applying getCanonicalHostName to those IPs would return > >> hostnames that would not match the TLS certificates. > >> > >> So once again I believe your solution and ours solve different use > cases. > >> > >> cheers > >> Edo > >> > >> On 16 May 2018 at 18:29, Skrzypek, Jonathan <jonathan.skrzy...@gs.com> > wrote: > >>> I think there are combinations that will break SASL and SSL auth. > >>> Could the trick be to have a single parameter that triggers dns > resolve both for bootstrap and advertised listeners, both using > getCanonicalHostName() ? > >>> > >>> Jonathan Skrzypek > >>> > >>> -----Original Message----- > >>> From: Edoardo Comar [mailto:edoco...@gmail.com] > >>> Sent: 16 May 2018 17:03 > >>> To: dev@kafka.apache.org > >>> Subject: Re: [DISCUSS] KIP-302 - Enable Kafka clients to use all DNS > resolved IP addresses > >>> > >>> Hi Rajini, > >>> > >>> In your example KIP-302 would attempt to connect to the first address > >>> returned, let's say > >>> > >>> www.apache.org/195.154.151.36 > >>> > >>> then, only if that fails, will in turn try the remaining: > >>> > >>> www.apache.org/40.79.78.1 > >>> www.apache.org/140.211.11.105 > >>> www.apache.org/2001:bc8:2142:300:0:0:0:0 > >>> > >>> You're right to say that we expect certificates served by those > >>> endpoints to be valid for "www.apache.org" > >>> > >>> Without KIP-302, only one would be attempted. > >>> Which is the first one, that can change every time > >>> (typically changes on every Java process restart, > >>> but may change also any time InetAddress.getAllByName it's invoked > >>> depending on the caching). > >>> > >>> The behavioral change that KIP-302 may introduce is that in the > example above, > >>> also an IPv6 connection may be attempted after some IPv4 ones. > >>> > >>> InetAddress.getAllByName() implementation uses a system property > >>> "java.net.preferIPv6Addresses" > >>> to decide which type of address to return first (default is still IPv4 > >>> in java 10) > >>> > >>> We will amend the KIP and PR so that the loop only uses IPs of the > >>> same type as the first one returned. > >>> > >>> A part from the above, KIP 302 does not seem to change any existing > >>> client behaviour, as any one of multiple IP addresses (of a given > >>> v4/v6 type) can currently be picked. > >>> We're happy however to keep the looping behavior optional with the > >>> discussed config property, disabled by default. > >>> > >>> As for KIP-235 that may introduce new hostnames in the bootstrap list > >>> (the current PR rewrites the bootstrap list) > >>> and we fail to see the conflict with KIP-302, whatever the set of > >>> configs chosen. > >>> > >>> We'd be happy to try understand what we are missing in a KIP call :-) > >>> > >>> cheers > >>> Edo > >>> > >>> On 15 May 2018 at 16:58, Rajini Sivaram <rajinisiva...@gmail.com> > wrote: > >>>> Hi Edo, > >>>> > >>>> I agree that KIP-235 and KIP-302 address different scenarios. And I > agree > >>>> that each one is not sufficient in itself to address both the > scenarios. > >>>> But I also think that they conflict and hence they need to be looked > at > >>>> together and perhaps use a single config. > >>>> > >>>> As an example: > >>>> > >>>> If I run: > >>>> > >>>> for (InetAddress address : InetAddress.getAllByName("www.apache.org")) > { > >>>> System.out.printf("HostName %s canonicalHostName %s IP %s\n", > >>>> address.getHostName(), address.getCanonicalHostName(), > >>>> address.getHostAddress()); > >>>> } > >>>> > >>>> I get: > >>>> > >>>> HostName www.apache.org canonicalHostName tlp-eu-west.apache.org IP > >>>> 195.154.151.36 > >>>> HostName www.apache.org canonicalHostName 40.79.78.1 IP 40.79.78.1 > >>>> HostName www.apache.org canonicalHostName themis.apache.org IP > >>>> 140.211.11.105 > >>>> HostName www.apache.org canonicalHostName 2001:bc8:2142:300:0:0:0:0 > IP > >>>> 2001:bc8:2142:300:0:0:0:0 > >>>> > >>>> > >>>> If www.apache.org is used as a bootstrap address, KIP-302 would > connect to ( > >>>> www.apache.org/195.154.151.36 and www.apache.org/140.211.11.105) > while > >>>> KIP-235 would connect to (tlp-eu-west.apache.org/195.154.151.3. and > >>>> themis.apache.org/140.211.11.105). This is a significant difference > not > >>>> just for Kerberos, but for any secure environment where hostname is > >>>> verified to prevent man-in-the-middle attacks. In your case, I > presume you > >>>> would have SSL certificates with the equivalent of www.apache.org on > both > >>>> the load balancers. In Jonathan's case, I presume he has Kerberos > >>>> principals for the equivalent of tlp-eu-west.apache.org and > >>>> themis.apache.org. We would want to support both scenarios > regardless of > >>>> the security protocol, just need to come up with configuration > options that > >>>> don't conflict. > >>>> > >>>> > >>>> On Mon, May 14, 2018 at 5:24 PM, Edoardo Comar <edoco...@gmail.com> > wrote: > >>>> > >>>>> Thanks Rajini > >>>>> > >>>>> I still don't see the overlap between the two KIPS > >>>>> > >>>>> KIP-235 allows an expansion of hostnames on the bootstrap list. > >>>>> > >>>>> KIP-302 allows alternative IPs to be used to attempt a connection > >>>>> (either at bootstrap and when processing the MetadataResponse) to a > >>>>> given hostname. > >>>>> > >>>>> A use case would be that of active/passive LB fronting the brokers. > >>>>> > >>>>> Arguably, if Java honored the DNS-set TTL, and the TTL was low and on > >>>>> subsequent requests, the order of IPs returned by the > >>>>> InetAddress.getAllByName() was random, we may not need such an > >>>>> enhancement. > >>>>> In practice, a Java client can get stuck on a "bad" IP forever if it > >>>>> only relies on the first IP returned. > >>>>> > >>>>> HTH, > >>>>> Edo > >>>>> > >>>>> On 14 May 2018 at 16:23, Rajini Sivaram <rajinisiva...@gmail.com> > wrote: > >>>>> > Hi Edo, > >>>>> > > >>>>> > Thanks for the KIP. I think it will be good to include a diagram > to make > >>>>> it > >>>>> > easier to distinguish this scenario from that of KIP-235 without > reading > >>>>> > the PR. > >>>>> > > >>>>> > It may be worth considering if KIP-235 and this KIP could use a > single > >>>>> > config name with different values instead of two boolean configs: > >>>>> > > >>>>> > bootstrap.reverse.dns.lookup = true/false > >>>>> > enable.all.dns.ips = true/false > >>>>> > > >>>>> > Not all values of (bootstrap.reverse.dns.lookup, > enable.all.dns.ips) seem > >>>>> > to make sense. And not all scenarios are handled. Even if we use > multiple > >>>>> > configs, it seems to me that we may want to name them differently. > >>>>> > > >>>>> > The possible combinations are: > >>>>> > > >>>>> > 1) Bootstrap > >>>>> > > >>>>> > a) No lookup > >>>>> > b) Use all DNS entries with host name > >>>>> > c) Use all DNS entries with canonical host name > >>>>> > > >>>>> > 2) Advertised listeners > >>>>> > > >>>>> > a) No lookup > >>>>> > b) Use all DNS entries with host name > >>>>> > c) Use all DNS entries with canonical host name > >>>>> > > >>>>> > The combinations that are enabled by the two boolean configs ( > >>>>> > bootstrap.reverse.dns.lookup, enable.all.dns.ips) are: > >>>>> > > >>>>> > - (false, false) => (1a, 2a) > >>>>> > - (true, false) => (1c, 2a) > >>>>> > - (false, true) => (1b, 2b) > >>>>> > - (true, true) => (??, 2b) > >>>>> > > >>>>> > It will be good if we can clearly identify which combinations we > want to > >>>>> > support and the scenarios where they may be useful. Perhaps (1a, > 2a), > >>>>> (1c, > >>>>> > 2a), (1b, 2b) and (1c, 2c) are useful? > >>>>> > > >>>>> > > >>>>> > On Mon, May 14, 2018 at 2:58 PM, Skrzypek, Jonathan < > >>>>> > jonathan.skrzy...@gs.com> wrote: > >>>>> > > >>>>> >> Ah, apologies didn't see there was already a decent amount of > discussion > >>>>> >> on this in the PR. > >>>>> >> > >>>>> >> This kind of sounds related to the environment you're running to > me. > >>>>> >> What is the rationale behind using the advertised listeners to do > your > >>>>> >> load balancing advertisement rather than a top level alias that > has > >>>>> >> everything ? > >>>>> >> > >>>>> >> It sounds like in your case there is a mismatch between > >>>>> bootstrap.servers > >>>>> >> and advertised.listeners, and you want advertised.listeners to > take > >>>>> >> precedence and have the client iterate over what is returned by > the > >>>>> broker. > >>>>> >> So the extra parameter doesn't only have to do with DNS but it's > also > >>>>> >> appending from the broker, maybe the parameter name should > reflect this > >>>>> ? > >>>>> >> > >>>>> >> Jonathan Skrzypek > >>>>> >> > >>>>> >> > >>>>> >> -----Original Message----- > >>>>> >> From: Skrzypek, Jonathan [Tech] > >>>>> >> Sent: 14 May 2018 14:46 > >>>>> >> To: dev@kafka.apache.org > >>>>> >> Subject: RE: [DISCUSS] KIP-302 - Enable Kafka clients to use all > DNS > >>>>> >> resolved IP addresses > >>>>> >> > >>>>> >> Hi, > >>>>> >> > >>>>> >> I see you noted the similarities with KIP-235. > >>>>> >> But KIP-235 might also solve what this KIP is trying to achieve. > >>>>> >> > >>>>> >> When parsing bootstrap.servers, KIP-235 has the client add all > >>>>> underlying > >>>>> >> hostnames and IPs. > >>>>> >> And this happens before hitting the NetworkClient. > >>>>> >> > >>>>> >> So to me the client will try every single endpoint behind any > >>>>> >> bootstrap.servers record. > >>>>> >> > >>>>> >> See > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_kafka_pull_4485_commits_24757eb7b0&d=DwIBaQ&c=7563p3e2zaQw0AB1wrFVgyagb2IE5rTZOYPxLxfZlX4&r=nNmJlu1rR_QFAPdxGlafmDu9_r6eaCbPOM0NM1EHo-E&m=_ud9m_JZJ87C7eGsKcmzgJgDpNQDIIv5R4i_7VlhkLc&s=TqaiA9uW_myYO6FN-gKPfPlioxZR6DhnlBTpEj5M2aQ&e= > >>>>> >> > 6bcf8c7d7649c85232c52b5d54f0e4#diff-89ef153462e64c250a21bd324ae1a851 > >>>>> >> which calls getAllByName like you suggested > >>>>> >> > >>>>> >> Jonathan Skrzypek > >>>>> >> > >>>>> >> > >>>>> >> -----Original Message----- > >>>>> >> From: Edoardo Comar [mailto:edoco...@gmail.com] > >>>>> >> Sent: 14 May 2018 14:17 > >>>>> >> To: dev@kafka.apache.org > >>>>> >> Subject: [DISCUSS] KIP-302 - Enable Kafka clients to use all DNS > >>>>> resolved > >>>>> >> IP addresses > >>>>> >> > >>>>> >> Hi all, > >>>>> >> > >>>>> >> We just opened a KIP to add support for the client to use all IPs > >>>>> returned > >>>>> >> by DNS for the brokers > >>>>> >> > >>>>> >> The details are here - > >>>>> >> > >>>>> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.a > >>>>> >> pache.org_confluence_display_KAFKA_KIP-2D302-2B-2D-2BEnable- > >>>>> >> 2BKafka-2Bclients-2Bto-2Buse-2Ball-2BDNS-2Bresolved-2BIP- > >>>>> >> 2Baddresses&d=DwIBaQ&c=7563p3e2zaQw0AB1wrFVgyagb2IE5rTZOYPxL > >>>>> >> xfZlX4&r=nNmJlu1rR_QFAPdxGlafmDu9_r6eaCbPOM0NM1EHo-E&m=EJafF > >>>>> >> l1clRyolgtcu2uCc4_cIOJnlxb1r1n-D2Dti4k&s=C-UZ6KUG7JFiPD_ > >>>>> >> CnHczDOVqH9-XC5f_OFkw4BTNrI4&e= > >>>>> >> > >>>>> >> The JIRA and provisional PR (where the discussion lead to the > creation > >>>>> of > >>>>> >> this KIP) are : > >>>>> >> > >>>>> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues. > >>>>> >> apache.org_jira_browse_KAFKA-2D6863&d=DwIBaQ&c=7563p3e2zaQw0 > >>>>> >> AB1wrFVgyagb2IE5rTZOYPxLxfZlX4&r=nNmJlu1rR_QFAPdxGlafmDu9_r6 > >>>>> >> eaCbPOM0NM1EHo-E&m=EJafFl1clRyolgtcu2uCc4_cIOJnlxb1r1n- > >>>>> >> D2Dti4k&s=3Puqs5iYoPsw6hARQr6gvokdFE-H5USMiNVGOUtNkJI&e= > >>>>> >> > >>>>> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__github. > >>>>> >> com_apache_kafka_pull_4987&d=DwIBaQ&c=7563p3e2zaQw0AB1wrFVgy > >>>>> >> agb2IE5rTZOYPxLxfZlX4&r=nNmJlu1rR_QFAPdxGlafmDu9_r6eaC > >>>>> >> bPOM0NM1EHo-E&m=EJafFl1clRyolgtcu2uCc4_cIOJnlxb1r1n-D2Dti4k& > >>>>> >> s=Hqn5dOgQy4-MHTIJLE49O8bNomry3SoGq9OVoHU-CRA&e= > >>>>> >> > >>>>> >> Looking forward to the community's feedback. > >>>>> >> It would be amazing to have it voted by May 22nd :-) :-) > >>>>> >> > >>>>> >> Edoardo & Mickael > >>>>> >> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> "When the people fear their government, there is tyranny; when the > >>>>> government fears the people, there is liberty." [Thomas Jefferson] > >>>>> > >>> > >>> > >>> > >>> -- > >>> "When the people fear their government, there is tyranny; when the > >>> government fears the people, there is liberty." [Thomas Jefferson] > >> > >> > >> > >> -- > >> "When the people fear their government, there is tyranny; when the > >> government fears the people, there is liberty." [Thomas Jefferson] > > > > > > > > -- > > "When the people fear their government, there is tyranny; when the > > government fears the people, there is liberty." [Thomas Jefferson] > > > > ________________________________ > > > > Your Personal Data: We may collect and process information about you > that may be subject to data protection laws. For more information about how > we use and disclose your personal data, how we protect your information, > our legal basis to use your information, your rights and who you can > contact, please refer to: www.gs.com/privacy-notices< > http://www.gs.com/privacy-notices> > > > > > > ---------- Forwarded message ---------- > > From: Rajini Sivaram <rajinisiva...@gmail.com> > > To: "Skrzypek, Jonathan" <jonathan.skrzy...@ln.email.gs.com>, dev < > dev@kafka.apache.org>, Ismael Juma <ism...@juma.me.uk> > > Cc: > > Bcc: > > Date: Tue, 22 May 2018 15:05:07 +0000 > > Subject: Re: FW: [VOTE] KIP-235 Add DNS alias support for secured > connection > > Hi Jonathan, > > > > I think it would make sense to convert the config in this KIP into an > enum so that we can add more variations later on. But since KIP-302 is > still under discussion, it is not clear what the config name should be. > Since today is the KIP deadline and the implementation itself is > straightforward, it would make sense to progress with this one for 2.0.0 if > we can get one more binding vote. > > > > Ismael, do you have time to take a look at KIP-235 today? > > > > Thanks, > > > > Rajini > > > > > > On Tue, May 22, 2018 at 3:45 PM, Skrzypek, Jonathan < > jonathan.skrzy...@gs.com> wrote: > >> > >> Hello Rajini, > >> > >> What do you think should be the next step here ? > >> > >> > >> Jonathan Skrzypek > >> > >> -----Original Message----- > >> From: Skrzypek, Jonathan [Tech] > >> Sent: 21 May 2018 10:51 > >> To: 'dev' > >> Subject: RE: [VOTE] KIP-235 Add DNS alias support for secured connection > >> > >> Hi, > >> > >> What would be the next step here ? > >> I know there's a discussion going on around KIP-302, but I'm also > conscious that the 2.0.0 deadline for KIPs is tomorrow. > >> I've opened this KIP in January and discussions have been productive > with an end solution I had the impression was reasonable, so I am keen to > see it make it the next release. > >> > >> > >> Jonathan Skrzypek > >> > >> -----Original Message----- > >> From: Skrzypek, Jonathan [Tech] > >> Sent: 14 May 2018 13:48 > >> To: dev > >> Subject: RE: [VOTE] KIP-235 Add DNS alias support for secured connection > >> > >> Sure, I modified the KIP to add more details > >> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-235%3A+Add+DNS+alias+support+for+secured+connection > >> > >> > >> Jonathan Skrzypek > >> > >> > >> -----Original Message----- > >> From: Ismael Juma [mailto:ism...@juma.me.uk] > >> Sent: 14 May 2018 11:53 > >> To: dev > >> Subject: Re: [VOTE] KIP-235 Add DNS alias support for secured connection > >> > >> Thanks for the KIP, Jonathan. It would be helpful to have more detail on > >> how SSL authentication could be broken if the new behaviour is the > default. > >> I know this was discussed in the mailing list thread, but it's > important to > >> include it in the KIP since it's the main reason why a new config is > needed > >> (and configs should be avoided whenever we can just do the right thing). > >> > >> Ismael > >> > >> On Fri, Mar 23, 2018 at 12:05 PM Skrzypek, Jonathan < > >> jonathan.skrzy...@gs.com> wrote: > >> > >> > Hi, > >> > > >> > I would like to start a vote for KIP-235 > >> > > >> > > >> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D235-253A-2BAdd-2BDNS-2Balias-2Bsupport-2Bfor-2Bsecured-2Bconnection&d=DwIBaQ&c=7563p3e2zaQw0AB1wrFVgyagb2IE5rTZOYPxLxfZlX4&r=nNmJlu1rR_QFAPdxGlafmDu9_r6eaCbPOM0NM1EHo-E&m=FM_uCHnnO2dqxWC0bi7_QOJKfKmQI80-Xduvb-URWOw&s=RpGkijfK-WHcU0s8ZtMXEkIr69QraJhYKaGSC9V_rnI&e= > >> > > >> > This is a proposition to add an option for reverse dns lookup of > >> > bootstrap.servers hosts, allowing the use of dns aliases on clusters > using > >> > SASL authentication. > >> > > >> > > >> > > >> > > >> > >> ________________________________ > >> > >> Your Personal Data: We may collect and process information about you > that may be subject to data protection laws. For more information about how > we use and disclose your personal data, how we protect your information, > our legal basis to use your information, your rights and who you can > contact, please refer to: www.gs.com/privacy-notices< > http://www.gs.com/privacy-notices> > > > > > > > > > > -- > "When the people fear their government, there is tyranny; when the > government fears the people, there is liberty." [Thomas Jefferson] > -- "When the people fear their government, there is tyranny; when the government fears the people, there is liberty." [Thomas Jefferson]
