________________________________
From: Joe Hammerman <jhammer...@squarespace.com.INVALID>
Sent: Tuesday, April 3, 2018 10:40 PM
To: users@kafka.apache.org
Subject: Re: advertised.listeners
Hi all,
Is it possible to run mixed mode with PLAINTEXT and SSL with no SASL?
MG>setup SSL on kafka
https://github.com/edenhill/librdkafka/wiki/Using-SSL-with-librdkafka
[https://avatars1.githubusercontent.com/u/524990?s=400&v=4]<https://github.com/edenhill/librdkafka/wiki/Using-SSL-with-librdkafka>
Using SSL with librdkafka · edenhill/librdkafka Wiki ·
GitHub<https://github.com/edenhill/librdkafka/wiki/Using-SSL-with-librdkafka>
github.com
The SSL support in librdkafka is completely configuration based, no new APIs
are introduced, this means that any existing applications dynamically linked
with librdkafka will get automatic SSL support by upgrading only the library.
This page serves as a simple HOWTO guide. Apache Kafka brokers ...
MG> and then enabling SSL on listeners and advertised.listeners
MG>kafka PLAINTEXT and SSL server properties:
listeners=PLAINTEXT://:9093,SSL://:9092
advertised.listeners = PLAINTEXT://IP_ADDRESS:9093,SSL://IP_ADDRESS:9092
producerMB>security.protocol=PLAINTEXT
consumerMB>security.protocol=PLAINTEXT
MG>
What
should port and advertised listeners values in kafka-server.properties be
set in order to configure such an access profile? We want to so in order to
be able to perform healthchecking on the loopback device without having to
negotiate an SSL connection.
Thanks in advance for any assistance anyone can provide,
MG>PLAINTEXT for advertised.listeners is inherently dangerous since PLAINTEXT
is allowed for MB but not for ZK
MG>Moreover SSL (not SASL-SSL) is only supported in ZK 3.5 which is alpha so
CAVEAT EMPTOR
MG>if ZK is SSL enabled
MG>you will need a entity placed between MB and ZK to prevent ZK returning
auth-fail on PLAINTEXT requests
MG>from MB
MG>"A current workaround while Kafka and ZooKeeper security features are not
available is to have a secured Kafka cluster which coexists with PLAINTEXT
clients such as the REST proxy."
MG>the suggested workaround for supporting PLAINTEXT and SSL
MG is to config
MG>network-segmentation (CIDR?)
http://www.itgeared.com/articles/1347-cidr-and-subnetting-tutorial/
CIDR and Subnetting |
ITGeared.com<http://www.itgeared.com/articles/1347-cidr-and-subnetting-tutorial/>
www.itgeared.com
Classless Inter Domain Routing (CIDR) is a method for assigning IP addresses
without using the standard IP address classes like Class A, Class B or Class C.
CIDR is a newer addressing scheme for IP Networks which allows for a more
efficient allocation of IP addresses than the older method which was by
assigning organizations a class of IPs.
MG>if you dont want network segmentation you can config your Kafka ACLs
explained here:
https://docs.confluent.io/3.0.0/kafka/platform-security.html
MG>HTH
Joseph Hammerman
On Tue, Apr 3, 2018 at 8:06 PM, Martin Gainty <mgai...@hotmail.com> wrote:
> (from MessageBroker consumer)
>
> >tracert MessageProducer
>
> if zk server is found in tracert
>
> then yes you have a MB quorom
>
> FWIR Mixing PLAINTEXT with SASL-SSL on ZK is not supported
> https://stackoverflow.com/questions/46912937/is-it-
[https://cdn.sstatic.net/Sites/stackoverflow/img/apple-touch-i...@2.png?v=73d79a89bded]<https://stackoverflow.com/questions/46912937/is-it->
Is it possible to connect zookeeper and kafka via SASL , kafka broker and its
clients via SSL without enabling plain
text?<https://stackoverflow.com/questions/46912937/is-it->
stackoverflow.com
I am using Kafka Version 0.10.1. I connected Kafka brokers and its clients via
SSL and its working fine.Now I have a query with some limitations. My
limitations are No Plain text communications al...
> possible-to-connect-zookeeper-and-kafka-via-sasl-kafka-broker-and-its-cl
>
> but there is a ZK SASL SASL-Auth exception backdoor in
> zookeeper.properties when
> zookeeper.main_connection_despite_sasl_failure="yes"
>
>
> catch (SaslException e) {
> LOG.warn("Client failed to SASL authenticate: " + e);
> if
> ((System.getProperty("zookeeper.maintain_connection_despite_sasl_failure")
> != null)
> &&
> (System.getProperty("zookeeper.maintain_connection_
> despite_sasl_failure").equals("yes"))) {
> LOG.warn("Maintaining client connection despite SASL authentication
> failure.");
> } else {
> LOG.warn("Closing client connection due to SASL authentication
> failure.");
> cnxn.close();
> ..
> }
>
>
> does this conform to your findings?
>
> M
> ______________________________________________
>
>
>
> ________________________________
> From: Darshan <purandare.dars...@gmail.com>
> Sent: Tuesday, April 3, 2018 6:47 PM
> To: users@kafka.apache.org
> Subject: Re: advertised.listeners
>
> We are using 0.10.2.1 and ZK 3.4.9. Can something be derived from this
> piece of info ? Thanks.
>
> On Tue, Apr 3, 2018 at 3:13 PM, Martin Gainty <mgai...@hotmail.com> wrote:
>
> > tracert MessageBrokerIP
> >
> > do you see ZK server in the trace?
> >
> > if yes then you are running kafka-cluster
> >
> > (ZK does not support mixed mode but there is a backdoor
> > zookeeper.properties config attribute that allows plaintext clients to
> > bypass sasl auth)
> >
> > ?
> >
> > Martin
> > ______________________________________________
> >
> >
> >
> > ________________________________
> > From: Darshan <purandare.dars...@gmail.com>
> > Sent: Tuesday, April 3, 2018 5:45 PM
> > To: rajinisiva...@gmail.com
> > Cc: users@kafka.apache.org
> > Subject: Re: advertised.listeners
> >
> > Hi Rajini
> >
> > The above configuration that you mentioned a while back helped me sort
> the
> > issue of listeners and I was also able to run Kafka 0.10.2.1 with SSL and
> > ACLs as well from one of your other posts.
> >
> > I wanted to ask you if it is possible to run Kafka in a mixed security
> > mode? i.e. external producers who are on 172.x.x.x interface can use the
> > SSL to send/receive data from/to our Kafka brokers, but our internal
> > consumer can read/write on PLAINTEXT channel to Kafka.
> >
> > Here is my server.properties, but my internal producer which does not
> have
> > any keystore and truststore is getting the following error:
> > *[2018-04-03 21:21:04,378] WARN Error while fetching metadata with
> > correlation id 1 : {Topic4006=TOPIC_AUTHORIZATION_FAILED}
> > (org.apache.kafka.clients.NetworkClient)*
> >
> >
> > *server.properties for our Kafka-1,2,3. They are identical except
> > broker.id
> > <http://broker.id> and super.users properties.*
> broker.id - This website is for sale! - broker Resources and
> Information.<http://broker.id/>
> broker.id
> This website is for sale! broker.id is your first and best source for all
> of the information you’re looking for. From general topics to more of what
> you would expect to find here, broker.id has it all. We hope you find
> what you are searching for!
>
>
>
> > broker.id - This website is for sale! - broker Resources and
> > Information.<http://broker.id/>
> > broker.id
> > This website is for sale! broker.id is your first and best source for
> all
> > of the information you’re looking for. From general topics to more of
> what
> > you would expect to find here, broker.id has it all. We hope you find
> > what you are searching for!
> >
> >
> >
> >
> > # ID and basic topic creation
> > broker.id=1
> > auto.create.topics.enable=true
> > delete.topic.enable=true
> >
> > # LISTERN Settings
> > listeners=INTERNAL://1.1.1.165:9092,EXTERNAL://172.21.190.176:9093
> > advertised.listeners=INTERNAL://1.1.1.165:9092,EXTERNAL://
> > 172.21.190.176:9093
> > listener.security.protocol.map=INTERNAL:PLAINTEXT,EXTERNAL:SSL
> > inter.broker.listener.name=INTERNAL
> > host.name=172.21.190.176
> >
> > # Security Settings
> > ssl.keystore.location=keystore.jks
> > ssl.keystore.password=password
> > ssl.key.password=password
> > ssl.truststore.location=truststore.jks
> > ssl.truststore.password=password
> > ssl.keystore.type=JKS
> > ssl.truststore.type=JKS
> > security.protocol=SSL
> > ssl.client.auth=required
> > allow.everyone.if.no.acl.found=false
> > authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
> > super.users=User:CN=Kafka1
> >
> > Can you please point out if anything needs to be modified ?
> >
> > Many thanks.
> >
> > --Darshan
> >
> > On Wed, May 31, 2017 at 11:31 AM, Rajini Sivaram <
> rajinisiva...@gmail.com>
> > wrote:
> >
> > > If you want to use different interfaces with the same security
> protocol,
> > > you can specify listener names. You can then also configure different
> > > security properties for internal/external if you need.
> > >
> > > listeners=INTERNAL://1.x.x.x:9092,EXTERNAL://172.x.x.x:9093
> > >
> > > advertised.listeners=INTERNAL://1.x.x.x:9092,EXTERNAL://172.x.x.x:9093
> > >
> > > listener.security.protocol.map=INTERNAL:SSL,EXTERNAL:SSL
> > >
> > > inter.broker.listener.name=INTERNAL
> > >
> > > On Wed, May 31, 2017 at 6:22 PM, Raghav <raghavas...@gmail.com> wrote:
> > >
> > > > Hello Darshan
> > > >
> > > > Have you tried SSL://0.0.0.0:9093 ?
> > > >
> > > > Rajani had suggested something similar to me a week back while I was
> > > > trying to get a ACL based setup.
> > > >
> > > > Thanks.
> > > >
> > > > On Wed, May 31, 2017 at 8:58 AM, Darshan <
> purandare.dars...@gmail.com>
> > > > wrote:
> > > >
> > > >> Hi
> > > >>
> > > >> Our Kafka broker has two IPs on two different interfaces.
> > > >>
> > > >> eth0 has 172.x.x.x for external leg
> > > >> eth1 has 1.x.x.x for internal leg
> > > >>
> > > >>
> > > >> Kafka Producer is on 172.x.x.x subnet, and Kafka Consumer is on
> > 1.x.x.x
> > > >> subnet.
> > > >>
> > > >> If we use advertised.listeners=SSL://172.x.x.x:9093, then Producer
> > can
> > > >> producer the message, but Consumer cannot receive the message.
> > > >>
> > > >> What value should we use for advertised.listeners so that Producer
> can
> > > >> write and Consumers can read ?
> > > >>
> > > >> Thanks.
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Raghav
> > > >
> > >
> >
>
