Thank you very much for your help! Antoine
-----Original Message----- From: Manikumar [mailto:manikumar.re...@gmail.com] Sent: Wednesday, April 25, 2018 1:50 AM To: Users Subject: Re: Transfer data between kerberized kafka clusters (different principals) like this: producerConfig.put("sasl.jaas.config", "com.sun.security.auth.module.Krb5LoginModule required " + "useTicketCache=false " + "renewTicket=true " + "serviceName=\"kafka\" " + "useKeyTab=true " + "keyTab=\"/home/test.keytab\" " + "principal=\"t...@example.com\";"); On Wed, Apr 25, 2018 at 12:01 AM, Zieger, Antoine < antoine.zie...@morganstanley.com> wrote: > Hi, > > Thanks for the link, I am sorry this might be a lack of java skills on my > side but I still don’t understand how I can use it in a java class. The > example is provided in case of a property file from what I understand. > > Would you mind providing a java example ? > producerConfig.put("sasl.jaas.config", <whatIsTheExpectedFormatHere?>) > > Thanks again. > > Antoine > > -----Original Message----- > From: Manikumar [mailto:manikumar.re...@gmail.com] > Sent: Tuesday, April 24, 2018 2:06 PM > To: Users > Subject: Re: Transfer data between kerberized kafka clusters (different > principals) > > Yes. Sample example/format here: > https://kafka.apache.org/documentation/#security_ > kerberos_sasl_clientconfig > > On Tue, Apr 24, 2018 at 11:30 PM, Zieger, Antoine < > antoine.zie...@morganstanley.com> wrote: > > > Hi, > > > > Thank you very much for this quick answer. I am not sure I understand > > where/how to use this property. I see an example here > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 85%3A+Dynamic+JAAS+ > > configuration+for+Kafka+clients but I don't understand how to use it > > while creating a Producer/Consumer instance. Could you help me with this > > please? > > > > A quick example would be the following? > > > > //Producer with specific config: principal 'abc' > > Properties producerConfig = new Properties(); > > producerConfig.put("sasl.jaas.config" , <what is the format here?>) > > KafkaProducer<String, String> producer = new KafkaProducer<>( > > producerConfig) > > > > //Consumer with specific config: principal 'xyz' > > Properties consumerConfig = new Properties(); > > consumerConfig.put("sasl.jaas.config" , <what is the format here?>) > > KafkaConsumerr<String, String> producer = new KafkaConsumer<>( > > consumerConfig) > > > > Thanks in advance. > > Antoine > > > > -----Original Message----- > > From: Manikumar [mailto:manikumar.re...@gmail.com] > > Sent: Tuesday, April 24, 2018 1:39 PM > > To: Users > > Subject: Re: Transfer data between kerberized kafka clusters (different > > principals) > > > > Hi, > > > > From Kafka 0.10.2.0, we can configure producer/consumer jaas > configuration > > using "sasl.jaas.config" config property. Using this we can configure > > different principals. > > > > On Tue, Apr 24, 2018 at 10:58 PM, Zieger, Antoine < > > antoine.zie...@morganstanley.com> wrote: > > > > > Hi, > > > > > > I am trying to transfer data between two kerberized kafka clusters. The > > > brokers are running under different users on both clusters and so, I am > > > providing a different Kerberos.service.name to consumer and producer. > > For > > > security reasons and access management rules in places, having the same > > > principal on both sides is not possible. Whenever I start my java > > program I > > > get the error "principals don't match" > > > > > > My understanding is that kafka mirror-maker is doing the same thing and > > > does not support different principals. (https://community. > > > hortonworks.com/articles/79891/kafka-mirror-maker-best-practices.html > ) > > > "In kafka 0.9.x and 0.10.0.1, 0.10.1.0, consumers and producers in > > > mirror-maker cannot run with different principals/keytabs as they both > > run > > > inside a single JVM" > > > > > > As I understand it this is a Kafka API limitation right? I looked at > the > > > release notes of versions > 0.10.1.0 and didn't see any ticket that > > > resolved this, unless I missed it? > > > > > > By any chance are you aware of a workaround? Or the only solution in > case > > > of two different principals is to have two different JVMs? (one for the > > > consumer and one for the producer, which means we need some kind of > > buffer > > > in between). > > > > > > Thanks in advance for your help. > > > > > > Antoine > > > > > > > > > ________________________________ > > > NOTICE: Morgan Stanley is not acting as a municipal advisor and the > > > opinions or views contained herein are not intended to be, and do not > > > constitute, advice within the meaning of Section 975 of the Dodd-Frank > > Wall > > > Street Reform and Consumer Protection Act. If you have received this > > > communication in error, please destroy all electronic and paper copies > > and > > > notify the sender immediately. Mistransmission is not intended to waive > > > confidentiality or privilege. Morgan Stanley reserves the right, to the > > > extent required and/or permitted under applicable law, to monitor > > > electronic communications, including telephone calls with Morgan > Stanley > > > personnel. This message is subject to the Morgan Stanley General > > > Disclaimers available at the following link: > > http://www.morganstanley.com/ > > > disclaimers. If you cannot access the links, please notify us by reply > > > message and we will send the contents to you. By communicating with > > Morgan > > > Stanley you acknowledge that you have read, understand and consent, > > (where > > > applicable), to the foregoing and the Morgan Stanley General > Disclaimers. > > > > > > > ------------------------------------------------------------ > > -------------------- > > NOTICE: Morgan Stanley is not acting as a municipal advisor and the > > opinions or views contained herein are not intended to be, and do not > > constitute, advice within the meaning of Section 975 of the Dodd-Frank > Wall > > Street Reform and Consumer Protection Act. If you have received this > > communication in error, please destroy all electronic and paper copies > and > > notify the sender immediately. Mistransmission is not intended to waive > > confidentiality or privilege. Morgan Stanley reserves the right, to the > > extent permitted under applicable law, to monitor electronic > > communications. This message is subject to terms available at the > following > > link: http://www.morganstanley.com/disclaimers If you cannot access > > these links, please notify us by reply message and we will send the > > contents to you. By communicating with Morgan Stanley you consent to the > > foregoing and to the voice recording of conversations with personnel of > > Morgan Stanley. > > ------------------------------------------------------------ > -------------------- > NOTICE: Morgan Stanley is not acting as a municipal advisor and the > opinions or views contained herein are not intended to be, and do not > constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall > Street Reform and Consumer Protection Act. If you have received this > communication in error, please destroy all electronic and paper copies and > notify the sender immediately. Mistransmission is not intended to waive > confidentiality or privilege. Morgan Stanley reserves the right, to the > extent permitted under applicable law, to monitor electronic > communications. This message is subject to terms available at the following > link: http://www.morganstanley.com/disclaimers If you cannot access > these links, please notify us by reply message and we will send the > contents to you. By communicating with Morgan Stanley you consent to the > foregoing and to the voice recording of conversations with personnel of > Morgan Stanley. > -------------------------------------------------------------------------------- NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.
