Hi Team,
I am trying to stream data from kafkaconsumer using:
https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html
Here my KAFKA is Kerberos secured and SSL enabled.
I am running my Flink streaming in yarn-cluster on EMR 5.31.
I have tried to pass keytab/principal in following *2 Ways*:
1) Passing as JVM property in Flink run Command.
/usr/lib/flink/bin/flink run
-yt ${app_install_path}/conf/
\
> -Dsecurity.kerberos.login.use-ticket-cache=false
> \
> -yDsecurity.kerberos.login.use-ticket-cache=false
> \
> -Dsecurity.kerberos.login.keytab=${app_install_path}/conf/keytab \
> -yDsecurity.kerberos.login.keytab=${app_install_path}/conf/.keytab \
> -Djava.security.krb5.conf=${app_install_path}/conf/krb5.conf
> \
> -yDjava.security.krb5.conf=${app_install_path}/conf/krb5.conf
> \
> -Dsecurity.kerberos.login.principal=x...@xx.net \
> -yDsecurity.kerberos.login.principal= x...@xx.net \
> -Dsecurity.kerberos.login.contexts=Client,KafkaClient
> \
> -yDsecurity.kerberos.login.contexts=Client,KafkaClient
>
*Here, I am getting the following Error, it seems like KEYTAB Was not
transported to the run environment and probably not found.*
*org.apache.kafka.common.KafkaException: Failed to construct kafka
consumerCaused by: java.lang.IllegalArgumentException: Could not find a
'KafkaClient' entry in the JAAS configuration. System property
'java.security.auth.login.config'*
2) Passing from flink config: * /usr/lib/flink/conf/flink-conf.yaml*
security.kerberos.login.use-ticket-cache: false
security.kerberos.login.keytab: ${app_install_path}/conf/keytab
security.kerberos.login.principal: x...@xx.net
security.kerberos.login.contexts: Client,KafkaClient
*Here, I am getting the following Error, *
*org.apache.kafka.common.KafkaException: Failed to construct kafka
consumerCaused by: org.apache.kafka.common.KafkaException:
java.lang.IllegalArgumentException: No serviceName defined in either JAAS
or Kafka config*
Could you please help find, what are probable causes and resolution?
Regards,
Vijay
