Hi,
I have an HDP 2.5 cluster with Kerberos enabled running Oozie 4.2, Spark 1.6.2 and HBase 1.1.2. In this cluster, I have a Spark job which writes to HBase which I want to schedule via Oozie. Due to Kerberos, I had to make changes to core-site.xml to get Spark and HBase play nice with each other, which I must pass to the driver and executors in spark-submit via --conf "spark.executor.extraClassPath=/usr/hdp/current/hbase-client/conf” --driver-class-path "/usr/hdp/current/hbase-client/conf" When I put this into the spark-opts tag of the Spark action the configuration files do not get picked up and the authentication against HBase does not work. I assume the reason why the files are not picked up is the distributed cache. The ShareLib folder for Spark contains a version of the hbase-site.xml. I also uploaded the core-site.xml to the ShareLib folder for Spark, but as soon as I do that the Launch Mapper for the Spark action fails. Reason could be that the core-site.xml conflicts with the Oozie configuration. Question: How can I pass the core-site.xml file to the Spark action? I saw a lot of work regarding the Spark action in Oozie 4.3, but updating to this version is currently not an option. Best, Jan
