Github user orenpai commented on the issue:
https://github.com/apache/zeppelin/pull/1970
@zjffdu @Leemoonsoo I believe this is a mistake, and poses a real problem
for me and other Zeppelin users.
While it's true that the properties file is being skimmed to remove any
non- spark.* properties, SparkConf.set allows it directly via code:
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkConf.scala#L84
There are many libraries like es-hadoop for ElasticSearch integration that
depend on the SparkConf having non spark.* properties, such as "es.nodes".
Since Zeppelin does not allow a user to recreate the sc, this important
mechanism of directly adding non spark properties via code is being blocked. I
can't find any workaround either.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
