[
https://issues.apache.org/jira/browse/SPARK-20435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15981730#comment-15981730
]
Mark Grover commented on SPARK-20435:
-------------------------------------
bq. I'm not saying redacting from logs is useless, but I'm saying that a user
that is providing secrets in the command line is giving up any security, and
redaction won't save him.
Thanks for the ps ax explanation. I appreciated your input and agree that
redacting from logs is not useless.
The way it is there are 2 ways to supply passwords:
1. The user copies over the entire conf (say from /etc/spark/conf to
$USER/custom-conf). And, then updates the spark-defaults.conf with the
appropriate properties containing the password. And, runs Spark jobs with this
custom configuration. The benefit is that without any change in Spark today,
they can run the jobs and the password won't be leaked anywhere. However, the
disadvantage is it is hard to keep the custom configuration in sync given the
lack of an overlay style config today in Spark. Moreover, the password is being
written by the user to possibly unencrypted disk in the custom configuration.
2. Supply the password over command line to spark-submit. The advantage is that
there's no custom configuration to be maintained, there's no password being
persisted to a file by the user. However, during the duration of the job, the
password is visible through output of commands like 'ps ax' and with the
current version of Spark, the password shows up in HDFS, in the event logs and
anything derived from them. And, the latter may not be secure. This change is
to make this case less worse by redacting passwords from HDFS event logs.
Furthermore, as a benefit, we get to add some unit tests that make sure none of
the redaction functionality regresses in the future.
I think both the above methods have their pros and cons and I think it's best
for us to document both ways and let the users choose which method they prefer.
This change makes #2 slight less worse and I think it's worth doing.
Your points make sense, but it seems still worth making #2 less worse. And, if
you agree, I'd really appreciate your review of the PR. Thanks!
> More thorough redaction of sensitive information from logs/UI, more unit tests
> ------------------------------------------------------------------------------
>
> Key: SPARK-20435
> URL: https://issues.apache.org/jira/browse/SPARK-20435
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.2.0
> Reporter: Mark Grover
>
> SPARK-18535 and SPARK-19720 were works to redact sensitive information (e.g.
> hadoop credential provider password, AWS access/secret keys) from event logs
> + YARN logs + UI and from the console output, respectively.
> While some unit tests were added along with these changes - they asserted
> when a sensitive key was found, that redaction took place for that key. They
> didn't assert globally that when running a full-fledged Spark app (whether or
> YARN or locally), that sensitive information was not present in any of the
> logs or UI. Such a test would also prevent regressions from happening in the
> future if someone unknowingly adds extra logging that publishes out sensitive
> information to disk or UI.
> Consequently, it was found that in some Java configurations, sensitive
> information was still being leaked in the event logs under the
> {{SparkListenerEnvironmentUpdate}} event, like so:
> {code}
> "sun.java.command":"org.apache.spark.deploy.SparkSubmit ... --conf
> spark.executorEnv.HADOOP_CREDSTORE_PASSWORD=secret_password ...
> {code}
> "secret_password" should have been redacted.
> Moreover, previously redaction logic was only checking if the key matched the
> secret regex pattern, it'd redact it's value. That worked for most cases.
> However, in the above case, the key (sun.java.command) doesn't tell much, so
> the value needs to be searched. So the check needs to be expanded to match
> against values as well.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
