[
https://issues.apache.org/jira/browse/SPARK-56597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-56597.
----------------------------------
Resolution: Invalid
Resolving as Invalid — this is a usage/how-to question rather than a specific
Spark defect or actionable change. Usage questions are best directed to
[email protected] (https://spark.apache.org/community.html) or Stack
Overflow (tag apache-spark). Findings from triage: Verified at HEAD
e9d2378b5a27: Spark core defines spark.io.encryption.enabled with
.createWithDefault(false)
(core/src/main/scala/org/apache/spark/internal/config/package.scala:1169) and
docs/security.md:281 documents the default as "false". The report's premise —
that Spark defaults IO encryption ON — is factually incorrect for Spark core; a
repo-wide grep found no code setting it to true. The actual target is the
Apache Spark Kubernetes Operator (apache/spark-kubernetes-operator), a separate
Apache project not present in this repo (only resource-managers/kubernetes
exists, with no io.encrypt
Please reopen with a concrete reproducer or a specific proposed change if this
is actually a bug or an actionable improvement.
> Default enablement of spark.io.encryption.enabled in spark apache operator
> leads to oom on spark jobs migrated from kubeflow one
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-56597
> URL: https://issues.apache.org/jira/browse/SPARK-56597
> Project: Spark
> Issue Type: Question
> Components: Kubernetes
> Affects Versions: 0.8.0
> Reporter: Nicolas Fraison
> Priority: Minor
>
> While migrating from kubeflow operator to spark kubernetes operator a subset
> of our jobs are facing some OOM
> The issue seems to be linked to the default enablement of the
> [spark.io.encryption.enabled|https://github.com/apache/spark-kubernetes-operator/blob/f90db67f0f704d7b298e637bca4763420f4a323e/spark-submission-worker/src/main/java/org/apache/spark/k8s/operator/SparkAppSubmissionWorker.java#L169C38-L169C65]
> [~dongjoon] would you be fine in adding capabilities to opt out this config?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
