Hi All,
Amazon inspector has detected a vulnerability in okio-1.15.0.jar JAR in Spark
3.4.1. It suggests to upgrade the jar version to 3.4.0. But when we try this
version of jar then the spark application is failing with below error:
py4j.protocol.Py4JJavaError: An error occurred while calling
None.org.apache.spark.api.java.JavaSparkContext.
: java.lang.NoClassDefFoundError: okio/BufferedSource
at okhttp3.internal.Util.<clinit>(Util.java:62)
at okhttp3.OkHttpClient.<clinit>(OkHttpClient.java:127)
at okhttp3.OkHttpClient$Builder.<init>(OkHttpClient.java:475)
at
io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newOkHttpClientBuilder(OkHttpClientFactory.java:41)
at
io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:56)
at
io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:68)
at
io.fabric8.kubernetes.client.okhttp.OkHttpClientFactory.newBuilder(OkHttpClientFactory.java:30)
at
io.fabric8.kubernetes.client.KubernetesClientBuilder.getHttpClient(KubernetesClientBuilder.java:88)
at
io.fabric8.kubernetes.client.KubernetesClientBuilder.build(KubernetesClientBuilder.java:78)
at
org.apache.spark.deploy.k8s.SparkKubernetesClientFactory$.createKubernetesClient(SparkKubernetesClientFactory.scala:120)
at
org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:111)
at
org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:3037)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:568)
at
org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown
Source)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
Source)
at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)
at py4j.Gateway.invoke(Gateway.java:238)
at
py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80)
at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69)
at
py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ClassNotFoundException: okio.BufferedSource
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown
Source)
at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown
Source)
at java.base/java.lang.ClassLoader.loadClass(Unknown Source)
... 26 more
Replaced the existing jar with the JAR file at
https://repo1.maven.org/maven2/com/squareup/okio/okio/3.4.0/okio-3.4.0.jar
PFB, the vulnerability details:
Link: https://nvd.nist.gov/vuln/detail/CVE-2023-3635
[cid:[email protected]]
Any guidance here would be of great help.
Thanks,
Sanket A.
This message (including any attachments) contains confidential information
intended for a specific individual and purpose, and is protected by law. If you
are not the intended recipient, you should delete this message and any
disclosure, copying, or distribution of this message, or the taking of any
action based on it, by you is strictly prohibited.
Deloitte refers to a Deloitte member firm, one of its related entities, or
Deloitte Touche Tohmatsu Limited ("DTTL"). Each Deloitte member firm is a
separate legal entity and a member of DTTL. DTTL does not provide services to
clients. Please see www.deloitte.com/about to learn more.
v.E.1
