I should back up Donjoon's comments with the observation that hadoop 2.10.x
is the only branch-2 release which get any security updates; on branch-3 it
is 3.2.x and 3.3.x which do. Donjoon's colleague Chao Sun was the release
manager on the 3.3.2 release, so it got thoroughly tested with Spark. (I'
Hi, Amin
In general, the Apache Spark community has received many feedbacks and been
moving forward to
- Use the latest Hadoop versions for more bug fixes including CVEs.
- Use Hadoop's shaded clients to minimize the dependency issues
Since the above is not achievable with Hadoop 2 clients, I be
>From Spark version 3.1.0 onwards, the clients provided for Spark are built
>with Hadoop 3 and placed in maven repository. Unfortunately we use Hadoop
>2.7.7 in our infrastructure currently.
1) Does Spark have a plan to publish the Spark client dependencies for Hadoop
2.x?
2) Are the new Spark
