Re: Spark client for Hadoop 2.x

2022-04-12 Thread Steve Loughran
I should back up Donjoon's comments with the observation that hadoop 2.10.x is the only branch-2 release which get any security updates; on branch-3 it is 3.2.x and 3.3.x which do. Donjoon's colleague Chao Sun was the release manager on the 3.3.2 release, so it got thoroughly tested with Spark. (I'

Re: Spark client for Hadoop 2.x

2022-04-10 Thread Dongjoon Hyun
Hi, Amin In general, the Apache Spark community has received many feedbacks and been moving forward to - Use the latest Hadoop versions for more bug fixes including CVEs. - Use Hadoop's shaded clients to minimize the dependency issues Since the above is not achievable with Hadoop 2 clients, I be