Yes, it does officially since 2.4.0. 2.4.5 is a maintenance release of 2.4.x line and the community didn't support Hadoop 3.x on 'branch-2.4'. We didn't run test at all.
Bests, Dongjoon. On Sun, Feb 2, 2020 at 22:58 Ajith shetty <ajith.she...@huawei.com> wrote: > Is hadoop-3.1 profile supported for this release.? i see lot of UTs > failing under this profile. > https://github.com/apache/spark/blob/v2.4.5-rc2/pom.xml > > *Example:* > [INFO] Running org.apache.spark.sql.hive.JavaMetastoreDataSourcesSuite > [ERROR] Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: > 1.717 s <<< FAILURE! - in > org.apache.spark.sql.hive.JavaMetastoreDataSourcesSuite > [ERROR] > saveExternalTableAndQueryIt(org.apache.spark.sql.hive.JavaMetastoreDataSourcesSuite) > Time elapsed: 1.675 s <<< ERROR! > java.lang.ExceptionInInitializerError > at > org.apache.spark.sql.hive.JavaMetastoreDataSourcesSuite.setUp(JavaMetastoreDataSourcesSuite.java:66) > Caused by: java.lang.IllegalArgumentException: *Unrecognized Hadoop major > version number: 3.1.0* > at > org.apache.spark.sql.hive.JavaMetastoreDataSourcesSuite.setUp(JavaMetastoreDataSourcesSuite.java:66) >
