+1 for upgrading Hadoop dependency versions. And I echo the same thoughts as Eduard for Iceberg's JDK 8 requirement. It will be good to have at least Java 11/17 as the minimum required version.
- Ajantha On Fri, Feb 16, 2024 at 7:47 PM Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Hi Fokko, > > I agree to remove Hadoop 2.7.3 and bump to 3.x for Iceberg 2.0.0. > > If we want to be based on Java 11+ (17 would be good imho), we have to > check the dependencies. I'm not sure all of our dependencies support > JDK17 today. > > Regards > JB > > On Fri, Feb 16, 2024 at 10:09 AM Fokko Driesprong <fo...@apache.org> > wrote: > > > > Hi everyone, > > > > I want to discuss adding the Hadoop upgrade to the list after moving to > Iceberg 2.0. We still compile against Hadoop 2.7.3 to ensure we support as > many users as possible. Hadoop 2.7.3 was released August 2016 and is not > maintained anymore for a long time. > > > > My main reason for doing the upgrade is that on the Parquet MR project, > I've been pushing back the Hadoop upgrade to ensure compatibility with > Iceberg. However, at some point, we have to pull the trigger here. This > will simplify things on the Parquet side and avoid having to check if the > Java API Exists and such. > > > > Since Hadoop 3.3+ officially supports Java 11, I would suggest dropping > everything below that. I wanted to check on the mailing list if there are > any thoughts and or concerns. > > > > Kind regards, > > Fokko > > > > >
