Thanks for starting the discussion Wei-Chiu. It will be good to hear the community views on this. Should we also cross-post this topic to GitHub discussions and the slack channel for more awareness because this will be an impactful change if we go ahead?
On Jul 26, 2023 at 10:38:19 AM, Wei-Chiu Chuang <weic...@apache.org> wrote: > Hi Ozoner community, it's me again. > > I'd like to get your feedback on dropping Hadoop 3.1 3.2 runtime support. > > Currently, Ozone is built with Hadoop 3.3.6 dependency. Our test matrix > certifies Ozone execution with Hadoop 2.7, 3.1, 3.2 and 3.3.6. I am not > familiar with the history of this, but given where we are, how do we feel > about dropping the test matrix against Hadoop 3.1, 3.2 and 3.3.6? > > Also Hadoop 2.7 is now at the end of life. If there are still Hadoop 2 > users, can we bump to Hadoop 2.10.1 which is the last Hadoop 2 release? > > I am asking because more and more Ozone features depend on newer Hadoop > releases. We have to workaround older releases by various hacks and it's > not scalable. As an example, to support unbuffer and bytebuffer-read, we > added CapableOzoneFSInputStream that is not in ozonefs-hadoop2. A recent > example is HDDS-8436 <https://issues.apache.org/jira/browse/HDDS-8436> > where > I had to copy several Hadoop classes into Ozone code. > > Looking forward to hearing from you, > Weichiu >
