> Additionally, having code that hasn't been touch for a while increases
the risk of it
Sorry about this incomplete confusing sentence. I was about to remove it
when accidentally pushing the shortcut for sending the message out to the
mailing list.

On Tue, Oct 25, 2022 at 10:18 AM Matthias Pohl <matthias.p...@aiven.io>
wrote:

> I guess upgrading the minimal version should also mean cleaning up the
> codebase, i.e. removing code segments that have been around to allow
> support for older versions. The overall goal should be to improve the Flink
> codebase in my opinion. Considering what David said in the old thread about
> Hadoop users usually lacking behind with version upgrades [1], would we do
> this version bump in two phases, i.e. adding some deprecation notes and
> doing the actual cleanup later on? I think Gabor has a point with it not
> being really mentioned anywhere in the docs (the only location I could find
> in the docs about Hadoop version is [2]). In this sense, the support for
> older Hadoop versions was kind of implicit: We're talking about compiling
> Flink with Hadoop 2.8.5 but also mention older Hadoop versions which leaves
> room for interpretation.
>
> Additionally, having code that hasn't been touch for a while increases the
> risk of it
>
> Matthias
>
> [1] https://lists.apache.org/thread/w7www13tossxrxo1mttgb68v81rf6fks
> [2]
> https://nightlies.a1pache.org/flink/flink-docs-master/docs/deployment/resource-providers/yarn/#supported-hadoop-versions
> <https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/yarn/#supported-hadoop-versions>
>
> On Fri, Oct 21, 2022 at 4:13 AM Xintong Song <tonysong...@gmail.com>
> wrote:
>
>> I believe there are some reflection based approaches in the `flink-yarn`
>> module, for supporting outdated APIs in early Hadoop versions.
>>
>> I haven't done a thorough check, and these are what I get.
>> - AMRMClientAsyncReflector
>> - ApplicationSubmissionContextReflector
>> - ContainerRequestReflector
>> - RegisterApplicationMasterResponseReflector
>> - ResourceInformationReflector
>>
>> Are we removing these as well? If yes, then Flink can no longer work with
>> the old hadoop versions. (That's how I understand "bumping the minimal
>> supported hadoop version".) I personally am not super eager to get rid of
>> theses, because the relevant parts of codes are no longer frequently
>> changing, thus the maintenance overhead is low.
>>
>> Best,
>>
>> Xintong
>>
>>
>>
>> On Thu, Oct 20, 2022 at 8:00 PM Yang Wang <danrtsey...@gmail.com> wrote:
>>
>> > Given that we do not bundle any hadoop classes in the Flink binary, do
>> you
>> > mean simply bump the hadoop version in the parent pom?
>> > If it is, why do not we use the latest stable hadoop version 3.3.4? It
>> > seems that our cron build has verified that hadoop3 could work.
>> >
>> > Best,
>> > Yang
>> >
>> > David Morávek <david.mora...@gmail.com> 于2022年10月19日周三 16:29写道:
>> >
>> > > +1; anything below 2.10.x seems to be EOL
>> > >
>> > > Best,
>> > > D.
>> > >
>> > > On Mon, Oct 17, 2022 at 10:48 AM Márton Balassi <
>> > balassi.mar...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hi Martjin,
>> > > >
>> > > > +1 for 2.10.2. Do you expect to have bandwidth in the near term to
>> > > > implement the bump?
>> > > >
>> > > > On Wed, Oct 5, 2022 at 5:00 PM Gabor Somogyi <
>> > gabor.g.somo...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hi Martin,
>> > > > >
>> > > > > Thanks for bringing this up! Lately I was thinking about to bump
>> the
>> > > > hadoop
>> > > > > version to at least 2.6.1 to clean up issues like this:
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/flink/blob/8d05393f5bcc0a917b2dab3fe81a58acaccabf13/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/util/HadoopUtils.java#L157-L159
>> > > > >
>> > > > > All in all +1 from my perspective.
>> > > > >
>> > > > > Just a question here. Are we stating the minimum Hadoop version
>> for
>> > > users
>> > > > > somewhere in the doc or they need to find it out from source code
>> > like
>> > > > > this?
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/flink/blob/3a4c11371e6f2aacd641d86c1d5b4fd86435f802/tools/azure-pipelines/build-apache-repo.yml#L113
>> > > > >
>> > > > > BR,
>> > > > > G
>> > > > >
>> > > > >
>> > > > > On Wed, Oct 5, 2022 at 5:02 AM Martijn Visser <
>> > > martijnvis...@apache.org>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi everyone,
>> > > > > >
>> > > > > > Little over a year ago a discussion thread was opened on
>> changing
>> > the
>> > > > > > minimal supported version of Hadoop and bringing that to 2.8.5.
>> [1]
>> > > In
>> > > > > this
>> > > > > > discussion thread, I would like to propose to bring that minimal
>> > > > > supported
>> > > > > > version of Hadoop to 2.10.2.
>> > > > > >
>> > > > > > Hadoop 2.8.5 is vulnerable for multiple CVEs which are
>> classified
>> > as
>> > > > > > Critical. [2] [3]. While Flink is not directly impacted by
>> those,
>> > we
>> > > do
>> > > > > see
>> > > > > > vulnerability scanners flag Flink as being vulnerable. We could
>> > > easily
>> > > > > > mitigate that by bumping the minimal supported version of
>> Hadoop to
>> > > > > 2.10.2.
>> > > > > >
>> > > > > > I'm looking forward to your opinions on this topic.
>> > > > > >
>> > > > > > Best regards,
>> > > > > >
>> > > > > > Martijn
>> > > > > > https://twitter.com/MartijnVisser82
>> > > > > > https://github.com/MartijnVisser
>> > > > > >
>> > > > > > [1]
>> > https://lists.apache.org/thread/81fhnwfxomjhyy59f9bbofk9rxpdxjo5
>> > > > > > [2] https://nvd.nist.gov/vuln/detail/CVE-2022-25168
>> > > > > > [3] https://nvd.nist.gov/vuln/detail/CVE-2022-26612
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Reply via email to