BTW, can we leverage flink-shaded-hadoop-2? Reason why I ask, if any Flink module is going to use Hadoop in any way, it will most probably include flink-shaded-hadoop-2 as a dependency. However, flink-shaded modules don't have any source files. Is that a strict convention that the community follows?
- Sivaprasanna On Tue, Mar 3, 2020 at 10:48 PM Sivaprasanna <sivaprasanna...@gmail.com> wrote: > Hi Arvid, > > Thanks for the quick reply. Yes, it actually makes sense to avoid Hadoop > dependencies from getting into Flink's core modules but I also wonder if it > will be an overkill to add flink-hadoop-fs as a dependency just because we > want to use a utility class from that module. > > - > Sivaprasanna > > On Tue, Mar 3, 2020 at 4:17 PM Arvid Heise <ar...@ververica.com> wrote: > >> Hi Sivaprasanna, >> >> we actually want to remove Hadoop from all core modules, so we could not >> place it in some very common place like flink-core. >> >> But I think the module flink-hadoop-fs could be a fitting place. >> >> On Tue, Mar 3, 2020 at 11:25 AM Sivaprasanna <sivaprasanna...@gmail.com> >> wrote: >> >> > Hi >> > >> > The flink-sequence-file module has a class named >> > SerializableHadoopConfiguration[1] which is nothing but a wrapper class >> for >> > Hadoop Configuration. I believe this class can be moved to a common >> module >> > since this is not necessarily tightly coupled with sequence-file module, >> > and also because it can be used by many other modules, for ex. >> > flink-compress. Thoughts? >> > >> > - >> > Sivaprasanna >> > >> >
