I would recommend to add "flink-shaded-hadoop2". That is a bundle of all Hadoop dependencies used by Flink.
On Fri, Dec 8, 2017 at 3:44 PM, Aljoscha Krettek <aljos...@apache.org> wrote: > I see, thanks for letting us know! > > > On 8. Dec 2017, at 15:42, shashank agarwal <shashank...@gmail.com> wrote: > > I had to include two dependencies. > > hadoop-hdfs (this for HDFS configuration) > hadoop-common (this for Path) > > > > > On Fri, Dec 8, 2017 at 7:38 PM, Aljoscha Krettek <aljos...@apache.org> > wrote: > >> I think hadoop-hdfs might be sufficient. >> >> >> On 8. Dec 2017, at 14:48, shashank agarwal <shashank...@gmail.com> wrote: >> >> Can you specifically guide which dependencies should I add to extend this >> : >> >> https://github.com/apache/flink/blob/release-1.4.0-rc3/flink >> -connectors/flink-connector-filesystem/src/main/java/org/ >> apache/flink/streaming/connectors/fs/Bucketer.java >> >> is the Hadoop-core is sufficient. >> >> >> >> >> On Fri, Dec 8, 2017 at 6:58 PM, shashank agarwal <shashank...@gmail.com> >> wrote: >> >>> It's a compilation error. I think I have to include the Hadoop >>> dependencies. >>> >>> >>> >>> >>> >>> On Fri, Dec 8, 2017 at 6:54 PM, Aljoscha Krettek <aljos...@apache.org> >>> wrote: >>> >>>> Hi, >>>> >>>> Is this a compilation error or at runtime. But in general, yes you have >>>> to include the Hadoop dependencies if they're not there. >>>> >>>> Best, >>>> Aljoscha >>>> >>>> >>>> On 8. Dec 2017, at 14:10, shashank agarwal <shashank...@gmail.com> >>>> wrote: >>>> >>>> Hello, >>>> >>>> I am trying to test 1.4.0-RC3, Hadoop libraries removed in this >>>> version. Actually, i have created custom Bucketer for the bucketing sink. >>>> I am extending >>>> >>>> >>>> org.apache.flink.streaming.connectors.fs.bucketing.Bucketer >>>> >>>> in the class, i have to use org.apache.hadoop.fs.Path but as hadoop >>>> libraries removed it's giving error >>>> >>>> "object hadoop is not a member of package org.apache" >>>> >>>> Should i have to include Hadoop client libs in build.sbt dependencies. >>>> >>>> >>>> Thanks >>>> Shashank >>>> >>>> >>>> >>>> >>> >>> >>> -- >>> Thanks Regards >>> >>> SHASHANK AGARWAL >>> --- Trying to mobilize the things.... >>> >>> >> >> >> -- >> Thanks Regards >> >> SHASHANK AGARWAL >> --- Trying to mobilize the things.... >> >> >> > > > -- > Thanks Regards > > SHASHANK AGARWAL > --- Trying to mobilize the things.... > > >
