Thanks alot for the feedback on the issue.
It seems that everybody agrees to drop Hadoop 1 support in Flink. I don't
think we need to vote on the issue.
I've filed a JIRA for the task:
https://issues.apache.org/jira/browse/FLINK-4895 Maybe I'll find time to
work on it next week.
On Fri, Oct 14,
Thanks for the pointer.
I'll start a separate discussion and push the PR forward if we come to an
agreement.
2016-10-14 11:04 GMT+02:00 Stephan Ewen :
> @Fabian - Someone started with that in
> https://issues.apache.org/jira/browse/FLINK-4315
> That could be changed to not remove the methods from
@Fabian - Someone started with that in
https://issues.apache.org/jira/browse/FLINK-4315
That could be changed to not remove the methods from the
ExecutionEnvironment.
On Fri, Oct 14, 2016 at 10:45 AM, Fabian Hueske wrote:
> Yes, I'm also +1 for removing the methods at some point.
>
> For 1.2 we
Yes, I'm also +1 for removing the methods at some point.
For 1.2 we could go ahead and move the Hadoop-MR connectors into a separate
module and mark the methods in ExecutionEnvironment as @deprecated.
In 1.3 (or 2.0 whatever comes next) we could make the switch.
2016-10-14 10:40 GMT+02:00 Stephan
@Fabian Good point. For Flink 2.0, I would suggest to remove them from the
Environment and add them to a Utility. The way it is now, it ties Flink
very strongly to Hadoop.
You are right, before we do that, there is no way to make a Hadoop
independent distribution.
On Fri, Oct 14, 2016 at 10:37 AM
+1 for dropping Hadoop1 support.
Regarding a binary release without Hadoop:
What would we do about the readHadoopFile() and createHadoopInput() on the
ExecutionEnvironment?
These methods are declared as @PublicEvolving, so we did not commit to keep
them.
However that does not necessarily mean we
@Greg
I think that would be amazing. It does require a bit of cleanup, though. As
far as I know, the Hadoop dependency is additionally used for some Kerberos
utilities and for its S3 file system implementation.
We would need to make the Kerberos part Hadoop independent and the
FileSystem loading d
Okay, this sounds prudent. Would this be the right time to implement
FLINK-2268 "Provide Flink binary release without Hadoop"?
On Thu, Oct 13, 2016 at 11:25 AM, Stephan Ewen wrote:
> +1 for dropping Hadoop1 support
>
> @greg There is quite some complexity in the build setup and release scripts
>
+1 for dropping Hadoop1 support
@greg There is quite some complexity in the build setup and release scripts
and testing to support Hadoop 1. Also, we have to prepare to add support
for Hadoop 3, and then supporting in addition Hadoop 1 seems very tough.
Stephan
On Thu, Oct 13, 2016 at 5:04 PM,
e.org
> *Subject:* [DISCUSS] Drop Hadoop 1 support with Flink 1.2
>
>
>
> Hi,
>
>
>
> The Apache Hadoop community has recently released the first alpha version
> for Hadoop 3.0.0, while we are still supporting Hadoop 1. I think its time
> to finally drop Hadoop 1 supp
Hi Robert,
What are the benefits to Flink for dropping Hadoop 1 support? Is there
significant code cleanup or would we simply be publishing one less set of
artifacts?
Greg
On Thu, Oct 13, 2016 at 10:47 AM, Robert Metzger
wrote:
> Hi,
>
> The Apache Hadoop community has recently released the fi
...@flink.apache.org
Subject: [DISCUSS] Drop Hadoop 1 support with Flink 1.2
Hi,
The Apache Hadoop community has recently released the first alpha version for
Hadoop 3.0.0, while we are still supporting Hadoop 1. I think its time to
finally drop Hadoop 1 support in Flink.
The last minor Hadoop 1
Hi,
The Apache Hadoop community has recently released the first alpha version
for Hadoop 3.0.0, while we are still supporting Hadoop 1. I think its time
to finally drop Hadoop 1 support in Flink.
The last minor Hadoop 1 release was in 27 June, 2014.
Apache Spark dropped Hadoop 1 support with thei
13 matches
Mail list logo
