Thanks alot for the feedback on the issue.
It seems that everybody agrees to drop Hadoop 1 support in Flink. I don't
think we need to vote on the issue.
I've filed a JIRA for the task:
https://issues.apache.org/jira/browse/FLINK-4895 Maybe I'll find time to
work on it next week.
On Fri, Oct 14,
Thanks for the pointer.
I'll start a separate discussion and push the PR forward if we come to an
agreement.
2016-10-14 11:04 GMT+02:00 Stephan Ewen :
> @Fabian - Someone started with that in
> https://issues.apache.org/jira/browse/FLINK-4315
> That could be changed to not remove the methods from
@Fabian - Someone started with that in
https://issues.apache.org/jira/browse/FLINK-4315
That could be changed to not remove the methods from the
ExecutionEnvironment.
On Fri, Oct 14, 2016 at 10:45 AM, Fabian Hueske wrote:
> Yes, I'm also +1 for removing the methods at some point.
>
> For 1.2 we
Yes, I'm also +1 for removing the methods at some point.
For 1.2 we could go ahead and move the Hadoop-MR connectors into a separate
module and mark the methods in ExecutionEnvironment as @deprecated.
In 1.3 (or 2.0 whatever comes next) we could make the switch.
2016-10-14 10:40 GMT+02:00 Stephan
@Fabian Good point. For Flink 2.0, I would suggest to remove them from the
Environment and add them to a Utility. The way it is now, it ties Flink
very strongly to Hadoop.
You are right, before we do that, there is no way to make a Hadoop
independent distribution.
On Fri, Oct 14, 2016 at 10:37 AM
+1 for dropping Hadoop1 support.
Regarding a binary release without Hadoop:
What would we do about the readHadoopFile() and createHadoopInput() on the
ExecutionEnvironment?
These methods are declared as @PublicEvolving, so we did not commit to keep
them.
However that does not necessarily mean we
@Greg
I think that would be amazing. It does require a bit of cleanup, though. As
far as I know, the Hadoop dependency is additionally used for some Kerberos
utilities and for its S3 file system implementation.
We would need to make the Kerberos part Hadoop independent and the
FileSystem loading d
Okay, this sounds prudent. Would this be the right time to implement
FLINK-2268 "Provide Flink binary release without Hadoop"?
On Thu, Oct 13, 2016 at 11:25 AM, Stephan Ewen wrote:
> +1 for dropping Hadoop1 support
>
> @greg There is quite some complexity in the build setup and release scripts
>
+1 for dropping Hadoop1 support
@greg There is quite some complexity in the build setup and release scripts
and testing to support Hadoop 1. Also, we have to prepare to add support
for Hadoop 3, and then supporting in addition Hadoop 1 seems very tough.
Stephan
On Thu, Oct 13, 2016 at 5:04 PM,
+1 to dropping Hadoop 1.x
I am fairly certain there are very few legacy Hadoop users. 2.x is heavily
used at the moment.
Spark actually changed not just Hadoop but Python versions as well.
Hadoop 3 would take a while to mature so I would suggest holding off on
that after it is well baked in and us
Hi Robert,
What are the benefits to Flink for dropping Hadoop 1 support? Is there
significant code cleanup or would we simply be publishing one less set of
artifacts?
Greg
On Thu, Oct 13, 2016 at 10:47 AM, Robert Metzger
wrote:
> Hi,
>
> The Apache Hadoop community has recently released the fi
I am totally agree with Robert. From the industry point of view, we are not
using in any client Hadoop 1.x . Even in legacy system, we have already
upgraded the software.
From: Robert Metzger [mailto:rmetz...@apache.org]
Sent: jueves, 13 de octubre de 2016 16:48
To: dev@flink.apache.org; u...@fl
12 matches
Mail list logo
