Re: Question about Commit Policy

2015-01-27 Thread Henry Saputra
Just found out about this, thanks Stephan =) - Henry On Wed, Jan 7, 2015 at 7:44 AM, Stephan Ewen wrote: > Hi all! > > Since the feedback was positive, I added the guidelines to the wiki, with a > disclaimer that this is being refined. > > https://cwiki.apache.org/confluence/display/FLINK/Apache

Re: Question about Commit Policy

2015-01-27 Thread Fabian Hueske
I did not propose to pick the 5 most suitable tags. ;-) Just to mark commits that break or extend the API. If [api-breaking] and [api-extending] is considered to waste too much space, I would also be happy with [api-] and [api+] (or any other meaningful marker). 2015-01-28 2:37 GMT+01:00 Márton Ba

Re: Question about Commit Policy

2015-01-27 Thread Márton Balassi
Originally I was going to support Fabian's suggestion - I still like it, but I have to admit that it can become a slight overkill like in de8e066. [1] [1] https://git-wip-us.apache.org/repos/asf/flink/commit/de8e066 On Tue, Jan 27, 2015 at 2:03 PM, Max Michels wrote: > I agree with Aljoscha. In

[jira] [Created] (FLINK-1458) Interfaces and abstract classes are not valid types

2015-01-27 Thread John Sandiford (JIRA)
John Sandiford created FLINK-1458: - Summary: Interfaces and abstract classes are not valid types Key: FLINK-1458 URL: https://issues.apache.org/jira/browse/FLINK-1458 Project: Flink Issue Typ

Re: API behavior with data sinks (lazy) and eager operations

2015-01-27 Thread Max Michels
Let's make it clear that count/collection type of actions execute the plan up till that point (including the data sinks). From a user perspective, this seems most logic to me. The user might even rely on the data generated by the sinks. On Mon, Jan 19, 2015 at 11:46 AM, Fabian Hueske wrote: > Thi

Re: Implementing a list accumulator

2015-01-27 Thread Max Michels
Thank you for your help, Ufuk and Stephan. I made some changes to immediately serialize the stored objects. On Thu, Jan 22, 2015 at 2:58 AM, Stephan Ewen wrote: > True, that is tricky. The user code does not necessarily respect the > non-reuse mode. That may be true for any user code. Can the lis

Re: Tagging Flink classes with InterfaceAudience and InterfaceStability

2015-01-27 Thread Alexander Alexandrov
I don't get the difference between Private and LimitedPrivate, but otherwise seems like quite a nice idea. It will be also good if we can agree upon what these tags actually mean and add this meaning to the documentation. 2015-01-27 15:46 GMT+01:00 Robert Metzger : > Hi, > > Hadoop has annotatio

Tagging Flink classes with InterfaceAudience and InterfaceStability

2015-01-27 Thread Robert Metzger
Hi, Hadoop has annotations for tagging the stability and audience of classes and methods. Through that, you can have @InterfaceAudience.Public, Private, LimitedPrivate and also @InterfaceStability.Evolving, Unstable, and Stable. I guess there are tools which allow to automatically check if inter

[jira] [Created] (FLINK-1457) RAT check fails on Windows

2015-01-27 Thread Sebastian Kruse (JIRA)
Sebastian Kruse created FLINK-1457: -- Summary: RAT check fails on Windows Key: FLINK-1457 URL: https://issues.apache.org/jira/browse/FLINK-1457 Project: Flink Issue Type: Bug Repo

RE: Gather a distributed dataset

2015-01-27 Thread Kruse, Sebastian
Hi everyone, I just wanted to give you the pointer FLINK-1038 https://github.com/apache/flink/pull/94 This is an output format that can send DataSet contents via Java RMI to, e.g., the driver. I am currently using it a lot and it seems to scale pretty well. Cheers, Sebastian -Original Mess

Re: Question about Commit Policy

2015-01-27 Thread Max Michels
I agree with Aljoscha. Instead, I would suggest to make a tag in the commit message mandatory. That way, you could list the associated commits using git log --grep=api-breaking On Tue, Jan 27, 2015 at 1:50 PM, Aljoscha Krettek wrote: > But thats very long, and together with the issue tag I almost

Re: Question about Commit Policy

2015-01-27 Thread Aljoscha Krettek
But thats very long, and together with the issue tag I almost always have I lose a lot of my precious 80 characters. On Tue, Jan 27, 2015 at 1:17 PM, Fabian Hueske wrote: > I know I argued against "enforcing" commit tags, but how about we make two > tags mandatory, i.e., [api-breaking] and [api-

Re: YARN ITCases fail, master broken?

2015-01-27 Thread Fabian Hueske
Robert, thanks for fixing the MacOS build! Building on my Ubuntu VM is still failing though. Will send you the Maven output. Cheers, Fabian 2015-01-26 15:49 GMT+01:00 Robert Metzger : > With the help of Fabian and Ufuk, I was able to identify the issue for OS > X. I already pushed the fix (see

Re: Question about Commit Policy

2015-01-27 Thread Fabian Hueske
I know I argued against "enforcing" commit tags, but how about we make two tags mandatory, i.e., [api-breaking] and [api-extending]? 2015-01-07 18:15 GMT+01:00 Gyula Fóra : > +1 > This was much needed :) > 2015.01.07. 18:10 ezt írta ("Max Michels" ): > > > Nice. > > > > > > On Wed, Jan 7, 2015 at

Re: Future directions for Flink’s YARN support?

2015-01-27 Thread Robert Metzger
The code from Daniel has been written for the old YARN client. I think the most important change is this one: https://github.com/warneke/flink/commit/9843a14637594fb7ee265f5326af9007f2a3191c and it can be backported easily to the new YARN client. On Tue, Jan 27, 2015 at 7:00 AM, Stephan Ewen wr

Re: MODERATE for dev@flink.apache.org

2015-01-27 Thread Robert Metzger
Hey Ankit, sorry for the issues with the mailing lists. Its our fault that we didn't properly mark the issues@ list as a read-only mailing list on our website. I've updated the website to make it more clear: http://flink.apache.org/community.html#mailing-lists Regarding the problem you've reporte

Re: flink build issue

2015-01-27 Thread Robert Metzger
Hi, thank you for trying out Flink. I'm sorry that you ran into this issue. Flink does not have support for Hadoop YARN before 2.2.0. The reason for that is that Hadoop has changed the YARN APIs with the 2.2.0 release. (the pre 2.2.0 APIs are marked as "alpha") If you take a closer look into the S

Re: Warning message in Scala type analysis

2015-01-27 Thread Aljoscha Krettek
Ok, if it is only a testing dependency it should be alright. The problem seems to be that some of the Joda classes have annotations that reside in joda-convert. The java compiler seems to be OK with missing annotations while the Scala compiler gives a warning. At least that's how I understood it fr

flink build issue

2015-01-27 Thread takun
Hello : I want build flink 0.8 on yarn ,but we use version : 2.0.0-cdh4.3.0 ,it failed how can i build it . command : mvn clean install -DskipTests -Pvendor-repos -Dhadoop.version=2.0.0-cdh4.3.0 The document say , not support before hadoop 2.2.0 ,but i want to run on yarn like spark on yarn .