[jira] [Created] (FLINK-2998) Support range partition comparison for multi input nodes.

Chengxiang Li created FLINK-2998: Summary: Support range partition comparison for multi input nodes. Key: FLINK-2998 URL: https://issues.apache.org/jira/browse/FLINK-2998 Project: Flink Issue

[jira] [Created] (FLINK-2997) Support range partition with user customized data distribution.

Chengxiang Li created FLINK-2997: Summary: Support range partition with user customized data distribution. Key: FLINK-2997 URL: https://issues.apache.org/jira/browse/FLINK-2997 Project: Flink

Re: [gelly] Spargel model rework

You could implement a Java Either type (similar to Scala's Either) that either has a Message or the VertexState and a corresponding TypeInformation and TypeSerializer that serializes a byte flag to indicate which both types is used. It might actually make sense, to add a generic Either type to the

Re: Tagging Flink classes with InterfaceAudience and InterfaceStability

I think we need anyways an annotation "@PublicExperimental". We can make this annotation such that it can be added to methods and can use that to declare Methods in an otherwise public class (such as DataSet) as experimental. On Tue, Nov 10, 2015 at 10:19 PM, Fabian Hueske wrote: > I am not sur

Re: Tagging Flink classes with InterfaceAudience and InterfaceStability

I am not sure if we always should declare complete classes as @PublicInterface. This does definitely make sense for interfaces and abstract classes such as MapFunction or InputFormat but not necessarily for classes such as DataSet that we might want to extend by methods which should not immediately

Re: [gelly] Spargel model rework

Hi, after running a few experiments, I can confirm that putting the combiner after the flatMap is indeed more efficient. I ran SSSP and Connected Components with Spargel, GSA, and the Pregel model and the results are the following: - for SSSP, Spargel is always the slowest, GSA is a ~1.2x faster

[VOTE] Release Apache Flink 0.10.0 (release-0.10.0-rc8)

Please note that this vote has a slightly shorter voting period of 48 hours. Only very small changes have been made since the last release candidate. Since the community has already done extensive testing of the previous release candidates, I'm assuming 48 hours will suffice to vote on this release

Re: Tagging Flink classes with InterfaceAudience and InterfaceStability

Yes, my opinion is that we shouldn't declare the Gelly API frozen yet. We can reconsider when we're closer to the 1.0 release, but if possible, I would give it some more time. -V. On 10 November 2015 at 21:06, Stephan Ewen wrote: > I think no component should be forced to be stable. It should b

[jira] [Created] (FLINK-2996) Add config entry to define BlobServer port

Stephan Ewen created FLINK-2996: --- Summary: Add config entry to define BlobServer port Key: FLINK-2996 URL: https://issues.apache.org/jira/browse/FLINK-2996 Project: Flink Issue Type: New Featur

Re: Tagging Flink classes with InterfaceAudience and InterfaceStability

I think no component should be forced to be stable. It should be an individual decision for each component, and in some cases even for individual classes. @Vasia If you think Gelly should not be declared interface-frozen, then this is a good point to raise and this should definitely be reflected.

Re: Tagging Flink classes with InterfaceAudience and InterfaceStability

Hi Robert, thanks for bringing this up! I generally like the idea, but I wouldn't rush to annotate the Gelly classes yet. Gelly hasn't had that many users and I'm quite sure we'll find things to improve as it gets more exposure. TBH, I think it's quite unfair to force Gelly (also e.g. ML, Table)

Re: Tagging Flink classes with InterfaceAudience and InterfaceStability

For what it's worth, the new Apache Yetus [0] project includes an interface audience annotations module [1]. We have (or intend to have, if it's not available yet) tools for validation of public API compatibility across release versions. For example, here's such a report [2] from a previous HBase R

Re: Tagging Flink classes with InterfaceAudience and InterfaceStability

Hi everyone, I would like to bring this discussion back to your attention as we seem to approach the 1.0 release of Flink. My suggestion back in January was to annotate all classes, but I think it'll be more feasible to just annotate public classes. So how about adding an annotation @PublicInterf

Re: Core Memory Error

Thanks for reporting this. Are you using any custom data types? If you can share your code, it would be very helpful in order to debug this. – Ufuk On Tuesday, 10 November 2015, Fabian Hueske wrote: > I agree with Robert. Looks like a bug in Flink. > Maybe an off-by-one issue (violating index

Re: Roadmap for the 1.0 release

Hi Stephan, I like the idea of having a wiki page for planning the 1.0 release. Lets try to focus on API stability, user-reported bugs etc. for the 1.0 release to bring it out as soon as possible. Many users see a 1.0 release as a clear sign for stability and maturity, and I think Flink has reache

Re: Core Memory Error

I agree with Robert. Looks like a bug in Flink. Maybe an off-by-one issue (violating index is 32768 and the default memory segment size is 32KB). Which Flink version are you using? In case you are using a custom build, can you share the commit ID (is reported in the first lines of the JobManager l

Re: Core Memory Error

Hi Ali, this could be a bug in Flink. Can you share the code of your program with us to debug the issue? On Tue, Nov 10, 2015 at 6:25 PM, Kashmar, Ali wrote: > Hello, > > I’m getting this error while running a streaming module on a cluster of 3 > nodes: > > > java.lang.ArrayIndexOutOfBoundsExce

Core Memory Error

Hello, I’m getting this error while running a streaming module on a cluster of 3 nodes: java.lang.ArrayIndexOutOfBoundsException: 32768 at org.apache.flink.core.memory.MemorySegment.get(MemorySegment.java:178) at org.apache.flink.runtime.io.network.api.serialization.SpillingAdaptiveSpanningRe

[jira] [Created] (FLINK-2995) Set default number of retries to larger than 0

Stephan Ewen created FLINK-2995: --- Summary: Set default number of retries to larger than 0 Key: FLINK-2995 URL: https://issues.apache.org/jira/browse/FLINK-2995 Project: Flink Issue Type: Wish

[jira] [Created] (FLINK-2994) Client sysout logging does not report exceptions

Stephan Ewen created FLINK-2994: --- Summary: Client sysout logging does not report exceptions Key: FLINK-2994 URL: https://issues.apache.org/jira/browse/FLINK-2994 Project: Flink Issue Type: Impr

[VOTE] [RESULT] Release Apache Flink 0.10.0 (release-0.10.0-rc7)

Good catch, Robert. Thank you for quickly fixing the issue! Hereby, this vote is cancelled. I will post a new release candidate ASAP. On Tue, Nov 10, 2015 at 12:14 PM, Robert Metzger wrote: > -1 > > What worked: > > - Deployed the flink-0.10.0-bin-hadoop26-scala_2.11.tgz >

[jira] [Created] (FLINK-2993) Set default elayBetweenExecutionRetries to 0

Stephan Ewen created FLINK-2993: --- Summary: Set default elayBetweenExecutionRetries to 0 Key: FLINK-2993 URL: https://issues.apache.org/jira/browse/FLINK-2993 Project: Flink Issue Type: Improvem

Re: Python Examples

Nice idea, +1 On Tue, Nov 10, 2015 at 11:53 AM, Matthias J. Sax wrote: > Hi, > > Slim recently twittered about Will McGinnis Python examples on Flink. > > https://github.com/wdm0006/flink-python-examples > > I think it would be nice to add it to the third party section on the web > page. Does an

Re: [VOTE] Release Apache Flink 0.10.0 (release-0.10.0-rc7)

-1 What worked: - Deployed the flink-0.10.0-bin-hadoop26-scala_2.11.tgz build on another Azure test cluster using YARN. Both issues I found yesterday are resolved - tested the new web interface. Seems to wor

[jira] [Created] (FLINK-2992) New Windowing code is using SerializationUtils with wrong classloader

Robert Metzger created FLINK-2992: - Summary: New Windowing code is using SerializationUtils with wrong classloader Key: FLINK-2992 URL: https://issues.apache.org/jira/browse/FLINK-2992 Project: Flink

Python Examples

Hi, Slim recently twittered about Will McGinnis Python examples on Flink. https://github.com/wdm0006/flink-python-examples I think it would be nice to add it to the third party section on the web page. Does anyone have any objections about it? If not, I will add it the next days. -Matthias

Re: [DISCUSS] Java code style

Thanks for wrapping up the discussion, Fabian! The tab size is adjustable in all the viewers I know of, e.g. DataSet with tab size of 2 on GitHub: https://github.com/apache/flink/blob/master/flink-java/src/main/java/org/apache/flink/api/java/DataSet.java?ts=2 You can even even use "less -x2 Source

[jira] [Created] (FLINK-2991) Extend Window Operators to Allow Efficient Fold Operation

Aljoscha Krettek created FLINK-2991: --- Summary: Extend Window Operators to Allow Efficient Fold Operation Key: FLINK-2991 URL: https://issues.apache.org/jira/browse/FLINK-2991 Project: Flink