Re: Apache Ignite

2015-04-17 Thread sirinath
I was thinking in the lines of using this as an alternative clustering / distributed computing engine. -- View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Apache-Ignite-tp5115p5199.html Sent from the Apache Flink Mailing List archive. mailing list arc

[jira] [Created] (FLINK-1902) Web interface reports false (the default) jobmanager.rpc.port on YARN

2015-04-17 Thread JIRA
Zoltán Zvara created FLINK-1902: --- Summary: Web interface reports false (the default) jobmanager.rpc.port on YARN Key: FLINK-1902 URL: https://issues.apache.org/jira/browse/FLINK-1902 Project: Flink

Re: TableAPI - Join on two keys

2015-04-17 Thread Aljoscha Krettek
So, the first thing is a "feature" of the Java API that removes duplicate fields in keys, so an equi-join on (0,0) with (0,1) would throw an error because one 0 is removed from the first key. The second thing is a feature of the Table API where the error message is hinting at the problem: Could no

[jira] [Created] (FLINK-1903) Joins where one side uses a field more than once don't work

2015-04-17 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-1903: --- Summary: Joins where one side uses a field more than once don't work Key: FLINK-1903 URL: https://issues.apache.org/jira/browse/FLINK-1903 Project: Flink

Re: TableAPI - Join on two keys

2015-04-17 Thread Till Rohrmann
Why not doing two separate joins, union the results and doing a distinct operation on the combined key? On Fri, Apr 17, 2015 at 9:42 AM, Aljoscha Krettek wrote: > So, the first thing is a "feature" of the Java API that removes > duplicate fields in keys, so an equi-join on (0,0) with (0,1) would

[jira] [Created] (FLINK-1904) Add cross() to Table API

2015-04-17 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-1904: --- Summary: Add cross() to Table API Key: FLINK-1904 URL: https://issues.apache.org/jira/browse/FLINK-1904 Project: Flink Issue Type: Improvement

Re: TableAPI - Join on two keys

2015-04-17 Thread Felix Neutatz
I am also against the manual cross method. Isn't it the idea of the table API to hide the actual implementation from the user? Best regards, Felix Am 17.04.2015 10:09 vorm. schrieb "Till Rohrmann" : > Why not doing two separate joins, union the results and doing a distinct > operation on the comb

Re: [Gelly] Vertex-centric iteration updateVertex does not get called

2015-04-17 Thread Vasiliki Kalavri
Hi Gabor, the vertex-centric iteration implements the Pregel model, according to which a vertex is "active" during a superstep only when there are messages for this vertex. So, the behavior you are seeing is intended, yes :-) The logic behind this is that a vertex value gets updated by performing

Re: TableAPI - Join on two keys

2015-04-17 Thread Aljoscha Krettek
Yes, that is the idea, but I think in this case the user must be protected from an operation that can get ridiculously expensive. On Fri, Apr 17, 2015 at 10:20 AM, Felix Neutatz wrote: > I am also against the manual cross method. Isn't it the idea of the table > API to hide the actual implementat

[jira] [Created] (FLINK-1905) Add tests for parallelism > 1 for Kafka sources

2015-04-17 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-1905: - Summary: Add tests for parallelism > 1 for Kafka sources Key: FLINK-1905 URL: https://issues.apache.org/jira/browse/FLINK-1905 Project: Flink Issue Type: S

Re: [Gelly] Vertex-centric iteration updateVertex does not get called

2015-04-17 Thread Hermann Gábor
Hey, Thank you both for your help :) I was missing this point in the Pregel model. I thought it would be expected to receive empty message iterators. Yes, I know it is not the best use-case for vertex-centric iteration as there are well-defined steps, and also know the Flink examples. I only want

Re: TableAPI - Join on two keys

2015-04-17 Thread Fabian Hueske
I agree with Aljoscha. Let's give a good error message and offer a cross operator. 2015-04-17 4:52 GMT-05:00 Aljoscha Krettek : > Yes, that is the idea, but I think in this case the user must be > protected from an operation that can get ridiculously expensive. > > On Fri, Apr 17, 2015 at 10:20 A

[jira] [Created] (FLINK-1906) Add tip to work around plain Tuple return type of project operator

2015-04-17 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-1906: Summary: Add tip to work around plain Tuple return type of project operator Key: FLINK-1906 URL: https://issues.apache.org/jira/browse/FLINK-1906 Project: Flink

Re: TableAPI - Join on two keys

2015-04-17 Thread Stephan Ewen
I also agree with Aljoscha. It is widely considered a big mistake in SQL that cross products are implicit rather than explicit (because they are so expensive) Let's not make the same mistake here to put the theoretical algebra over the practical user experience and program safety. On Fri, Apr 17

Re: About Operator and OperatorBase

2015-04-17 Thread Stephan Ewen
Okay, it seems the consensus forms around not breaking the API. When it comes to the *OperatorBase - should we rename them or simply get rid of them (remove the common API). If we want to remove them, a precondition is to remove the Record API, and for that, we should migrate the Record-API-based

[jira] [Created] (FLINK-1907) Scala Interactive Shell

2015-04-17 Thread Nikolaas Steenbergen (JIRA)
Nikolaas Steenbergen created FLINK-1907: --- Summary: Scala Interactive Shell Key: FLINK-1907 URL: https://issues.apache.org/jira/browse/FLINK-1907 Project: Flink Issue Type: New Feature