Flink optimizer optimizations

2016-04-15 Thread CPC
Hi When i look for what kind of optimizations flink does, i found https://cwiki.apache.org/confluence/display/FLINK/Optimizer+Internals is it up to date? Also i couldnt understand: "Reusing of partitionings and sort orders across operators. If one operator leaves the data in partitioned fashion

[jira] [Created] (FLINK-3769) RabbitMQ Sink ability to publish to a different exchange

2016-04-15 Thread Robert Batts (JIRA)
Robert Batts created FLINK-3769: --- Summary: RabbitMQ Sink ability to publish to a different exchange Key: FLINK-3769 URL: https://issues.apache.org/jira/browse/FLINK-3769 Project: Flink Issue Ty

[jira] [Created] (FLINK-3768) Clustering Coefficient

2016-04-15 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-3768: - Summary: Clustering Coefficient Key: FLINK-3768 URL: https://issues.apache.org/jira/browse/FLINK-3768 Project: Flink Issue Type: New Feature Components:

Re: Issue regarding the submission of a topology to a remote flink cluster.

2016-04-15 Thread star jlong
Hi Till/Ned, Soory I thought this was my post. Le Vendredi 15 avril 2016 17h28, ned dogg a écrit : Hi Till/Jstar, Thanks for the the reply. Well I'm facing the same issue as Jstar. Here is my scenarios, I have app that is creating flink cluster on VMs for users. This app cannot create

Re: Issue regarding the submission of a topology to a remote flink cluster.

2016-04-15 Thread star jlong
Hi Till, Thanks for the reply. The idea of ssh of the instance is a good one. I thought of that but in my case it is not applicable because I setting up a cluster some employees of a company. And ssh the instance by employees will mean giving them the instance's key-pair, which I can not. Matte

Re: Surprising order of events in union of two streams

2016-04-15 Thread Gary Verhaegen
Hi Aljoscha, What I'm looking for is an operator that joins two streams together, but keeps the events in timestamp order. What I was trying to do with the window specification comes down to: for each event on that stream, I want to call this function with this event and all of the events that ar

[jira] [Created] (FLINK-3767) Show timeline also for running tasks, not only for finished ones

2016-04-15 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-3767: - Summary: Show timeline also for running tasks, not only for finished ones Key: FLINK-3767 URL: https://issues.apache.org/jira/browse/FLINK-3767 Project: Flink

[jira] [Created] (FLINK-3766) Add a new tab for monitoring metrics in the web interface

2016-04-15 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-3766: - Summary: Add a new tab for monitoring metrics in the web interface Key: FLINK-3766 URL: https://issues.apache.org/jira/browse/FLINK-3766 Project: Flink Iss

Re: FLINK-3750 (JDBCInputFormat)

2016-04-15 Thread Stefano Bortoli
If we share the connection, then we should also be careful with the close() implementation. I did not see changes for this method in the PR. saluti, Stefano 2016-04-15 11:01 GMT+02:00 Flavio Pompermaier : > Following your suggestions I've fixed the connection reuse in my PR at > https://github.c

Re: [VOTE] Release Apache Flink 1.0.2 (RC2)

2016-04-15 Thread Fabian Hueske
There is a request from the Mahout community to include a fix for FLINK-3762 in 1.0.2. Stephan gave a +1 [1] and I would also like to include it. I'm running tests right now and will merge the fix they provided later today. Cheers, Fabian [1] https://mail-archives.apache.org/mod_mbox/flink-dev/2

[VOTE] Release Apache Flink 1.0.2 (RC2)

2016-04-15 Thread Ufuk Celebi
Dear Flink community, Please vote on releasing the following candidate as Apache Flink version 1.0.2. The commit to be voted on: 058cc5de789331983830ce4720cd98c3af786109 Branch: release-1.0.2-rc2 (see https://git1-us-west.apache.org/repos/asf/flink/?p=flink.git;a=shortlog;h=refs/heads/release-1.

[jira] [Created] (FLINK-3765) ScalaShellITCase deadlocks spuriously

2016-04-15 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-3765: Summary: ScalaShellITCase deadlocks spuriously Key: FLINK-3765 URL: https://issues.apache.org/jira/browse/FLINK-3765 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-3764) LeaderChangeStateCleanupTest.testStateCleanupAfterListenerNotification fails on Travis

2016-04-15 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-3764: Summary: LeaderChangeStateCleanupTest.testStateCleanupAfterListenerNotification fails on Travis Key: FLINK-3764 URL: https://issues.apache.org/jira/browse/FLINK-3764

Re: Issue deploying a topology to flink with a java api

2016-04-15 Thread Matthias J. Sax
What is the compile error? Or could you already resolve it? there should not be a difference from 1.0 to 1.1-SNAPSHOT for both classes... And yes, prefix _2.11 is only available for released but not for current development branch. But it is the same code (no need to worry about it). -Matthias On

Re: Issue regarding the submission of a topology to a remote flink cluster.

2016-04-15 Thread Till Rohrmann
Hi Ned, what you also could do is to ssh to your remote cluster and submit the job using the private IP address which is reachable from within your cluster. I don't know whether that would be applicable to your use case. Cheers, Till On Fri, Apr 15, 2016 at 9:22 AM, Till Rohrmann wrote: > The

Re: FLINK-3750 (JDBCInputFormat)

2016-04-15 Thread Flavio Pompermaier
Following your suggestions I've fixed the connection reuse in my PR at https://github.com/apache/flink/pull/1885. I simply check in the establishConnection() if dbConn!=null and, in that case, I simply return immediately. Thus, the only remaining thin to fix is the null handling. Do you have any s

Re: Flink Interpreter w/ yarn-session

2016-04-15 Thread Till Rohrmann
In HA mode, the host and port information you provide to the Shell should be simply ignored. So you don't have to retrieve them from the .yarn-properties file. Could you maybe run the FlinkInterpreter with debug log level and share the logs with me? You can also do that privately, if you don't wan

Re: Sqoop-like module in Flink

2016-04-15 Thread Stefano Bortoli
Hi Flavio, I think this can be very handy when you have to run jobs Sqoop-like but you need to run the process with few resources. As for Cascading, Flink could do the heavy-lifting and make the scan of large relational databases more robust. Of course to make it work in real world, the JDBC Input

Re: Issue regarding the submission of a topology to a remote flink cluster.

2016-04-15 Thread Till Rohrmann
The log says: Unable to allocate on port 6123, due to error: Cannot assign requested address Thus, I would assume that something with your cluster configuration is not entirely correct. Could you check that? On Thu, Apr 14, 2016 at 11:19 AM, ned dogg wrote: > Here a try that I given. As first I