[jira] [Created] (FLINK-4126) Unstable test ZooKeeperLeaderElectionTest.testZooKeeperReelection

2016-06-27 Thread Kostas Kloudas (JIRA)
Kostas Kloudas created FLINK-4126: - Summary: Unstable test ZooKeeperLeaderElectionTest.testZooKeeperReelection Key: FLINK-4126 URL: https://issues.apache.org/jira/browse/FLINK-4126 Project: Flink

support launch more than one pipeline in a yarn session

2016-06-27 Thread Chen Qin
Hi there, We are researching launch more than one pipeline in a flink yarn session. It's sort of like cluster mode where user pick off multiple pipelines on a job manager. Is there any plan to support this use case? If not, why not? Is it because of better isolation per pipeline or some implementa

Re: [Discussion] Query regarding Join

2016-06-27 Thread Vinay Patil
Hi, Following is the timestamp I am getting from DTO, here is the timestamp difference between the two records : 1466115892162154279 1466116026233613409 So the time difference is roughly 3 min, even if I apply the window of 5min , I am not getting the last record (last timestamp value above), usi

[jira] [Created] (FLINK-4125) Yarn CLI incorrectly calculates slotsPerTM when parallelism < task manager count

2016-06-27 Thread Chris Hogue (JIRA)
Chris Hogue created FLINK-4125: -- Summary: Yarn CLI incorrectly calculates slotsPerTM when parallelism < task manager count Key: FLINK-4125 URL: https://issues.apache.org/jira/browse/FLINK-4125 Project: F

[jira] [Created] (FLINK-4124) Unstable test WrapperSetupHelperTest.testCreateTopologyContext

2016-06-27 Thread Kostas Kloudas (JIRA)
Kostas Kloudas created FLINK-4124: - Summary: Unstable test WrapperSetupHelperTest.testCreateTopologyContext Key: FLINK-4124 URL: https://issues.apache.org/jira/browse/FLINK-4124 Project: Flink

Re: Adding a Histogram Metric

2016-06-27 Thread Steve Cosenza
Excellent! Thanks!!! On Thursday, June 23, 2016, Till Rohrmann wrote: > Hi Steve, > > that should be the corresponding JIRA ticket [1]. I think Chesnay already > opened a PR for this feature. I think it will be reviewed and probably > merged next week. > > [1] https://issues.apache.org/jira/brow

Re: [Discussion] Query regarding Join

2016-06-27 Thread Vinay Patil
Just an update, when I keep IngestionTime and remove the timestamp I am generating, I am getting all the records, but for Event Time I am getting one less record, I checked the Time Difference between two records, it is 3 min, I tried keeping the window time to 5 mins, but that even did not work.

Re: IT Test

2016-06-27 Thread Vijay Srinivasaraghavan
I just pulled the latest code and ran below commands. Both of them failed and it is consistent. 1) "mvn clean install" Results : Failed tests:   LocalFlinkMiniClusterITCase.testLocalFlinkMiniClusterWithMultipleTaskManagers:166 Thread Thread[initialSeedUniquifierGenerator,5,main] was started by

Re: [Discussion] Query regarding Join

2016-06-27 Thread Vinay Patil
Hi , Actually I am only publishing 5 messages each to two different kafka topics (using Junit), even if I keep the window to 500 seconds the result is same. I am not understanding why it is not sending the 5th element to co-group operator even when the keys are same. I actually cannot share the

Re: IT Test

2016-06-27 Thread Aljoscha Krettek
Hi, the first command you mentioned doesn't work because the Yarn tests are not executed in the traditional phases and therefore maven will complain. If you do it like this it will work (I tried it): mvn clean verify -pl flink-yarn-tests -Pinclude-yarn-tests -Dtest=YARNSessionFIFOITCase#testJavaAP

Re: [Discussion] Query regarding Join

2016-06-27 Thread Aljoscha Krettek
Hi, what timestamps are you assigning? Is it guaranteed that all of them would fall into the same 30 second window? The issue with duplicate printing in the ElementSelector is strange? Could you post a more complete code example so that I can reproduce the problem? Cheers, Aljoscha On Mon, 27 Ju

Re: IT Test

2016-06-27 Thread Vijay Srinivasaraghavan
I am on Ubuntu 16.x, Java OpenJDK 1.8.0_91. Can you try below commands and see if its working with the latest trunk code. mvn clean verify -pl flink-yarn-tests -Pinclude-yarn-tests -Dtest=YARNSessionFIFOITCase#testJavaAPI mvn clean install RegardsVijay On Monday, June 27, 2016 5:44 AM, Al

Re: IT Test

2016-06-27 Thread Aljoscha Krettek
Hi, I just ran a "mvn clean verify" and it passed on my machine (latest master, OS X El Capitan, Java 1.8.0_40, Maven 3.3.9). What's your environment? Cheers, Aljoscha On Fri, 24 Jun 2016 at 16:47 Vijay Srinivasaraghavan wrote: > I am seeing below failure consistently with the latest trunk code

Re: [DISCUSS] Releasing Flink 1.1.0

2016-06-27 Thread Robert Metzger
Sure Ufuk! Thanks a lot for taking care of the release management. I'll be on vacation in three weeks, for three weeks and I'm not sure if we get the release done until then. On Mon, Jun 27, 2016 at 12:08 PM, Ufuk Celebi wrote: > I would like to do it if that's OK with you Robert. I would follow

Re: [Discussion] Query regarding Join

2016-06-27 Thread Vinay Patil
Hi , I am able to get the matching and non-matching elements. However when I am unit testing the code , I am getting one record less inside the overriden cogroup function. Testing the following way : 1) Insert 5 messages into local kafka topic (test1) 2) Insert different 5 messages into local ka

[jira] [Created] (FLINK-4123) CassandraWriteAheadSink can hang on cassandra failure

2016-06-27 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-4123: --- Summary: CassandraWriteAheadSink can hang on cassandra failure Key: FLINK-4123 URL: https://issues.apache.org/jira/browse/FLINK-4123 Project: Flink Iss

[jira] [Created] (FLINK-4122) Cassandra jar contains 2 guava versions

2016-06-27 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-4122: --- Summary: Cassandra jar contains 2 guava versions Key: FLINK-4122 URL: https://issues.apache.org/jira/browse/FLINK-4122 Project: Flink Issue Type: Bug

Re: [DISCUSS] Releasing Flink 1.1.0

2016-06-27 Thread Ufuk Celebi
I would like to do it if that's OK with you Robert. I would follow your suggestion and wait a few days until the following important fixes are in: - Savepoint headers and proper disposal (FLINK-4067 and https://github.com/apache/flink/pull/2083) - Metrics (https://github.com/apache/flink/pull/2146)

Re: forward()

2016-06-27 Thread Till Rohrmann
Maybe we could document this fact in the code base. I think this is not a very obvious behaviour. Cheers, Till On Mon, Jun 27, 2016 at 11:24 AM, Aljoscha Krettek wrote: > Hi, > you are right, this seems a bit strange. The reason why this works is that > selectChannels of ForwardPartitioner is n

Re: forward()

2016-06-27 Thread Aljoscha Krettek
Hi, you are right, this seems a bit strange. The reason why this works is that selectChannels of ForwardPartitioner is never called. In StreamingJobGraphGenerator.connect() operators are connected by "physical" edges. If the ForwardPartitioner is set on a stream this will choose the POINTWISE conn

Re: Understanding Operator Chaining

2016-06-27 Thread Aljoscha Krettek
Hi Marius, the chaining code is still somewhat fragile and some stuff in there are leftovers. For example, StreamGroupedFold can only be used on a KeyedStream, which means that it can never be within a chain because the shuffle always breaks a chain. Specifying HEAD here is therefore redundant. I t

[jira] [Created] (FLINK-4121) Add timeunit (ms) to docs for timestamps and watermarks

2016-06-27 Thread Jonas Traub (JIRA)
Jonas Traub created FLINK-4121: -- Summary: Add timeunit (ms) to docs for timestamps and watermarks Key: FLINK-4121 URL: https://issues.apache.org/jira/browse/FLINK-4121 Project: Flink Issue Type: