[jira] [Created] (FLINK-3841) RocksDB statebackend creates empty dbs for stateless operators

Gyula Fora created FLINK-3841: - Summary: RocksDB statebackend creates empty dbs for stateless operators Key: FLINK-3841 URL: https://issues.apache.org/jira/browse/FLINK-3841 Project: Flink Issue

[jira] [Created] (FLINK-3840) RocksDB local parent dir is polluted with empty folders with random names

Gyula Fora created FLINK-3840: - Summary: RocksDB local parent dir is polluted with empty folders with random names Key: FLINK-3840 URL: https://issues.apache.org/jira/browse/FLINK-3840 Project: Flink

flatMap issue

I have a test file which has a json per line. When I do a flatMap on it, it automatically splits the whole json line on every character. Why does this happen? So if I do: val data=env.readTextFile("file:///home/punit/vik-in") val j=data.flatMap { x=>x } j.first(1).print() This prints "{" -- T

[jira] [Created] (FLINK-3839) Support wildcards in classpath parameters

Ken Krugler created FLINK-3839: -- Summary: Support wildcards in classpath parameters Key: FLINK-3839 URL: https://issues.apache.org/jira/browse/FLINK-3839 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-3838) CLI parameter parser is munging application params

Ken Krugler created FLINK-3838: -- Summary: CLI parameter parser is munging application params Key: FLINK-3838 URL: https://issues.apache.org/jira/browse/FLINK-3838 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-3837) Create FLOOR/CEIL function

Timo Walther created FLINK-3837: --- Summary: Create FLOOR/CEIL function Key: FLINK-3837 URL: https://issues.apache.org/jira/browse/FLINK-3837 Project: Flink Issue Type: Sub-task Compone

Re: Eclipse Problems

I guess, removing .fromElements(Object..) would fix the problem. Not sure so, if we can remove the method due to API stability... I don't see any other good solution (even if the current implementation gives a nice behavior by accident...): If you have a complex class hierarchy, it would be quite

[jira] [Created] (FLINK-3836) Change Histogram to enable Long counters

Maximilian Bode created FLINK-3836: -- Summary: Change Histogram to enable Long counters Key: FLINK-3836 URL: https://issues.apache.org/jira/browse/FLINK-3836 Project: Flink Issue Type: Improv

Re: [DISCUSS] Release Flink 1.0.3

Just pushed the fix. 2016-04-27 15:16 GMT+02:00 Fabian Hueske : > I have an easy fix for FLINK-3835 coming up. > It's not an urgent fix but if you wait for 1h it is in. > > 2016-04-27 14:57 GMT+02:00 Ufuk Celebi : > >> The issues mentioned in this thread have been resolved now. Are there >> any o

Re: Backporting a patch to a release branch

Sure, porting it is not an issue. At my current status I actually have to port it to the master (as I've written it based on the 1.0.x release branch), but fortunately it's not a lot of code. Thanks for the feedback as well! On Wed, Apr 27, 2016 at 3:35 PM, Maximilian Michels wrote: > Hi Stefano

Re: Backporting a patch to a release branch

Hi Stefano, It is a bit unfortunate for you that you have to spend additional time to port your code to a different branch. A PR is the way to go if a commit can't be simply cherry-picked. If your changes are too major then it is probably good idea just to include them in the next major release.

Re: Master test stability poor

+1 for making an effort to tackle test stability problems and potential involved bugs. On Wed, Apr 27, 2016 at 2:13 PM, Ufuk Celebi wrote: > @Max: I think you wanted to look into whether we can use Apache's > Jenkins server for our builds instead of Travis. Did you ever get > around at looking in

Re: Backporting a patch to a release branch

Ok, thanks for the feedback. On Wed, Apr 27, 2016 at 3:16 PM, Till Rohrmann wrote: > Hi Stefano, > > in this case I think it's best if you opened a PR against the release > branch so that a committer can pull it in. > > Cheers, > Till > > On Wed, Apr 27, 2016 at 3:03 PM, Stefano Baghino < > stef

Re: Backporting a patch to a release branch

Hi Stefano, in this case I think it's best if you opened a PR against the release branch so that a committer can pull it in. Cheers, Till On Wed, Apr 27, 2016 at 3:03 PM, Stefano Baghino < stefano.bagh...@radicalbit.io> wrote: > Hi Ufuk, > > thanks for getting back to me, I understand. > The pr

Re: [DISCUSS] Release Flink 1.0.3

I have an easy fix for FLINK-3835 coming up. It's not an urgent fix but if you wait for 1h it is in. 2016-04-27 14:57 GMT+02:00 Ufuk Celebi : > The issues mentioned in this thread have been resolved now. Are there > any other fixes you want in for 1.0.3? > > – Ufuk > > On Mon, Apr 25, 2016 at 3:3

Re: Eclipse Problems

You’re completely right Mathias. The compiler shouldn’t allow something like env.fromElements(SubClass.class, new ParentClass()) if it weren’t for the overloaded method. Thus, the test case is somewhat bogus. I’m actually wondering why the initial problem https://issues.apache.org/jira/browse/FLIN

Re: Backporting a patch to a release branch

Hi Ufuk, thanks for getting back to me, I understand. The problem with this patch in particular is that the code would be slightly different between the 1.0.x and 1.1.x, so cherry-picking the commit from the master branch to the release branch wouldn't be a viable option, unfortunately. In this ca

Re: [DISCUSS] Release Flink 1.0.3

The issues mentioned in this thread have been resolved now. Are there any other fixes you want in for 1.0.3? – Ufuk On Mon, Apr 25, 2016 at 3:34 PM, Ufuk Celebi wrote: > I think FLINK-3803 can be included as well for 1.0.3 (checkpoint stats > tracker configuration was not picked up). > > The PR

Re: Backporting a patch to a release branch

Hi, I think what we did for now is this: When merging a PR onto the master that we also consider to be required in a future release of an older release branch we cherry-pick it there and make the required changes. This only works for committers, of course, since we can just do that. I think in you

Re: Master test stability poor

Filed an issue with INFRA: https://issues.apache.org/jira/browse/INFRA-11773 @Robert: I agree, but still we see failing builds over and over again. At best it is annoying, at worst it "hides" new bugs being introduced. On Wed, Apr 27, 2016 at 2:41 PM, Till Rohrmann wrote: > That is good to hear

[jira] [Created] (FLINK-3835) JSON execution plan not helpful to debug plans with KeySelectors

Fabian Hueske created FLINK-3835: Summary: JSON execution plan not helpful to debug plans with KeySelectors Key: FLINK-3835 URL: https://issues.apache.org/jira/browse/FLINK-3835 Project: Flink

Re: Master test stability poor

That is good to hear that we can so easily solve most of the failing builds. We should then iterate over the open test-stability issues to see whether they are still valid after we've merged PR 1915. On Wed, Apr 27, 2016 at 2:25 PM, Robert Metzger wrote: > I'm not sure if the issues is as big as

Re: Master test stability poor

I'm not sure if the issues is as big as it seems on a first sight. The reason why all the builds of master are red on travis is that the cache of the 5th build is invalid. We have to ask infra to delete the caches and then they'll be green again. On Wed, Apr 27, 2016 at 2:13 PM, Ufuk Celebi wrote

Re: Master test stability poor

Along the lines of what Greg already mentioned, I would like to re-iterate that Travis is often a problem too: - long build times and we are reaching the time limit - unreliable I/O - unreliable resolving of build dependencies @Max: I think you wanted to look into whether we can use Apache's Jenki

Re: Master test stability poor

We just issued a PR about this (FLINK-1827 - https://github.com/apache/ flink/pull/1915) that improves test stability (and allow to skip entirely their compilation when it's not required) except for the ml library that has still some one error to solve ( in the hadoop-1 build and in the ml-library

Re: Master test stability poor

We have also started running over Travis' 2 hour limit for the longest build. Greg > On Apr 27, 2016, at 7:53 AM, Ufuk Celebi wrote: > > Hi Till, > > thank you for bringing this up. We really need to fix this. > > Filing JIRAs with critical priority was how we tried to solve it in > the past

Re: Master test stability poor

Hi Till, thank you for bringing this up. We really need to fix this. Filing JIRAs with critical priority was how we tried to solve it in the past, but obviously it did not work. There seems to be a mismatch between assigned and actual priorities. As a first step, I would volunteer to gather a li

Backporting a patch to a release branch

I'm currently working on FLINK-3239 (support Kerberos on the Kafka connector). I almost have a working prototype, however now I have a doubt regarding how to properly merge my code (when done): right now I'm working on a branch out of the 1.0.x re

Re: Eclipse Problems

Hi Till, but StreamExecutionEnvironmentTest.fromElementWithBaseTypeTest2 does not test was you describe -- even if it is intended to test it. It would test your describe scenario, if fromElements(Class, X...) would be called, But this call is not possible because X is defined a type Subclass and

Re: Read JSON file as input

I have my Apache Flink program: import org.apache.flink.api.scala._import scala.util.parsing.json._ object numHits extends App { val env = ExecutionEnvironment.getExecutionEnvironment val data=env.readTextFile("file:///path/to/json/file") val j=data.map { x => ("\"\"\""+x+"\"\"\"") }

Master test stability poor

Hi Flink community, I just wanted to raise awareness that in the last 16 days there was just a single Travis build of master which passed all tests. This indicates that we have some serious problems with our test stability or even worse a problem with the master itself. Having an unstable master m

[jira] [Created] (FLINK-3831) flink-streaming-java

Till Rohrmann created FLINK-3831: Summary: flink-streaming-java Key: FLINK-3831 URL: https://issues.apache.org/jira/browse/FLINK-3831 Project: Flink Issue Type: Sub-task Reporter:

[jira] [Created] (FLINK-3832) flink-streaming-scala

Till Rohrmann created FLINK-3832: Summary: flink-streaming-scala Key: FLINK-3832 URL: https://issues.apache.org/jira/browse/FLINK-3832 Project: Flink Issue Type: Sub-task Reporter

[jira] [Created] (FLINK-3833) flink-test-utils

Till Rohrmann created FLINK-3833: Summary: flink-test-utils Key: FLINK-3833 URL: https://issues.apache.org/jira/browse/FLINK-3833 Project: Flink Issue Type: Sub-task Reporter: Til

[jira] [Created] (FLINK-3834) flink-statebackend-rocksdb

Till Rohrmann created FLINK-3834: Summary: flink-statebackend-rocksdb Key: FLINK-3834 URL: https://issues.apache.org/jira/browse/FLINK-3834 Project: Flink Issue Type: Sub-task Rep

[jira] [Created] (FLINK-3828) flink-runtime

Till Rohrmann created FLINK-3828: Summary: flink-runtime Key: FLINK-3828 URL: https://issues.apache.org/jira/browse/FLINK-3828 Project: Flink Issue Type: Sub-task Reporter: Till R

[jira] [Created] (FLINK-3830) flink-scala

Till Rohrmann created FLINK-3830: Summary: flink-scala Key: FLINK-3830 URL: https://issues.apache.org/jira/browse/FLINK-3830 Project: Flink Issue Type: Sub-task Reporter: Till Roh

[jira] [Created] (FLINK-3829) flink-java

Till Rohrmann created FLINK-3829: Summary: flink-java Key: FLINK-3829 URL: https://issues.apache.org/jira/browse/FLINK-3829 Project: Flink Issue Type: Sub-task Reporter: Till Rohr

[jira] [Created] (FLINK-3827) Flink modules include unused dependencies

Till Rohrmann created FLINK-3827: Summary: Flink modules include unused dependencies Key: FLINK-3827 URL: https://issues.apache.org/jira/browse/FLINK-3827 Project: Flink Issue Type: Improveme

[jira] [Created] (FLINK-3826) Broken test: StreamCheckpointingITCase

Kostas Kloudas created FLINK-3826: - Summary: Broken test: StreamCheckpointingITCase Key: FLINK-3826 URL: https://issues.apache.org/jira/browse/FLINK-3826 Project: Flink Issue Type: Bug

Re: Eclipse Problems

Thanks for looking into this problem Mathias. I think the Scala test should be fixed as you've proposed. Concerning the StreamExecutionEnvironmentTest.fromElementWithBaseTypeTest2, I think it shouldn't be changed. The reason is that the class defines the common base class of the elements. And the

Re: Read JSON file as input

I just tried it and it still cannot parse it. It still takes the input as a dataset object rather than a string. On Wed, Apr 27, 2016 at 12:36 PM, Punit Naik wrote: > Okay Thanks a lot Fabian! > > On Wed, Apr 27, 2016 at 12:34 PM, Fabian Hueske wrote: > >> You should do the parsing in a Map ope

Re: Read JSON file as input

Okay Thanks a lot Fabian! On Wed, Apr 27, 2016 at 12:34 PM, Fabian Hueske wrote: > You should do the parsing in a Map operator. Map applies the MapFunction to > each element in the DataSet. > So you can either implement another MapFunction or extend the one you have > to call the JSON parser. >

Re: Read JSON file as input

You should do the parsing in a Map operator. Map applies the MapFunction to each element in the DataSet. So you can either implement another MapFunction or extend the one you have to call the JSON parser. 2016-04-27 6:40 GMT+02:00 Punit Naik : > Hi > > So I managed to do the map part. I stuc with