Re: [PROPOSAL] Structure the Flink Open Source Development

2016-06-01 Thread Vasiliki Kalavri
Hi, we could go for something like "sponsor" or "champion" :) I'm fine with the proposal. Good to see more than 1 person for both Gelly and Table API. cheers, -V. On 1 June 2016 at 05:46, Tzu-Li (Gordon) Tai wrote: > I'd like to be added to the Streaming Connectors component (already edited >

Re: [PROPOSAL] Structure the Flink Open Source Development

2016-06-01 Thread Aljoscha Krettek
Hi, I think maintainer is also fine if we clearly specify that they are not meant as dictators or gatekeepers of the component that they are responsible for. -Aljoscha On Wed, 1 Jun 2016 at 09:48 Vasiliki Kalavri wrote: > Hi, > > we could go for something like "sponsor" or "champion" :) > I'm f

Re: [PROPOSAL] Structure the Flink Open Source Development

2016-06-01 Thread Chesnay Schepler
I'd also go with maintainer. On 01.06.2016 10:32, Aljoscha Krettek wrote: Hi, I think maintainer is also fine if we clearly specify that they are not meant as dictators or gatekeepers of the component that they are responsible for. -Aljoscha On Wed, 1 Jun 2016 at 09:48 Vasiliki Kalavri wrote:

Re: [PROPOSAL] Structure the Flink Open Source Development

2016-06-01 Thread Aljoscha Krettek
Btw, in Jira, if we clean up our components we can also set a component Lead that would get notified of issues for that component. On Wed, 1 Jun 2016 at 10:43 Chesnay Schepler wrote: > I'd also go with maintainer. > > On 01.06.2016 10:32, Aljoscha Krettek wrote: > > Hi, > > I think maintainer is

baseurl not overridden in documentation's Jekyll serve for local previewing

2016-06-01 Thread Tzu-Li (Gordon) Tai
Hi all, Small problem here regarding locally previewing documentation using Jekyll serve. When the -p flag is used on docs/build_docs.sh for a local server to preview documentation, the intended behaviour of Jekyll is to override the baseurl setting in the main config docs/_config.yml with an emp

Re: baseurl not overridden in documentation's Jekyll serve for local previewing

2016-06-01 Thread Ufuk Celebi
Hey Tzu-Li! Which commit are you referring to exactly? We just changed something yesterday and the build docs command now makes sure to use a specific Jekyll version via Rubygems. The Jekyll version we use is 2.5.3. And with this it works as expected for me. Previously, we had issues with differen

Re: baseurl not overridden in documentation's Jekyll serve for local previewing

2016-06-01 Thread Tzu-Li (Gordon) Tai
Hi Ufuk, Turns out I was a few commits behind the latest master and just missed yesterday's changes on the Jekyll build. The latest changes works perfectly. Thanks :) Cheers, Gordon -- View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/baseurl-not-ove

[jira] [Created] (FLINK-4001) Add event time support to filesystem connector

2016-06-01 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-4001: - Summary: Add event time support to filesystem connector Key: FLINK-4001 URL: https://issues.apache.org/jira/browse/FLINK-4001 Project: Flink Issue Type: Ne

Re: [PROPOSAL] Structure the Flink Open Source Development

2016-06-01 Thread Ufuk Celebi
Thanks! I like the idea of renaming it. I'm fine with shepherd and I also like Vasia's suggestion "champion". I would like to add "Distributed checkpoints" as a separate component to track development for check- and savepoints. On Wed, Jun 1, 2016 at 10:59 AM, Aljoscha Krettek wrote: > Btw, i

HA Cluster restart behaviour

2016-06-01 Thread Gyula Fóra
Hi, I have noticed some strange behaviour on a streaming cluster running in HA mode. I have stopped a cluster with some deployed jobs (stop-cluster.sh without cancelling the jobs) and when I bring the cluster back up the jobs that were running before are restarted. Is this the expected behaviour

[jira] [Created] (FLINK-4002) [py] Improve testing infraestructure

2016-06-01 Thread Omar Alvarez (JIRA)
Omar Alvarez created FLINK-4002: --- Summary: [py] Improve testing infraestructure Key: FLINK-4002 URL: https://issues.apache.org/jira/browse/FLINK-4002 Project: Flink Issue Type: Bug Co

Re: HA Cluster restart behaviour

2016-06-01 Thread Ufuk Celebi
Yes, it's expected, but you are certainly not the first one to be confused by this behaviour. The reasoning behind the current behaviour is that we don't users accidentally removing jobs, which seems worse than requiring users to cancel manually. We thought about adding a flag to the start scripts

Re: HA Cluster restart behaviour

2016-06-01 Thread Márton Balassi
I also think that the current mechanism is weird. IMHO it makes sense to add the flag to both the start and stop scripts. On Wed, Jun 1, 2016 at 2:09 PM, Ufuk Celebi wrote: > Yes, it's expected, but you are certainly not the first one to be > confused by this behaviour. > > The reasoning behind

Re: HA Cluster restart behaviour

2016-06-01 Thread Gyula Fóra
So you mean that you don't want people accidentally remove all jobs by shutting down the cluster? I think it is a bigger risk that people will actually click the cancel button on the website by accident :D For me it would seem intuitive that when I stop the cluster it stops the jobs. Definitely a

Re: HA Cluster restart behaviour

2016-06-01 Thread Maximilian Michels
At the moment, the stop-cluster script simply sends a TERM signal to all processes using "kill". Shutting down the cluster cleanly is a bit more complicated and would block for a longer time. I think the current approach is the safest for most users. I agree that it would be nice to have an option

Client communication and akka.framesize

2016-06-01 Thread fschueler
Hi, we were running into the same problem that was described here[0] and are wondering if anyone is still working on/interested in this PR[1]. It would be great if this could be merged since we are collecting and distributing data of unknown sizes at the time of configuration and this leads to

Re: Client communication and akka.framesize

2016-06-01 Thread Maximilian Michels
Hi Felix, I don't think this PR will go in any time soon. Transferring data back to the client is not a very efficient anyways. Perhaps you could re-formulate your problem in a different way? Adjusting the framesize could also be an option: https://ci.apache.org/projects/flink/flink-docs-master/s

[jira] [Created] (FLINK-4003) Use intrinsics for MathUtils logarithms

2016-06-01 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4003: - Summary: Use intrinsics for MathUtils logarithms Key: FLINK-4003 URL: https://issues.apache.org/jira/browse/FLINK-4003 Project: Flink Issue Type: Improvement

Re: [PROPOSAL] Structure the Flink Open Source Development

2016-06-01 Thread Fabian Hueske
I think calling the role maintainer is not a good idea. The Spark community had a maintainer process which they just voted to remove. From my understanding, a maintainer in Spark had a more active role than the role we are currently discussing. I would prefer to not call the role "maintainer" to m

Re: [PROPOSAL] Structure the Flink Open Source Development

2016-06-01 Thread Chesnay Schepler
sounds like "Observer" would fit. On 01.06.2016 19:11, Fabian Hueske wrote: I think calling the role maintainer is not a good idea. The Spark community had a maintainer process which they just voted to remove. From my understanding, a maintainer in Spark had a more active role than the role we a

[jira] [Created] (FLINK-4004) Do not pass custom flink kafka connector properties to Kafka to avoid warnings

2016-06-01 Thread Robert Metzger (JIRA)
Robert Metzger created FLINK-4004: - Summary: Do not pass custom flink kafka connector properties to Kafka to avoid warnings Key: FLINK-4004 URL: https://issues.apache.org/jira/browse/FLINK-4004 Projec

Re: [PROPOSAL] Structure the Flink Open Source Development

2016-06-01 Thread Greg Hogan
Is "Observer" too passive? Maintainer -> Guide and/or Shepherd -> Reviewer? Are the component leads the first name in each list? If so, +1 from me :) On Wed, Jun 1, 2016 at 1:59 PM, Chesnay Schepler wrote: > sounds like "Observer" would fit. > > > On 01.06.2016 19:11, Fabian Hueske wrote: > >>

Re: [PROPOSAL] Structure the Flink Open Source Development

2016-06-01 Thread Stephan Ewen
I like the name "shepherd". It implies a non-authorative role, and implies guidance, which is very fitting. I also thing there is no problem with having a "component shepherd" and a "pull request shepherd". Stephan On Wed, Jun 1, 2016 at 7:11 PM, Fabian Hueske wrote: > I think calling the rol

[jira] [Created] (FLINK-4005) LocalExecutorITCase.testLocalExecutorWithWordCount fails

2016-06-01 Thread Ufuk Celebi (JIRA)
Ufuk Celebi created FLINK-4005: -- Summary: LocalExecutorITCase.testLocalExecutorWithWordCount fails Key: FLINK-4005 URL: https://issues.apache.org/jira/browse/FLINK-4005 Project: Flink Issue Type

[jira] [Created] (FLINK-4006) ExecutionGraph.restartStrategy field can't be serialized

2016-06-01 Thread ZhengBowen (JIRA)
ZhengBowen created FLINK-4006: - Summary: ExecutionGraph.restartStrategy field can't be serialized Key: FLINK-4006 URL: https://issues.apache.org/jira/browse/FLINK-4006 Project: Flink Issue Type:

[jira] [Created] (FLINK-4007) Flink Kafka Consumer throws Null Pointer Exception when using DataStream keyBy

2016-06-01 Thread Akshay Shingote (JIRA)
Akshay Shingote created FLINK-4007: -- Summary: Flink Kafka Consumer throws Null Pointer Exception when using DataStream keyBy Key: FLINK-4007 URL: https://issues.apache.org/jira/browse/FLINK-4007 Proj

[jira] [Created] (FLINK-4008) Hello,I am not able to create jar of flink-streaming-connectors ...I am able to create jar of others like twitter,kafka,flume but I am not able to create jar of flink-str

2016-06-01 Thread Akshay Shingote (JIRA)
Akshay Shingote created FLINK-4008: -- Summary: Hello,I am not able to create jar of flink-streaming-connectors ...I am able to create jar of others like twitter,kafka,flume but I am not able to create jar of flink-streaming connectors ?? How can