[jira] [Created] (FLINK-10578) Support writable state in QueryableStateClient

2018-10-16 Thread Jiayi Liao (JIRA)
Jiayi Liao created FLINK-10578: -- Summary: Support writable state in QueryableStateClient Key: FLINK-10578 URL: https://issues.apache.org/jira/browse/FLINK-10578 Project: Flink Issue Type: New Fe

Re: [DISCUSS] [Contributing] (3) - Review Tooling

2018-10-16 Thread Jin Sun
+1 On Tue, Oct 16, 2018 at 7:51 PM Tzu-Li Chen wrote: > Hi Fabian, > > +1 for your proposal. > > For the downside, I think after adding the review process template, > the pull request template would be high level into 3 parts: > > 1. Greeting and community guiding. > 2. User completed template.

Re: [DISCUSS] [Contributing] (3) - Review Tooling

2018-10-16 Thread Tzu-Li Chen
Hi Fabian, +1 for your proposal. For the downside, I think after adding the review process template, the pull request template would be high level into 3 parts: 1. Greeting and community guiding. 2. User completed template. 3. Reviewer complete template. If we can visually separate them, i.e.,

[jira] [Created] (FLINK-10577) CEP's greedy() doesn't work

2018-10-16 Thread zhanghao (JIRA)
zhanghao created FLINK-10577: Summary: CEP's greedy() doesn't work Key: FLINK-10577 URL: https://issues.apache.org/jira/browse/FLINK-10577 Project: Flink Issue Type: Bug Components: CEP

Re: [DISCUSS] [Contributing] (3) - Review Tooling

2018-10-16 Thread vino yang
+1, Agree to use the PR template. Fabian Hueske 于2018年10月17日周三 上午12:48写道: > Hi everyone, > > Instead of manually adding the review progress template as a comment to > every new PR, we could also append it to the PR description template. > The benefits would be: > + no need to add it manually si

[jira] [Created] (FLINK-10576) Introduce Machine/Node/TM health management

2018-10-16 Thread JIN SUN (JIRA)
JIN SUN created FLINK-10576: --- Summary: Introduce Machine/Node/TM health management Key: FLINK-10576 URL: https://issues.apache.org/jira/browse/FLINK-10576 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Release 1.5.5 and 1.6.2

2018-10-16 Thread Till Rohrmann
My concern was that it changes Flink's behaviour in a non trivial way which could break existing setups. It is true that Flink's exactly once guarantees are thwarted by that, though. The only problematic case is when users delete the savepoint files without the corresponding meta state savepoint f

[jira] [Created] (FLINK-10575) Remove deprecated ExecutionGraphBuilder.buildGraph method

2018-10-16 Thread JIN SUN (JIRA)
JIN SUN created FLINK-10575: --- Summary: Remove deprecated ExecutionGraphBuilder.buildGraph method Key: FLINK-10575 URL: https://issues.apache.org/jira/browse/FLINK-10575 Project: Flink Issue Type: S

[jira] [Created] (FLINK-10574) Update documents of new batch job failover strategy

2018-10-16 Thread JIN SUN (JIRA)
JIN SUN created FLINK-10574: --- Summary: Update documents of new batch job failover strategy Key: FLINK-10574 URL: https://issues.apache.org/jira/browse/FLINK-10574 Project: Flink Issue Type: Sub-tas

[jira] [Created] (FLINK-10573) Support task revocation

2018-10-16 Thread JIN SUN (JIRA)
JIN SUN created FLINK-10573: --- Summary: Support task revocation Key: FLINK-10573 URL: https://issues.apache.org/jira/browse/FLINK-10573 Project: Flink Issue Type: Sub-task Components: JobM

[jira] [Created] (FLINK-10572) Enable Per-job level failover strategy.

2018-10-16 Thread JIN SUN (JIRA)
JIN SUN created FLINK-10572: --- Summary: Enable Per-job level failover strategy. Key: FLINK-10572 URL: https://issues.apache.org/jira/browse/FLINK-10572 Project: Flink Issue Type: Sub-task

Re: Sharing state between subtasks

2018-10-16 Thread Jamie Grier
Here's a doc I started describing some changes we would like to make starting with the Kinesis Source.. It describes a refactoring of that code specifically and also hopefully a pattern and some reusable code we can use in the other sources as well. The end goal would be best-effort event-time syn

[jira] [Created] (FLINK-10571) Remove support for topologies

2018-10-16 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-10571: Summary: Remove support for topologies Key: FLINK-10571 URL: https://issues.apache.org/jira/browse/FLINK-10571 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-10570) State grows unbounded when "within" constraint not applied

2018-10-16 Thread Thomas Wozniakowski (JIRA)
Thomas Wozniakowski created FLINK-10570: --- Summary: State grows unbounded when "within" constraint not applied Key: FLINK-10570 URL: https://issues.apache.org/jira/browse/FLINK-10570 Project: Flin

Re: [DISCUSS] [Contributing] (3) - Review Tooling

2018-10-16 Thread Fabian Hueske
Hi everyone, Instead of manually adding the review progress template as a comment to every new PR, we could also append it to the PR description template. The benefits would be: + no need to add it manually since it is automatically added to each PR + the template is versioned in the Flink Git rep

Re: Creating a slide set for a Flink intro talk

2018-10-16 Thread Fabian Hueske
Hi everybody, Thank you for your encouraging feedback and your valuable comments. As you've probably noticed, the slides are not done yet and need more work. I'll continue to complete and refine them in the next days. Once the slides are in a better shape, I'll let you know. Best, Fabian Am Di.,

[jira] [Created] (FLINK-10569) Clean up uses of Scheduler and Instance in valid tests

2018-10-16 Thread TisonKun (JIRA)
TisonKun created FLINK-10569: Summary: Clean up uses of Scheduler and Instance in valid tests Key: FLINK-10569 URL: https://issues.apache.org/jira/browse/FLINK-10569 Project: Flink Issue Type: Su

Re: [DISCUSS] Release 1.5.5 and 1.6.2

2018-10-16 Thread Aljoscha Krettek
Hi, I think the "savepoints for recovery" change is a fix for likely violation of exactly-once guarantees while it has close to zero downsides in real-world use cases. Therefore I think we should not revert it and release the fix for 1.5.5 and 1.6.2. Best, Aljoscha > On 16. Oct 2018, at 13:48

[jira] [Created] (FLINK-10568) Show correct time attributes when calling Table.getSchema()

2018-10-16 Thread Timo Walther (JIRA)
Timo Walther created FLINK-10568: Summary: Show correct time attributes when calling Table.getSchema() Key: FLINK-10568 URL: https://issues.apache.org/jira/browse/FLINK-10568 Project: Flink

[jira] [Created] (FLINK-10567) Lost serialize fields when ttl state store with the mutable serializer

2018-10-16 Thread aitozi (JIRA)
aitozi created FLINK-10567: -- Summary: Lost serialize fields when ttl state store with the mutable serializer Key: FLINK-10567 URL: https://issues.apache.org/jira/browse/FLINK-10567 Project: Flink I

Re: Spreading Tasks across TaskManagers

2018-10-16 Thread Piotr Nowojski
Hi, > In general, I'm not 100% sure whether spreading out tasks is always the > best strategy. Especially if you have a network heavy job co-locating tasks > on the same TM could have benefits over spreading the tasks out. Definitely spreading out tasks is not always the best, but I would guess t

[jira] [Created] (FLINK-10566) Flink Planning is exponential in the number of stages

2018-10-16 Thread Robert Bradshaw (JIRA)
Robert Bradshaw created FLINK-10566: --- Summary: Flink Planning is exponential in the number of stages Key: FLINK-10566 URL: https://issues.apache.org/jira/browse/FLINK-10566 Project: Flink I

Re: Spreading Tasks across TaskManagers

2018-10-16 Thread Till Rohrmann
Yes, the ResourceSpec is not yet fully functional. The idea is to allow the user to specify how many resources an operator needs. Depending on these requirements, the RM should allocate slots which can fulfill these requirements. Cheers, Till On Tue, Oct 16, 2018 at 2:29 PM Maximilian Michels wr

[jira] [Created] (FLINK-10565) Refactor SchedulerTestBase

2018-10-16 Thread TisonKun (JIRA)
TisonKun created FLINK-10565: Summary: Refactor SchedulerTestBase Key: FLINK-10565 URL: https://issues.apache.org/jira/browse/FLINK-10565 Project: Flink Issue Type: Sub-task Components:

Re: Spreading Tasks across TaskManagers

2018-10-16 Thread Maximilian Michels
the community is currently working on Flink's scheduler component [1] That sounds great! I agree that spreading tasks across the nodes is not always desirable but it would be nice to give users an option to provide hints to the scheduler. The location aware bulk scheduling you mentioned would b

Re: [DISCUSS] Release 1.5.5 and 1.6.2

2018-10-16 Thread Chesnay Schepler
I agree that these change the behavior of Flink and could cause issues for users, hence I would be in favor of reverting said changes. On 16.10.2018 13:46, Till Rohrmann wrote: Sorry for the late notification, but I think we have some changes in the release branches which we might consider for

Re: [DISCUSS] Release 1.5.5 and 1.6.2

2018-10-16 Thread Till Rohrmann
Sorry for the late notification, but I think we have some changes in the release branches which we might consider for reverting. 1) Savepoints being considered for recovery [1] The problem is that we change Flink's savepoint contract in the sense that savepoint's are no longer exclusively under t

[jira] [Created] (FLINK-10564) tm costs too much time to communicate jm

2018-10-16 Thread chenlf (JIRA)
chenlf created FLINK-10564: -- Summary: tm costs too much time to communicate jm Key: FLINK-10564 URL: https://issues.apache.org/jira/browse/FLINK-10564 Project: Flink Issue Type: Bug Repo

[jira] [Created] (FLINK-10563) Expose shaded Presto S3 filesystem under "s3p" scheme

2018-10-16 Thread Aljoscha Krettek (JIRA)
Aljoscha Krettek created FLINK-10563: Summary: Expose shaded Presto S3 filesystem under "s3p" scheme Key: FLINK-10563 URL: https://issues.apache.org/jira/browse/FLINK-10563 Project: Flink

[jira] [Created] (FLINK-10562) Relax (or document) table name constraints

2018-10-16 Thread Flavio Pompermaier (JIRA)
Flavio Pompermaier created FLINK-10562: -- Summary: Relax (or document) table name constraints Key: FLINK-10562 URL: https://issues.apache.org/jira/browse/FLINK-10562 Project: Flink Issue

Contribution to FlinkML

2018-10-16 Thread Alejandro Alcalde
Hello all I've been some time developing a library for data preprocessing in Flink. I reach out to you because this library is almost finished and this month I will be submitting a paper to a journal (pre-print available at arxiv: https://arxiv.org/abs/1810.06021) I've checked Flink's roadmap (

Re: Add checkpoint hooks to window triggers

2018-10-16 Thread Fabian Hueske
Hi Paul, I guess it depends on your use case whether the transactional sink "needs to cooperate with a trigger that fires right before each checkpoint". You could use a regular Trigger that fires every minute (in processing time) and configure checkpointing to an interval of 1 minute. In worst-cas

[jira] [Created] (FLINK-10561) Flink doesn't start locally if remove log package

2018-10-16 Thread Aleksei Starikov (JIRA)
Aleksei Starikov created FLINK-10561: Summary: Flink doesn't start locally if remove log package Key: FLINK-10561 URL: https://issues.apache.org/jira/browse/FLINK-10561 Project: Flink Iss