Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-10-07 Thread Matthias Pohl
I find the ratchet feature you're suggesting interesting. But Arvid has a point referring to the blog post about ignoring revisions in git blame [1]. Adding the configuration file for commits to ignore revs as proposed in the blog post makes it even easier. One problem I see is that this is not sup

[jira] [Created] (FLINK-19518) Duration of running job is shown as 0 in web UI

2020-10-07 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-19518: -- Summary: Duration of running job is shown as 0 in web UI Key: FLINK-19518 URL: https://issues.apache.org/jira/browse/FLINK-19518 Project: Flink Issue Typ

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-10-07 Thread Chesnay Schepler
To me, ratchet seems to combine the worst aspects of both approaches. You get disruptive changes, but only in singular files, for something mundane as a typo fix or import change, which would be annoying to keep separate from the actual functional changes in a PR. On 10/7/2020 10:04 AM, Matthia

Re: IllegalStateException from incompatible state type - better exception type?

2020-10-07 Thread Till Rohrmann
Hi Hwanju, thanks for starting this discussion. I pretty much like the idea to be able to distinguish between user code faults and framework faults. This also helps in deciding on the recovery action to take. Hence, I would be +1 for using certain exceptions consistently in order to indicate the o

[jira] [Created] (FLINK-19519) Add support for port range in taskmanager.data.port

2020-10-07 Thread Truong Duc Kien (Jira)
Truong Duc Kien created FLINK-19519: --- Summary: Add support for port range in taskmanager.data.port Key: FLINK-19519 URL: https://issues.apache.org/jira/browse/FLINK-19519 Project: Flink Iss

[jira] [Created] (FLINK-19520) Add reliable test randomization for checkpointing

2020-10-07 Thread Arvid Heise (Jira)
Arvid Heise created FLINK-19520: --- Summary: Add reliable test randomization for checkpointing Key: FLINK-19520 URL: https://issues.apache.org/jira/browse/FLINK-19520 Project: Flink Issue Type: T

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-10-07 Thread Kostas Kloudas
Hi all, +1 for enforcing "a" codestyle using "a" tool. As the project grows both in terms of LOCs and contributors, this becomes more and more important as it eliminates some potential points of friction without any additional effort. >From the discussion, I am leaning towards having a single co

[jira] [Created] (FLINK-19521) Support dynamic properties on DefaultCLI

2020-10-07 Thread Aljoscha Krettek (Jira)
Aljoscha Krettek created FLINK-19521: Summary: Support dynamic properties on DefaultCLI Key: FLINK-19521 URL: https://issues.apache.org/jira/browse/FLINK-19521 Project: Flink Issue Type:

Re: [ANNOUNCE] New PMC member: Zhu Zhu

2020-10-07 Thread Zhu Zhu
Thank you all! It's my great pleasure to work with you to continuously improve Flink and grow the community! Thanks, Zhu Matthias Pohl 于2020年10月6日周二 下午9:39写道: > Congratulations! :) > > Best, > Matthias > > On Tue, Oct 6, 2020 at 10:55 AM Xingbo Huang wrote: > > > Congratulations Zhu Zhu > > >

Re: autoCommit for postgres jdbc streaming in Table/SQL API

2020-10-07 Thread Dylan Forciea
I hadn’t heard a response on this, so I’m going to expand this to the dev email list. If this is indeed an issue and not my misunderstanding, I have most of a patch already coded up. Please let me know, and I can create a JIRA issue and send out a PR. Regards, Dylan Forciea Oseberg From: Dyla

Re: autoCommit for postgres jdbc streaming in Table/SQL API

2020-10-07 Thread Dylan Forciea
I appreciate it! Let me know if you want me to submit a PR against the issue after it is created. It wasn’t a huge amount of code, so it’s probably not a big deal if you wanted to redo it. Thanks, Dylan From: Shengkai Fang Date: Wednesday, October 7, 2020 at 9:06 AM To: Dylan Forciea Subject:

Re: autoCommit for postgres jdbc streaming in Table/SQL API

2020-10-07 Thread Till Rohrmann
Hi Dylan, thanks for reaching out to the Flink community and excuse our late response. I am not an expert for the Table API and its JDBC connector but what you describe sounds like a missing feature. Also given that FLINK-12198 enabled this feature for the JDBCInputFormat indicates that we might s

[jira] [Created] (FLINK-19522) Add ability to set auto commit on jdbc driver from Table/SQL API

2020-10-07 Thread Dylan Forciea (Jira)
Dylan Forciea created FLINK-19522: - Summary: Add ability to set auto commit on jdbc driver from Table/SQL API Key: FLINK-19522 URL: https://issues.apache.org/jira/browse/FLINK-19522 Project: Flink

Re: autoCommit for postgres jdbc streaming in Table/SQL API

2020-10-07 Thread Dylan Forciea
Ok, I have created FLINK-19522 describing the issue. I have the code I made so far checked in at https://github.com/apache/flink/compare/master...dforciea:FLINK-19522 but this only fixes the SQL API. It sounds like there may be another change needed for the Table API… I’ll look into that and se

Re: autoCommit for postgres jdbc streaming in Table/SQL API

2020-10-07 Thread Dylan Forciea
Actually…. It looks like what I did covers both cases. I’ll see about getting some unit tests and documentation updated. Dylan From: Dylan Forciea Date: Wednesday, October 7, 2020 at 11:47 AM To: Till Rohrmann , dev Cc: Shengkai Fang , "u...@flink.apache.org" , "j...@apache.org" , Leonard Xu

[jira] [Created] (FLINK-19523) Secrets should be filtered out from logs

2020-10-07 Thread Qinghui Xu (Jira)
Qinghui Xu created FLINK-19523: -- Summary: Secrets should be filtered out from logs Key: FLINK-19523 URL: https://issues.apache.org/jira/browse/FLINK-19523 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-19524) Google Season of Docs - FLIP 60

2020-10-07 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-19524: Summary: Google Season of Docs - FLIP 60 Key: FLINK-19524 URL: https://issues.apache.org/jira/browse/FLINK-19524 Project: Flink Issue Type: Bug Com

[jira] [Created] (FLINK-19525) Update Table API Overview page

2020-10-07 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-19525: Summary: Update Table API Overview page Key: FLINK-19525 URL: https://issues.apache.org/jira/browse/FLINK-19525 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-19527) Update SQL Pages

2020-10-07 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-19527: Summary: Update SQL Pages Key: FLINK-19527 URL: https://issues.apache.org/jira/browse/FLINK-19527 Project: Flink Issue Type: Sub-task Components: D

[jira] [Created] (FLINK-19528) Setup and SQL execution

2020-10-07 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-19528: Summary: Setup and SQL execution Key: FLINK-19528 URL: https://issues.apache.org/jira/browse/FLINK-19528 Project: Flink Issue Type: Sub-task Repo

[jira] [Created] (FLINK-19526) Update Table API specific section

2020-10-07 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-19526: Summary: Update Table API specific section Key: FLINK-19526 URL: https://issues.apache.org/jira/browse/FLINK-19526 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-19529) Table connectors docs

2020-10-07 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-19529: Summary: Table connectors docs Key: FLINK-19529 URL: https://issues.apache.org/jira/browse/FLINK-19529 Project: Flink Issue Type: Sub-task Componen

[jira] [Created] (FLINK-19530) Table Concepts Page

2020-10-07 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-19530: Summary: Table Concepts Page Key: FLINK-19530 URL: https://issues.apache.org/jira/browse/FLINK-19530 Project: Flink Issue Type: Sub-task Components

Re: IllegalStateException from incompatible state type - better exception type?

2020-10-07 Thread Kim, Hwanju
Hi Till, Thanks for the opinion and insight. I think your concern is valid, as still nontrivial number of issues would be somewhat subtle to confidently say it's framework or user error. Serialization-related issue is one of them, as even user-triggered issue leads to exceptions in many places

[jira] [Created] (FLINK-19531) Implement the `WriterOperator`

2020-10-07 Thread Guowei Ma (Jira)
Guowei Ma created FLINK-19531: - Summary: Implement the `WriterOperator` Key: FLINK-19531 URL: https://issues.apache.org/jira/browse/FLINK-19531 Project: Flink Issue Type: Sub-task Compo

[jira] [Created] (FLINK-19532) FlinkZooKeeperQuorumPeer ignores purge settings

2020-10-07 Thread M L (Jira)
M L created FLINK-19532: --- Summary: FlinkZooKeeperQuorumPeer ignores purge settings Key: FLINK-19532 URL: https://issues.apache.org/jira/browse/FLINK-19532 Project: Flink Issue Type: Bug Rep

[jira] [Created] (FLINK-19533) Add state assignment to channel state during rescaling

2020-10-07 Thread Arvid Heise (Jira)
Arvid Heise created FLINK-19533: --- Summary: Add state assignment to channel state during rescaling Key: FLINK-19533 URL: https://issues.apache.org/jira/browse/FLINK-19533 Project: Flink Issue Ty

Re: IllegalStateException from incompatible state type - better exception type?

2020-10-07 Thread Tzu-Li (Gordon) Tai
Hi, Thanks for the discussions so far. I agree that the use of the StateMigrationException and it's semantics is rather undefined and inconsistent as of the current state. On Thu, Oct 8, 2020 at 7:38 AM Kim, Hwanju wrote: > Hi Till, > > Thanks for the opinion and insight. I think your concern i