[jira] [Created] (FLINK-14165) HadoopRecoverableWriter do not support viewfs scheme

haoyuwen created FLINK-14165: Summary: HadoopRecoverableWriter do not support viewfs scheme Key: FLINK-14165 URL: https://issues.apache.org/jira/browse/FLINK-14165 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-14166) Reuse cache from previous history server run

David Moravek created FLINK-14166: - Summary: Reuse cache from previous history server run Key: FLINK-14166 URL: https://issues.apache.org/jira/browse/FLINK-14166 Project: Flink Issue Type: Im

[jira] [Created] (FLINK-14167) Move python-related scripts in flink-dist to flink-python

Wei Zhong created FLINK-14167: - Summary: Move python-related scripts in flink-dist to flink-python Key: FLINK-14167 URL: https://issues.apache.org/jira/browse/FLINK-14167 Project: Flink Issue Typ

Re: [DISCUSS] Contribute Pulsar Flink connector back to Flink

Okay, I see your point, Becket. Then let us prominently link the Pulsar connector from the Flink connector docs then, so that users can find it easily. As soon as FLIP 27 is done, we reach out the Pulsar folks to contribute a new connector. On Mon, Sep 23, 2019 at 3:11 AM Becket Qin wrote: > H

Re: [DISCUSS] FLIP-63: Rework table partition support

Thanks for your discussion on google document. Comments addressed and added FileSystem connector chapter, and introduce code prototype for file system connector to unify flink file system and hive connectors. Looking forward to your feedbacks. Thank you. Best, Jingsong Lee -

Re: Confluence permission for FLIP creation

Hi Forward, I gave you the permissions. Best, Fabian Am So., 22. Sept. 2019 um 13:03 Uhr schrieb Forward Xu < forwardxu...@gmail.com>: > Hi devs, > > I'd like to create a page about the Support SQL 2016-2017 JSON functions in > Flink SQL > < > https://docs.google.com/document/d/1JfaFYIFOAY8P2pF

[jira] [Created] (FLINK-14168) Remove unused BootstrapTools#generateTaskManagerConfiguration

Zhu Zhu created FLINK-14168: --- Summary: Remove unused BootstrapTools#generateTaskManagerConfiguration Key: FLINK-14168 URL: https://issues.apache.org/jira/browse/FLINK-14168 Project: Flink Issue Ty

[jira] [Created] (FLINK-14169) Cleanup expired jobs from history server

David Moravek created FLINK-14169: - Summary: Cleanup expired jobs from history server Key: FLINK-14169 URL: https://issues.apache.org/jira/browse/FLINK-14169 Project: Flink Issue Type: Improv

[jira] [Created] (FLINK-14170) Support hadoop < 2.7 with StreamingFileSink.BulkFormatBuilder

Bhagavan created FLINK-14170: Summary: Support hadoop < 2.7 with StreamingFileSink.BulkFormatBuilder Key: FLINK-14170 URL: https://issues.apache.org/jira/browse/FLINK-14170 Project: Flink Issue

Re: [DISCUSS] FLIP-57 - Rework FunctionCatalog

+1 for CREATE TEMPORARY SYSTEM FUNCTION xxx Cheers, Fabian Am Sa., 21. Sept. 2019 um 06:58 Uhr schrieb Bowen Li : > "SYSTEM" sounds good to me. FYI, this FLIP only impacts low level of the > SQL function stack and won't actually involve any DDL, thus I will just > document the decision and we sh

Re: Per Key Grained Watermark Support

Hi There was a discussion about this issue[1], as the previous discussion said at the moment this is not supported out of the box by Flink, I think you can try keyed process function as Lasse said. [1] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Watermark-for-each-key-td274

Re: [DISCUSS] FLIP-57 - Rework FunctionCatalog

"SYSTEM" sounds good to me too. Best, Jark On Mon, 23 Sep 2019 at 19:04, Fabian Hueske wrote: > +1 for CREATE TEMPORARY SYSTEM FUNCTION xxx > > Cheers, Fabian > > Am Sa., 21. Sept. 2019 um 06:58 Uhr schrieb Bowen Li >: > > > "SYSTEM" sounds good to me. FYI, this FLIP only impacts low level of

[jira] [Created] (FLINK-14171) Add 'SHOW USER FUNCTIONS' support in SQL CLI

Canbin Zheng created FLINK-14171: Summary: Add 'SHOW USER FUNCTIONS' support in SQL CLI Key: FLINK-14171 URL: https://issues.apache.org/jira/browse/FLINK-14171 Project: Flink Issue Type: New

Re: [DISCUSS] Add ARM CI build to Flink (information-only)

This sounds good Xiyuan. I'd also be in favour of running the ARM builds regularly as cron jobs and once we see that they are stable we could run them for every master commit. Hence, I'd say let's fix the above mentioned problems and then set the nightly cron job up. Cheers, Till On Fri, Sep 20,

Re: java8 lambdas and exceptions lead to compile error

If there is such a check, then I'd say let's enable it for the moment. Cheers, Till On Fri, Sep 20, 2019 at 1:50 PM zz wrote: > thanks for reply. "add some context/comment" is very necessary, but I am > not sure where to add to remind others for avoiding similar mistakes, so Is > that a better

Re: [DISCUSS] FLIP-66: Support time attribute in SQL DDL

Hi, Thanks Fabian for your reply. I agree with your point that the histogram-based case need the function to be stateful which is not supported currently and in this design. Maybe we can support stateful scalar function like TableAggregateFunction. We can further discuss how to support this in the

Re: [FLINK-14076] non-deserializable root cause in DeclineCheckpoint

Hi Jeffrey, thanks for reporting this issue and starting a discussion how to solve this problem. I've pulled in Piotr who is working on the checkpointing part of Flink. If a user generated exception can get reported, then we need to make sure that it is properly handled. Approach 2. would be easi

Re: [DISCUSS] Releasing Flink 1.9.1

+1 for the 1.9.1 release and for Jark being the RM. I'll help with the review of FLINK-14010. Cheers, Till On Mon, Sep 23, 2019 at 8:32 AM Debasish Ghosh wrote: > I hope https://issues.apache.org/jira/browse/FLINK-12501 will also be part > of 1.9.1 .. > > regards. > > On Mon, Sep 23, 2019 at 11

Re: Per Key Grained Watermark Support

Hi Congxian, Thanks but by doing that, we will lose some features like output of the late data. Original Message Sender: Congxian Qiu Recipient: Lasse Nedergaard Cc: 廖嘉逸; u...@flink.apache.org; dev@flink.apache.org Date: Monday, Sep 23, 2019 19:56 Subject: Re: Per Key Grained Watermark Suppo

Re: [FLINK-14076] non-deserializable root cause in DeclineCheckpoint

Hi, I guess the TaskManager should have logged the original exception somewhere (I’m not saying that we shouldn’t solve this, just to make sure that the basics are covered), so you should already be able to deduce the reason of failure, right? I think that option 2. would not only be easier, b

Re: [VOTE] FLIP-56: Dynamic Slot Allocation

@Till @Andrey According to the comments, I just updated the FLIP document [1], with the following changes: - Remove SlotID (in the section Protocol Changes) - Updated implementation steps to reduce separated code paths. As far as I can see at the moment, we do not need the feature option

Re: [VOTE] FLIP-56: Dynamic Slot Allocation

Thanks for updating the Flip. It looks good to me. +1 (binding) Cheers, Till On Mon, Sep 23, 2019 at 4:12 PM Xintong Song wrote: > @Till @Andrey > > According to the comments, I just updated the FLIP document [1], with the > following changes: > >- Remove SlotID (in the section Protocol Ch

[jira] [Created] (FLINK-14172) Implement KubeClient with official Java client library for kubernetes

Yang Wang created FLINK-14172: - Summary: Implement KubeClient with official Java client library for kubernetes Key: FLINK-14172 URL: https://issues.apache.org/jira/browse/FLINK-14172 Project: Flink

[jira] [Created] (FLINK-14173) ANSI-style JOIN with Temporal Table Function fails

Benoît Paris created FLINK-14173: Summary: ANSI-style JOIN with Temporal Table Function fails Key: FLINK-14173 URL: https://issues.apache.org/jira/browse/FLINK-14173 Project: Flink Issue Type

[jira] [Created] (FLINK-14174) Don't swallow exception when rethrowing type mismatches with side outputs

John Lonergan created FLINK-14174: - Summary: Don't swallow exception when rethrowing type mismatches with side outputs Key: FLINK-14174 URL: https://issues.apache.org/jira/browse/FLINK-14174 Project:

[VOTE] FLIP-68: Extend Core Table System with Modular Plugins

Hi all, I'd like to start a vote for FLIP-68 [1], since there's no more concern in the discussion thread [2] The vote will be open for minimum 3 days till 5:30pm UTC, Sep 26. Thanks, Bowen [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-68%3A+Extend+Core+Table+System+with+Modular+Plu

[VOTE] FLIP-57: Rework FunctionCatalog

Hi all, I'd like to start a voting thread for FLIP-57 [1], which we've reached consensus in [2]. This voting will be open for minimum 3 days till 6:30pm UTC, Sep 26. Thanks, Bowen [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-57%3A+Rework+FunctionCatalog [2] http://apache-flink-mai

Re: [DISCUSS] FLIP-57 - Rework FunctionCatalog

Thanks all for your input! I've updated FLIP-57 accordingly. To summarize the changes: - introduced new concept of "Temporary system functions", which has no namespace and override built-in functions - repositioned "temporary functions" to be those with namespaces and override catalog

Re: [DISCUSS] FLIP-63: Rework table partition support

Hi Jingsong, Thanks for driving this effort! Besides a few further comments on Catalog APIs that I just left, it LGTM. Not sure why, but the voting thread in gmail shows in the same thread as the discussion's. After addressing all the comments, could you start a new, separate thread to let other

Re: [DISCUSS] FLIP 69 - Flink SQL DDL Enhancement

Hi Terry, Thanks for driving the effort! I left some comments in the doc. AFAIU, the biggest motivation is to support DDLs in sql parser so that both Table API and SQL CLI can share the stack, despite that SQL CLI has already supported some commands itself. However, I don't see details on how SQL

[jira] [Created] (FLINK-14175) Upgrade KPL version in flink-connector-kinesis to fix application OOM

Abhilasha Seth created FLINK-14175: -- Summary: Upgrade KPL version in flink-connector-kinesis to fix application OOM Key: FLINK-14175 URL: https://issues.apache.org/jira/browse/FLINK-14175 Project: Fl

[COMMITTER] repo locked due to synchronization issues

Hi committers, Recently I've run a repo issue multiple times in different days. When I tried to push a commit to master, git reports the following error: ``` remote: This repository has been locked due to synchronization issues: remote: - /x1/gitbox/broken/flink.txt exists due to a previous erro

Re: [SURVEY] How many people are using customized RestartStrategy(s)

When we setup alert like "fullRestarts > 1" for some rolling window, we want to use counter. if it is a Gauge, "fullRestarts" will never go below 1 after a first full restart. So alert condition will always be true after first job restart. If we can apply a derivative to the Gauge value, I guess al

Re: [DISCUSS] FLIP-48: Pluggable Intermediate Result Storage

Hi Stephan, In terms of the performance concern, please see my understanding below. ## Breaking the pipeline v.s. adding a sink. If two operators are initially chained, they will belong to the same stage in the DAG and the same task, therefore the main processing path will just have one task with

Re: [DISCUSS] Contribute Pulsar Flink connector back to Flink

Thanks, Stephan. Sounds good to me. We can still try our best to get new Pulsar connector in Flink 1.10. In case we do not have time to do that, we will prominently link the Pulsar connector from the Flink connector docs. Thanks, Jiangjie (Becket) Qin On Mon, Sep 23, 2019 at 4:11 PM Stephan Ewe

[jira] [Created] (FLINK-14176) add taskmanager link in vertex‘s page of taskmanager

lining created FLINK-14176: -- Summary: add taskmanager link in vertex‘s page of taskmanager Key: FLINK-14176 URL: https://issues.apache.org/jira/browse/FLINK-14176 Project: Flink Issue Type: Improvem

[jira] [Created] (FLINK-14177) Bump Curator to 3.5.5

lamber-ken created FLINK-14177: -- Summary: Bump Curator to 3.5.5 Key: FLINK-14177 URL: https://issues.apache.org/jira/browse/FLINK-14177 Project: Flink Issue Type: Improvement Reporte

Re: [DISCUSS] FLIP-63: Rework table partition support

Thanks for you review, I will send a another vote thread from my apache email. Best, Jingsong Lee -- From:Bowen Li Send Time:2019年9月24日(星期二) 03:06 To:JingsongLee Cc:dev Subject:Re: [DISCUSS] FLIP-63: Rework table partition suppo

[VOTE] FLIP-63: Rework table partition support

Hi Flink devs, after another round of discussion. I would like to re-start the voting for FLIP-63 Rework table partition support. FLIP wiki:

[jira] [Created] (FLINK-14178) maven-shade-plugin 3.2.1 doesn't work on ARM for Flink

wangxiyuan created FLINK-14178: -- Summary: maven-shade-plugin 3.2.1 doesn't work on ARM for Flink Key: FLINK-14178 URL: https://issues.apache.org/jira/browse/FLINK-14178 Project: Flink Issue Type

Re: [VOTE] FLIP-57: Rework FunctionCatalog

+1 Best, Kurt On Tue, Sep 24, 2019 at 2:30 AM Bowen Li wrote: > Hi all, > > I'd like to start a voting thread for FLIP-57 [1], which we've reached > consensus in [2]. > > This voting will be open for minimum 3 days till 6:30pm UTC, Sep 26. > > Thanks, > Bowen > > [1] > > https://cwiki.apache.o

Re: [DISCUSS] Releasing Flink 1.9.1

+1 for the 1.9.1 release and for Jark being the RM. Thanks Jark for the volunteering. Best, Kurt On Mon, Sep 23, 2019 at 9:17 PM Till Rohrmann wrote: > +1 for the 1.9.1 release and for Jark being the RM. I'll help with the > review of FLINK-14010. > > Cheers, > Till > > On Mon, Sep 23, 2019 at

Re: [SURVEY] How many people are using customized RestartStrategy(s)

Steven, In my mind, Flink counter only stores its accumulated count and reports that value. Are you using an external counter directly? Maybe Flink Meter/MeterView is what you need? It stores the count and calculates the rate. And it will report its "count" as well as "rate" to external metric ser

Re: [DISCUSS] FLIP-66: Support time attribute in SQL DDL

For histogram-based watermark strategy, one possible solution is that we still use the stateless scalar function, and keep the stateful objects directly in the function. By doing that we will loose some information after the job get restarted, but I think it might acceptable because histogram-based

[jira] [Created] (FLINK-14179) Wrong description of SqlCommand.SHOW_FUNCTIONS

Canbin Zheng created FLINK-14179: Summary: Wrong description of SqlCommand.SHOW_FUNCTIONS Key: FLINK-14179 URL: https://issues.apache.org/jira/browse/FLINK-14179 Project: Flink Issue Type: Bu

Confluence permission for FLIP creation

Hi all, As communicated in an email thread, I'm proposing to support computed column in Flink SQL . I have a draft design doc that I'd like to convert it to a FLIP. Thus, it would be great if anyone who can grant me the write access to Confluence. My Confluence ID is danny0405. It would be nic

Re: [VOTE] FLIP-63: Rework table partition support

Looks like the wiki is not aligned with latest google doc, could you update it first? Best, Kurt On Tue, Sep 24, 2019 at 10:19 AM Jingsong Lee wrote: > Hi Flink devs, after another round of discussion. > > I would like to re-start the voting for FLIP-63 > Rework table partition support. > > FL

Re: [VOTE] FLIP-63: Rework table partition support

Thank you for your reminder. Updated. Best, Jingsong Lee On Tue, Sep 24, 2019 at 11:36 AM Kurt Young wrote: > Looks like the wiki is not aligned with latest google doc, could > you update it first? > > Best, > Kurt > > > On Tue, Sep 24, 2019 at 10:19 AM Jingsong Lee > wrote: > > > Hi Flink dev

[jira] [Created] (FLINK-14180) Enable config of maximum capacity of FileArchivedExecutionGraphStore.

Yingjie Cao created FLINK-14180: --- Summary: Enable config of maximum capacity of FileArchivedExecutionGraphStore. Key: FLINK-14180 URL: https://issues.apache.org/jira/browse/FLINK-14180 Project: Flink

Re: [DISCUSS] Releasing Flink 1.9.1

+1 for 1.9.1 release and Jark being the RM. Thanks Jark for kicking off this release and the volunteering. Regards, Dian > 在 2019年9月24日，上午10:45，Kurt Young 写道： > > +1 for the 1.9.1 release and for Jark being the RM. > Thanks Jark for the volunteering. > > Best, > Kurt > > > On Mon, Sep 23,

[jira] [Created] (FLINK-14181) TableEnvironment explain maybe causes StreamGraph to be inconsistent

jianlong miao created FLINK-14181: - Summary: TableEnvironment explain maybe causes StreamGraph to be inconsistent Key: FLINK-14181 URL: https://issues.apache.org/jira/browse/FLINK-14181 Project: Flink

Re: [DISCUSS] Releasing Flink 1.9.1

Thanks Till for reviewing FLINK-14010. Hi Jeff, I think it makes sense to merge FLINK-13708 before the release (PR has been reviewed). Hi Debasish, FLINK-12501 has already been merged in 1.10.0. I'm fine to cherry-pick it to 1.9 if we have a consensus this issue could be viewed as a bug. We can c

Re: [DISCUSS] Releasing Flink 1.9.1

+1 for the 1.9.1 release and for Jark being the RM. Thanks Jark for driving on this. Best, Terry Wang > 在 2019年9月24日，下午2:19，Jark Wu 写道： > > Thanks Till for reviewing FLINK-14010. > > Hi Jeff, I think it makes sense to merge FLINK-13708 before the release (PR > has been reviewed). > > Hi Deb