[jira] [Created] (FLINK-27569) Terminated Flink job restarted from empty state when execution.shutdown-on-application-finish is false

2022-05-11 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-27569: -- Summary: Terminated Flink job restarted from empty state when execution.shutdown-on-application-finish is false Key: FLINK-27569 URL: https://issues.apache.org/jira/browse/FLINK-27569

Re: Flink job restarted from empty state when execution.shutdown-on-application-finish is false

2022-05-11 Thread Gyula Fóra
I have opened a JIRA: https://issues.apache.org/jira/browse/FLINK-27569 with more details, logs and screenshots. Gyula On Wed, May 11, 2022 at 7:03 AM Gyula Fóra wrote: > Hi Devs! > > I ran into a concerning situation and would like to hear your thoughts on > this. > > I am running Flink 1.15 o

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-05-11 Thread Becket Qin
Hi Sebastian, Thanks for the reply and patient discussion. I agree this is a tricky decision. > Nevertheless, Piotr has valid concerns about Option c) which I see as > follows: > (1) An interface with default NOOP implementation makes the implementation > optional. And in my opinion, a default i

Re: [ANNOUNCE] Apache Flink Table Store 0.1.0 released

2022-05-11 Thread Becket Qin
Really excited to see the very first release of the flink-table-store! Kudos to everyone who helped with this effort! Cheers, Jiangjie (Becket) Qin On Wed, May 11, 2022 at 1:55 PM Jingsong Lee wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink Table S

Re: Flink job restarted from empty state when execution.shutdown-on-application-finish is enabled

2022-05-11 Thread Yang Wang
I assume this is the responsibility of job result store[1]. However, it seems that it does not work as expected. [1]. https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=195726435 Best, Yang Gyula Fóra 于2022年5月11日周三 12:55写道: > Sorry I messed up the email, I meant false . > > So wh

Re: Re: [ANNOUNCE] New Flink PMC member: Yang Wang

2022-05-11 Thread Yang Wang
Thanks for your warm welcome. It is my pleasure to work in such a nice community. Best, Yang Thomas Weise 于2022年5月11日周三 00:10写道: > Congratulations, Yang! > > On Tue, May 10, 2022 at 3:15 AM Márton Balassi > wrote: > > > > Congrats, Yang. Well deserved :-) > > > > On Tue, May 10, 2022 at 9:1

[jira] [Created] (FLINK-27570) A checkpoint path error does not cause the job to stop

2022-05-11 Thread Underwood (Jira)
Underwood created FLINK-27570: - Summary: A checkpoint path error does not cause the job to stop Key: FLINK-27570 URL: https://issues.apache.org/jira/browse/FLINK-27570 Project: Flink Issue Type:

[jira] [Created] (FLINK-27571) Recognize "less is better" benchmarks in regression detection script

2022-05-11 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-27571: - Summary: Recognize "less is better" benchmarks in regression detection script Key: FLINK-27571 URL: https://issues.apache.org/jira/browse/FLINK-27571 Projec

[jira] [Created] (FLINK-27572) Verify HA Metadata present before performint last-state restore

2022-05-11 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-27572: -- Summary: Verify HA Metadata present before performint last-state restore Key: FLINK-27572 URL: https://issues.apache.org/jira/browse/FLINK-27572 Project: Flink

Re: [DISCUSS ] HybridSouce Table & Sql api timeline

2022-05-11 Thread Ran Tao
Thanks, Thomas. I noticed this issue and asked Jiang for details. If he doesn't finish this work yet, I will draft the initial FLIP, otherwise we can wait for Jiang's implementation. Thomas Weise 于2022年5月11日周三 02:29写道: > fyi there is a related ticket here: > https://issues.apache.org/jira/brow

Re: [Discuss] Creating an Apache Flink slack workspace

2022-05-11 Thread Konstantin Knauf
I don't think we can maintain two additional channels. Some people have already concerns about covering one additional channel. I think, a forum provides a better user experience than a mailing list. Information is structured better, you can edit messages, sign up and search is easier. To make so

[jira] [Created] (FLINK-27573) Configuring a new random job result store directory

2022-05-11 Thread Matyas Orhidi (Jira)
Matyas Orhidi created FLINK-27573: - Summary: Configuring a new random job result store directory Key: FLINK-27573 URL: https://issues.apache.org/jira/browse/FLINK-27573 Project: Flink Issue T

[jira] [Created] (FLINK-27574) [QUESTION] In Flink k8s Application mode with HA can not using History Server for history backend

2022-05-11 Thread tanjialiang (Jira)
tanjialiang created FLINK-27574: --- Summary: [QUESTION] In Flink k8s Application mode with HA can not using History Server for history backend Key: FLINK-27574 URL: https://issues.apache.org/jira/browse/FLINK-27574

Re: [DISCUSS] FLIP-222: Support full query lifecycle statements in SQL client

2022-05-11 Thread Paul Lam
Hi Jark, Thanks a lot for your opinions and suggestions! Please see my replies inline. > 1) the display of savepoint_path Agreed. Adding it to the FLIP. > 2) Please make a decision on multiple options in the FLIP. Okay. I’ll keep one and move the other to the rejected alternatives section.

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-05-11 Thread Becket Qin
+dev Hi Sebastian, Thank you for the summary. Please see the detailed replies inline. As a recap of my suggestions. 1. Pausable splits API. a) Add default implementations to methods "pauseOrResumeSplits" in both SourceReader and SplitReader where both default implementations throw Unsupported

[jira] [Created] (FLINK-27575) Support PyFlink in Python3.9

2022-05-11 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-27575: Summary: Support PyFlink in Python3.9 Key: FLINK-27575 URL: https://issues.apache.org/jira/browse/FLINK-27575 Project: Flink Issue Type: New Feature

[QUESTION] In Flink k8s Application mode with HA can not using History Server for history backend

2022-05-11 Thread 谭家良
In Flink k8s application mode with high-availability, it's job id always 00, but in history server, it make job's id for the key. Can I modify the job id in HA? Maybe the History Server must keep the ClusterID and the JobID for it's key? Best, tanjialiang. | | tanjialiang | | tanjl

[jira] [Created] (FLINK-27576) Flink will request new pod when jm pod is delete, but will remove when TaskExecutor exceeded the idle timeout

2022-05-11 Thread zhisheng (Jira)
zhisheng created FLINK-27576: Summary: Flink will request new pod when jm pod is delete, but will remove when TaskExecutor exceeded the idle timeout Key: FLINK-27576 URL: https://issues.apache.org/jira/browse/FLINK-2

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-05-11 Thread Sebastian Mattheis
Hi Becket, Thanks a lot for your fast and detailed response. For me, it converges and dropping the supportsX method sounds very reasonable to me. (Side note: With "pausable splits" enabled as "default" I think we misunderstood. As you described now "default" I understand as that it should be the n

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-05-11 Thread Piotr Nowojski
Hi, Actually previously I thought about having a decorative interface and whenever watermark alignment is enabled, checking that the source implements the decorative interface. If not, throwing an exception. The option with default methods in the source interfaces throwing `UnsupportedOperationEx

Re: [QUESTION] In Flink k8s Application mode with HA can not using History Server for history backend

2022-05-11 Thread 谭家良
Sorry, I found this is not a the right place to ask some question. I'll ask question in the u...@flink.apache.org. | | 谭家良 | | tanjl_w...@126.com | On 5/11/2022 19:52,谭家良 wrote: In Flink k8s application mode with high-availability, it's job id always 00, but in history server, it make j

[DISCUSS] Release first version of Elasticsearch connector from external repository

2022-05-11 Thread Martijn Visser
Hi everyone, As mentioned in the previous update [1] we were working on the final step for moving out the Elasticsearch connector, which was related to the documentation integration. This has now been completed: the documentation you see on https://nightlies.apache.org/flink/flink-docs-master/docs

Re: [DISCUSS] Release first version of Elasticsearch connector from external repository

2022-05-11 Thread Konstantin Knauf
Hi Martijn, +1 to do a release which is compatible with Flink 1.5.x. With the release, we should add something like a compatibility matrix to the documentation, but I am sure that's already on your list. Cheers, Konstantin Am Mi., 11. Mai 2022 um 20:10 Uhr schrieb Martijn Visser < martijnvis

Incompatible data types while using firehose sink

2022-05-11 Thread Zain Haider Nemati
Hi Folks, Getting this error when sinking data to a firehosesink, would really appreciate some help ! DataStream inputStream = env.addSource(new FlinkKafkaConsumer<>("xxx", new SimpleStringSchema(), properties)); Properties sinkProperties = new Properties(); sinkProperties.put(AWSCon

Re: [DISCUSS] FLIP-226: Introduce Schema Evolution on Table Store

2022-05-11 Thread Jingsong Li
Hi all, If there are no more comments, I'm going to start a vote. Best, Jingsong On Tue, May 10, 2022 at 10:37 AM Jingsong Li wrote: > Hi Jark, > > Thanks for your feedback. > > > 1) Does table-store support evolve schemas multiple times during a > checkpoint? > > In this case this checkpoint

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-05-11 Thread Becket Qin
Thanks for the clarification, Piotr and Sebastian. It looks like the key problem is still whether the implementation of pausable splits in the Sources should be optional or not. I think it might be helpful to agree on the definition of optional in our case. To me: Optional = "You CAN leave the me

Re: [Discuss] Creating an Apache Flink slack workspace

2022-05-11 Thread Xintong Song
> > To make some progress, maybe we decide on chat vs forum vs none and then > go into a deeper discussion on the implementation or is there anything > about Slack that would be complete blocker for the implementation? > Sure, then I'd be +1 for chat. From my side, the initiative is more about mak

Re: [Discuss] Creating an Apache Flink slack workspace

2022-05-11 Thread Jingsong Li
Hi all, Regarding using ASF slack. I share the problems I saw in the Apache Druid community. [1] > As you may have heard, it’s become increasingly difficult for new users without an @apache.org email address to join the ASF #druid Slack channel. ASF Infra disabled the option to publicly provide a

Re: [Discuss] Creating an Apache Flink slack workspace

2022-05-11 Thread Yun Tang
Hi all, I think forum might be a good choice for search and maintain. However, unlike slack workspace, it seems no existing popular product could be leveraged easily. Thus, I am +1 to create an Apache Flink slack channel. If the ASF slack cannot be joined easily for most of users, I prefer to s

Re: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

2022-05-11 Thread Александр Смирнов
Hi Jark! Sorry for the late response. I would like to make some comments and clarify my points. 1) I agree with your first statement. I think we can achieve both advantages this way: put the Cache interface in flink-table-common, but have implementations of it in flink-table-runtime. Therefore if