Re: How/where to check the operator id

2023-10-06 Thread megh vidani
Thank you Chen. There's no specific need so far for the info. I just wanted to check that the implementation (I mean assignment of operator IDs) has been done correctly and if I try to restore the job from a checkpoint with some changes in the job graph for other stateless operators, I would still

回复: How/where to check the operator id

2023-10-06 Thread Chen Zhanghao
Hi Megh, Unfortunately, Flink currently does not expose operator IDs via REST API, nor log them unless some operators' states are missing during state recovery as operator IDs are mostly used internally. Could you further collaborate a bit on why you need the operator ID info? Maybe you can cre

UDF support's parity in Flink Table/Sql api

2023-10-06 Thread mojhaha kiklasds
Hi team, I am looking to understand parity in Table API and SQL API. Would you say that we have parity for UDF? I see the TableAggregateFunction can not be represented in Flink SQL but only in Table API. Have you known it to work like other UDFs?

Re: Rolling back a bad deployment of FlinkDeployment on kubernetes

2023-10-06 Thread Tony Chen
So, I was able to get the rollback to work after I changed my upgradeMode to *last-state*. Previously, my upgradeMode was *savepoint*, and when I deployed a bad commit, the jobmanager-leader configmap would get deleted. Once I changed the upgradeMode to *last-state*, the configmap was retained when

Checkpoint/Savepoint restore of S3 file reads using continuous read mode

2023-10-06 Thread Mark Petronic
I am trying to understand the Flink design pattern for consuming files from S3 continuously as they appear. I have written the below minimal program to do that and it works as expected wrt detecting newly-uploaded S3 files within the configured 5 second monitoring poll period. Then it just prints t

Delayed Window

2023-10-06 Thread Kenan Kılıçtepe
Hi, Is it possible to delay a window trigger without changing window-end and window-start times? Thanks