Re: Change In Behavior for EventTimeSessionWindow between Flink 1.11.1 and 1.14.0

2021-11-11 Thread Caizhi Weng
Hi! Did you specify source idleness time in watermark strategy? If yes source will be marked idle if no records are sent for a period of time, causing the 1.14 behavior you described. Could you also share your unit test case so we can take a deeper look? Jai Patel 于2021年11月12日周五 上午3:33写道: > Hey

Unsubscribe

2021-11-11 Thread Mack Liu
Unsubscribe

Unsubscribe

2021-11-11 Thread Uday Garikipati
Unsubscribe

Change In Behavior for EventTimeSessionWindow between Flink 1.11.1 and 1.14.0

2021-11-11 Thread Jai Patel
Hey all, I observed what appears to be a change in behavior for EventTimeSessionWindows when upgrading from 1.11.1 to 1.14.0. This was identified in a unit test. For a window with a defined time gap of 10 seconds. - Publish KEY_1 with eventtime 1 second - Publish KEY_1 with eventtime 3 sec

Re: How to specify slot task sharing group for a task manager?

2021-11-11 Thread Yangze Guo
Hi, Morten, Sorry for the belated reply. With the doc provided by Chesnay, you can start the TaskManager with GPU. However, currently with active ResourceManager, the resource profiles of TaskManager are all the same, which means all of your TaskManager will have at least one GPU. If you only want

Re: How to specify slot task sharing group for a task manager?

2021-11-11 Thread Chesnay Schepler
The external resource documentation should contain instructions to do so. On 10/11/2021 09:18, Morten Gunnar Bjørner Lindeberg wrote: Hi :) I am trying the fine-grained resource management

Re: Flink docker on k8s job submission timeout

2021-11-11 Thread Chesnay Schepler
You should be able to increase the timeout by setting client.timeout. On 10/11/2021 15:32, dhanesh arole wrote: Hello all, We are trying to run a Flink job in standalone mode using the official docker image on k8s. As per this documentation

Re: unaligned checkpoint metric

2021-11-11 Thread Chesnay Schepler
The semantics for this metric are identical between un-/aligned checkpoints, in that it describes the time between the processing of the first and last checkpoint barrier. On 11/11/2021 02:28, Kelvin Fann wrote: hi all, how do I interpret the checkpointAlignmentTime value when i am using una

Re: Flink 1.14 stuck on INITIALIZING state after job restarting

2021-11-11 Thread Xiangyu Su
Thanks Jake! I use one pod/taskmanager on one instance, on the same taskmanager/instance some slots are able to switch from INITIALIZING to RUNNING like normal, but some slots on the same instance are not. so due to that the connection of the new task manager should be ok. available slots are cor

Re: Flink 1.14 stuck on INITIALIZING state after job restarting

2021-11-11 Thread Jake
Checkout new task manager instances connection status, if connect to job manager is normal, check available slot, check create connection at function initializes. > On Nov 11, 2021, at 18:01, Xiangyu Su wrote: > > Thank you Jake! > Enable debug level logging have to ask system engineer

Re: Flink 1.14 stuck on INITIALIZING state after job restarting

2021-11-11 Thread Xiangyu Su
Thank you Jake! Enable debug level logging have to ask system engineer first ;) do you know how to resolve this issue? Best On Thu, 11 Nov 2021 at 10:57, Jake wrote: > > Set log root level is DEBUG and check Job manager logs, you will get it. > > > > On Nov 11, 2021, at 17:02, Xiangyu Su

Re: Flink 1.14 stuck on INITIALIZING state after job restarting

2021-11-11 Thread Jake
Set log root level is DEBUG and check Job manager logs, you will get it. > On Nov 11, 2021, at 17:02, Xiangyu Su wrote: > > Hello Everyone, > > We are facing an issue on Flink 1.14.0. > Every time if the job gets restarted, some tasks/slots get stuck in > INITIALIZING state, and will neve

Flink 1.14 stuck on INITIALIZING state after job restarting

2021-11-11 Thread Xiangyu Su
Hello Everyone, We are facing an issue on Flink 1.14.0. Every time if the job gets restarted, some tasks/slots get stuck in INITIALIZING state, and will never switch to RUNNING. Any idea/suggestion about a solution to this issue? btw, our flink cluster runs on EKS, and using AWS spot instance fo

Re: Re: Dependency injection for TypeSerializer?

2021-11-11 Thread Arvid Heise
Thanks for clarifying the issue. I'm assuming you can't go the state migration route because the old endpoint may not be reachable anymore? Is that correct? Why do you need to use a schema registry for internal state anyways? That's a very atypical use case. You would usually like to exert full co

Re: How to express the datatype of sparksql collect_list(named_struct(...))inflinksql?

2021-11-11 Thread JING ZHANG
Hi, vtygoss +1 on Timo's solutions. I tried those two solutions in 1.12. Both them could work well. Thanks @Timo for good suggestions. Best, JING ZHANG

Re: Flink SQL build-in function questions.

2021-11-11 Thread Martijn Visser
Hi, I don't think there's currently anyone in the community who is working on the bit operation functions. Would you be interested and able to make a contribution on that? Best regards, Martijn On Thu, 11 Nov 2021 at 03:54, JIN FENG wrote: > hi all, > I met two problems when I use FlinkSQL. >