Re: Allocation-preserving scheduling and task-local recovery

2021-09-10 Thread Xiang Zhang
askManager logs? > > > [1] > https://github.com/apache/flink/blob/release-1.10/flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/slotpool/PreviousAllocationSlotSelectionStrategy.java > > On Wed, Sep 8, 2021 at 9:44 PM Xiang Zhang wrote: > > > We also have

Re: Allocation-preserving scheduling and task-local recovery

2021-09-08 Thread Xiang Zhang
We also have this configuration set in case it makes any difference when allocation tasks: cluster.evenly-spread-out-slots. On 2021/09/08 18:09:52, Xiang Zhang wrote: > Hello, > > We have an app running on Flink 1.10.2 deployed in standalone mode. We > enabled task-local recover

Allocation-preserving scheduling and task-local recovery

2021-09-08 Thread Xiang Zhang
Hello, We have an app running on Flink 1.10.2 deployed in standalone mode. We enabled task-local recovery by setting both *state.backend.local-recovery *and *state.backend.rocksdb.localdir*. The app has over 100 task managers and 2 job managers (active and passive). This is what we have observed.

Create window before the first event

2016-07-11 Thread Xiang Zhang
Hi, I am trying to have a trigger fires every 5 mins, even when sometimes no event comes (just output default for empty window). The closest solution I got to work is this: datastream.windowAll(GlobalWindows.create()) .trigger(ContinuousProcessingTimeTrigger.of(Time.minutes(5)))

Re: ContinuousProcessingTimeTrigger on empty

2016-07-11 Thread Xiang Zhang
Hi Kostas, Yes, so I tried GlobalWindows. Is it possible to trigger every 5 mins on GlobalWindows? From the comments in the source for ContinuousProcessingTimeTrigger, it says: * A {@link Trigger} that continuously fires based on a given time interval as measured by * the clock of the machine on

ContinuousProcessingTimeTrigger on empty

2016-07-11 Thread Xiang Zhang
Hi, I want to have a trigger fires every 5 seconds in processing time even when no event comes. I tried datastream.windowAll(GlobalWindows.create()) .trigger(ContinuousProcessingTimeTrigger.of(Time.seconds(5))) .apply { MY_APPLY_FUNCTION} However, ContinuousProce