Zakelly Lan created FLINK-37089:
-----------------------------------
Summary: Advanced scheduling for derived async state processing
Key: FLINK-37089
URL: https://issues.apache.org/jira/browse/FLINK-37089
Project: Flink
Issue Type: Sub-task
Components: Runtime / Async State Processing, Runtime / Task
Reporter: Zakelly Lan
Assignee: Zakelly Lan
Currently, there is a limit for in-flight records in async state processing.
However, we allow developers to initialize a new processing in-middle of
another. These derived processings are treated as normal ones for timers or
upstreaming records, which is problematic:
* The new derived processing may increase the number of in-flight requests and
cause force draining when it reaches the limit. But the draining may require
the current processing also finish. This is a deadlock.
* The derived processing will queue behind normal processing, which is not the
behavior the user wants. The derived ones should be fired right after the
current processing or ASAP.
Thus, I will change the behavior in `AsyncExecutionController` and its related
buffer:
* Avoid drain when creating derived processing.
* Provide priority for the queuing processing and make derived one have greater
priority.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
