ysymi opened a new issue, #1109:
URL: https://github.com/apache/flink-agents/issues/1109

   ### Search before asking
   
   - [x] I searched in the 
[issues](https://github.com/apache/flink-agents/issues) and found nothing 
similar.
   
   ### Description
   
   
   ## Background
   
   When a Java action suspends on `executeAsync` (e.g. waiting for an LLM 
call), `ActionExecutionOperator` resumes it by **polling**: it unconditionally 
re-submits a
   `tryProcessActionTaskForKey` mail, which checks `pending.isDone()` and 
re-queues if not done.
   
   ## Problem
   
   There is no backoff between poll rounds, so the cost depends entirely on 
mailbox traffic:
   
   - **Busy mailbox** (many concurrent keys): poll mails interleave with real 
work — waste is amortized, barely noticeable.
   - **Idle mailbox** (few keys, long async waits — typical for LLM inference 
workloads): the re-submitted mail is processed almost immediately, degrading to 
near busy-waiting for the entire
   suspension (seconds).
   
   Each poll round pays a full RocksDB list read + write-back in 
`pollNextActionTask` / `removeProcessingKey`, even when the future is not done 
and no progress is possible. For workloads
   where every record suspends on an LLM call, this is a per-record, 
per-poll-round I/O tax.
   
   ## Proposal
   
   Let the completion do the wakeup instead of the mailbox checking for it:
   
   - Register a one-shot `whenComplete` callback on the async future; when it 
completes, the worker thread submits **exactly one** wakeup mail keyed by the 
suspended record's Flink key.
   - The operator's own re-submit step then only runs when there is no 
callback-owned wakeup pending for the key — the wakeup has a single owner at 
any time. (This single-owner rule is the
   correctness-critical part: both paths firing for one suspension leads to a 
duplicate resume mail, which surfaces as the `removedCount == 1` checkState in 
`processActionTaskForKey` failing
   on cleanup.)
   - Polling remains as fallback when no callback is registered (tests, local 
environments).
   
   ## Notes
   
   - Unit tests don't cover this path: surefire's `target/classes` classpath 
doesn't resolve `META-INF/versions/21`, so the JDK21 continuation variant is 
never loaded — needs e2e/driver-level
   verification.
   - We have a working, validated implementation and are happy to contribute it 
as a PR.
   
   We'd like feedback on whether the callback-wakeup model is something the 
community wants to adopt.
   
   
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to