rosemarYuan commented on issue #841: URL: https://github.com/apache/flink-agents/issues/841#issuecomment-4901457669
Thanks for sharing the design direction @addu390 @zxs1633079383 . I took a look at the current runtime/event-log path, and one identity detail around `runId` seems worth settling before the schema lands. Using `runId = inputEvent.id` may not be stable enough. In Java, `InputEvent` is freshly created in `wrapToInputEvent` with a random UUID, so source replay after failover can give the same logical input a different run id. In Python, event ids are content-derived, so two distinct inputs with identical payloads can share the same run id and merge into one run. A more replay-stable option could be deriving `runId` from the stable job identifier, key, and per-key sequence number, for example `jobIdentifier:key:sequenceNumber` with a well-defined key encoding. This does not add a new identity concept: `key + sequenceNumber` is already part of the identity `ActionStateUtil.generateKey` relies on to match action state across failover. I am currently working on Memory observability, which seems closely related to event-log trace work. Do you have any near-term plans to work on this issue? If not, I would be happy to pick it up after finishing Memory observability. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
