Siying Dong created SPARK-51682:
-----------------------------------
Summary: State Store Checkpoint V2 should handle offset log ahead
of commit log correctly
Key: SPARK-51682
URL: https://issues.apache.org/jira/browse/SPARK-51682
Project: Spark
Issue Type: Task
Components: Structured Streaming
Affects Versions: 4.0.0
Reporter: Siying Dong
Right now, there is a bug of reading checkpoint ID from commit log when the
query restarts. If the offset log is ahead of commit log, it doesn't read it,
and the tasks won't hve checkpoint ID to recover from.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
