Andy Huang created HDDS-17201:
---------------------------------
Summary: Recon SCM tasks can record incorrect status for
overlapping runs
Key: HDDS-17201
URL: https://issues.apache.org/jira/browse/HDDS-17201
Project: Apache Ozone
Issue Type: Bug
Components: Ozone Recon
Reporter: Andy Huang
Follow-up to HDDS-17065, identified during review of PR #11585:
https://github.com/apache/ozone/pull/11585#discussion_r4326362272
ReconScmTask uses one shared ReconTaskStatusUpdater for scheduled and
node-event-triggered invocations. initializeAndRunTask() sets success before
calling runTask(), but status initialization and completion are outside the
lock used by PipelineSyncTask.runTask().
Example interleaving with the proposed HDDS-17065 changes:
# Scheduled run A sets status to 0 and starts processing.
# A dead-node-triggered run B sets status to 0, then waits for the runTask()
lock.
# A fails, sets the shared status to -1, and records completion.
# B acquires the lock and succeeds.
# B records completion with the shared status still -1, so a successful run is
reported as failed.
Overlapping invocations can also overwrite another run's result or mark
isCurrentTaskRunning as 0 while another invocation is active. The shared
updater and overlapping invocation paths already exist; setting success before
execution exposes the sequence above.
Scope:
* Make execution and status recording consistent across scheduled and
node-event-triggered runs, including PipelineSyncTask and ContainerHealthTask.
* Evaluate serializing the entire invocation, including status updates, or
maintaining separate per-run results with defined completion/running semantics.
Review lifecycle methods and existing locks before choosing an approach.
* Preserve partial-failure reporting by ContainerSizeCountTask; simply setting
success after runTask() would overwrite a deliberately recorded failure.
Validation:
* Add deterministic concurrency regression tests using latches to reproduce
overlapping successful and failed runs.
* Verify that one run cannot save another run's result and that the running
flag follows the chosen concurrency semantics.
* Retain coverage for partial failures, interruption handling, and retry
behavior.
This issue is based on code inspection; no runtime reproduction has been
performed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
