yujun777 commented on code in PR #66488:
URL: https://github.com/apache/doris/pull/66488#discussion_r3780849376
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/AnalysisManager.java:
##########
@@ -548,6 +550,13 @@ public void updateTaskStatus(AnalysisInfo info,
AnalysisState taskState, String
if (MetricRepo.isInit) {
MetricRepo.COUNTER_STATISTICS_FAILED_ANALYZE_JOB.increase(1L);
}
+ // The job reached a terminal state: all tasks share the
job's
+ // partitionUpdateRows map, so clearing it here releases
the memory
+ // retained by every task record in the history at once.
The success
+ // path clears it inside updateTableStats.
+ if (job.partitionUpdateRows != null) {
+ job.partitionUpdateRows.clear();
Review Comment:
Fixed. The task-side writers now check the killed flag before writing to the
shared partitionUpdateRows map (BaseAnalysisTask.doPartitionTable,
OlapAnalysisTask.deleteNotExistPartitionStats), so a cancelled task stops
writing. AnalysisJob.taskFailed now also clears the shared map in a finally
block after all task cancels have been issued, releasing it even when the
terminal-state clear in updateTaskStatus missed in-flight writes. Added
AnalysisJobTest.testTaskFailedClearsSharedPartitionUpdateRows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
