xiaokang commented on code in PR #35724:
URL: https://github.com/apache/doris/pull/35724#discussion_r1626020233
##########
fe/fe-core/src/main/java/org/apache/doris/alter/IndexChangeJob.java:
##########
@@ -307,10 +353,36 @@ protected void runWaitingTxnJob() throws
AlterCancelException {
protected void runRunningJob() throws AlterCancelException {
Preconditions.checkState(jobState == JobState.RUNNING, jobState);
+ // must check if db or table still exist first.
+ // or if table is dropped, the tasks will never be finished,
+ // and the job will be in RUNNING state forever.
+ Database db = Env.getCurrentInternalCatalog()
+ .getDbOrException(dbId, s -> new
AlterCancelException("Database " + s + " does not exist"));
+ OlapTable tbl;
+ try {
+ tbl = (OlapTable) db.getTableOrMetaException(tableId,
TableType.OLAP);
+ } catch (MetaNotFoundException e) {
+ throw new AlterCancelException(e.getMessage());
+ }
if (!invertedIndexBatchTask.isFinished()) {
LOG.info("inverted index tasks not finished. job: {}, partitionId:
{}", jobId, partitionId);
- // TODO: task failed limit
+ List<AgentTask> tasks =
invertedIndexBatchTask.getUnfinishedTasks(2000);
+ for (AgentTask task : tasks) {
+ if (task.getFailedTimes() > 3) {
+ LOG.warn("alter inverted index task failed: " +
task.getErrorMsg());
+ List<Long> failedBackends =
failedTabletBackends.computeIfAbsent(task.getTabletId(),
+ k -> Lists.newArrayList());
+ failedBackends.add(task.getBackendId());
+ int expectSucceedTaskNum = tbl.getPartitionInfo()
+
.getReplicaAllocation(task.getPartitionId()).getTotalReplicaNum();
+ int failedTaskCount = failedBackends.size();
+ if (expectSucceedTaskNum - failedTaskCount <
expectSucceedTaskNum / 2 + 1) {
Review Comment:
expectSucceedTaskNum is based on partition. failedTaskCount is based on
task. They are not on the same dimension.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
