[Bug] [TaskExecutionRunnable] Sub workflow task always repeat run in master-server failover

[reele]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

Normally, FailoverCoordinator.getFailoverWorkflowsForMaster() finds all workflows that need failover. When the sub-workflow task's TaskExecutionRunnable.failover() method is called, takeOverTaskFromExecutor() now returns false if the task is a logic task. This results in the creation of a new sub-workflow task instance and the publication of its TaskStartLifecycleEvent, causing the sub-workflow to run again during the failover process.

dolphinscheduler/dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/engine/task/runnable/TaskExecutionRunnable.java

Lines 120 to 135 in 0719949

	@Override
	public void failover() {
	checkState(isTaskInstanceInitialized(), "The task instance is not initialized, can't failover.");
	if (takeOverTaskFromExecutor()) {
	log.info("Failover task success, the task {} has been taken-over from executor", taskInstance.getName());
	return;
	}
	this.taskInstance = applicationContext.getBean(TaskInstanceFactories.class)
	.failoverTaskInstanceFactory()
	.builder()
	.withTaskInstance(taskInstance)
	.build();
	initializeTaskExecutionContext();
	getWorkflowEventBus().publish(TaskStartLifecycleEvent.of(this));
	}

dolphinscheduler/dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/engine/task/runnable/TaskExecutionRunnable.java

Lines 160 to 164 in 0719949

	private boolean takeOverTaskFromExecutor() {
	checkState(isTaskInstanceInitialized(), "The task instance is null, can't take over from executor.");
	if (TaskTypeUtils.isLogicTask(taskInstance.getTaskType())) {
	return false;
	}

so i think it would be better to check the sub-workflow instance properly(by dao or server communicate) and take it over, instead of creating a whole new task instance.

What you expected to happen

Take over sub-workflow task if the sub-workflow instance is in good status.

How to reproduce

execute a workflow with sub-workflow, restart the master-server, query the database

Anything else

No response

Version

dev

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[SbloodyS]

The sub-workflow task is a special task type. During the fault tolerance period, all task types will generate new task instances, which is a unified operation. So this is not a bug.

[reele]

The sub-workflow task is a special task type. During the fault tolerance period, all task types will generate new task instances, which is a unified operation. So this is not a bug.

agree that too, perhaps we can make some improvement to avoid task repeating execute in future.

[SbloodyS]

agree that too, perhaps we can make some improvement to avoid task repeating execute in future.

Task instances usually run as Linux processes, and the same process id may be occupied by other programs after downtime. So it's difficult to achieve.

[reele]

agree that too, perhaps we can make some improvement to avoid task repeating execute in future.

Task instances usually run as Linux processes, and the same process id may be occupied by other programs after downtime. So it's difficult to achieve.

yes, but i mean, the sub-workflow is already be takeover in master's failover process, it just invisible and still running in background, then the father-workflow in failover process, will rerun's the sub-workflow again(by the sub-workflow task), maybe the original workflow and the new workflow are both running in time.