add task status named ReleasingFailed

[ycfnana]

if pod being terminating for long time because of zombie process，the task will be scheduled this node，and then this job may hang until the pod with zombie process killed force

[volcano-sh-bot]

Welcome @ycfnana! It looks like this is your first PR to volcano-sh/volcano 🎉

[hwdef]

How does this PR fix the problems you raised?

[ycfnana]

How does this PR fix the problems you raised?

job may be status pipelined when this case happens. if job request resource is future idle, it will be pipelined status, the releasing resource will be used in the way of the future idle.
I think if the pod has something wrong with termimating process, it should be detected.  the process of pod terminating, will send signal TERM and wait pod.Spec.TerminationGracePeriodSeconds(default 30s) to send signal kill. I think pod terminating process has something wrong if wait time exceed TerminationGracePeriodSeconds.
however, I think this code will result in task status from Releasing to Running, it's not good but I have no idea to chang it to another status

[hwdef]

From the code point of view, you just make the conditions for returning releasing more stringent

[ycfnana]

From the code point of view, you just make the conditions for returning releasing more stringent

my fault, its title is vague. I'll update it
I met a issue, pod always be Pending when one node has zombie pod and another node has enough reource, due to the relaesing status, I think waiting time larger than TerminationGracePeriodSeconds it should not be releaseing, may happen something error, its should be another status.

[hwdef]

time.Now().Unix()-pod.DeletionTimestamp.Unix()will keep growing, If once the time point of less than or equal to is missed, then this judgment will always be false and will never return release. Is this what we want?

[ycfnana]

In Running case, if cost time of deleting pod exceed gracePeriodSeconds . I think the pod has something error, but the request resource is allocated for scheduler, So this pod should not continue stay releasing state.
In pending case, it means this pod has been deleted but node name has already assigned , resource also cannot be freed and allocated for scheduler. although I have not met this case... just think should add it

[ycfnana]

time.Now().Unix()-pod.DeletionTimestamp.Unix()will keep growing, If once the time point of less than or equal to is missed, then this judgment will always be false and will never return release. Is this what we want?

I think gracePeriodSeconds can set bigger, and it should be another status not "Running" status when waiting time bigger than gracePeriodSeconds, but I have no idea what status to set

[hwdef]

Sorry I don't think this is a perfect fix

[ycfnana]

yeah, but this need to fix, I think it should add a status like "ReleaseFailed", in the code of resource calculation, it should be the same as Running. what do you think?

[hwdef]

@wangyang0616
Can you help review this PR

[ycfnana]

I fix it in #2943, can you help review it again?

[volcano-sh-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign k82cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @k82cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[volcano-sh-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please assign k82cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @k82cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[volcano-sh-bot]

@ycfnana: Adding label do-not-merge/contains-merge-commits because PR contains merge commits, which are not allowed in this repository.
Use git rebase to reapply your commits on top of the target branch. Detailed instructions for doing so can be found here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.