[FEATURE] Support dropping Flint index in Failed state

[dai-chen]

Is your feature request related to a problem?

The current implementation of the FlintIndexOpDrop class in the OpenSearch Spark project does not handle the case where an index is in the FAILED state. This can occur when a Spark streaming job terminates with an exception, causing the index state to transition to FAILED. In such cases, the current DROP index statement is unable to clean up the failed index.

What solution would you like?

The proposed solution is adding support for the FAILED precondition in the FlintIndexOpDrop class, specifically in the checkPreconditions method. This would allow the drop index statement to successfully remove indices that are in the FAILED state.

What alternatives have you considered?

One alternative would be to manually delete the failed index using the OpenSearch API. However, this requires deleting index meta log entry in internal request index too.

Do you have any additional context?

• PR related: Transition Flint index state to Failed upon refresh job termination opensearch-spark#362
• Code: https://github.com/opensearch-project/sql/blob/main/spark/src/main/java/org/opensearch/sql/spark/flint/operation/FlintIndexOpDrop.java#L28