[Feature/agent_framework] Add Delete Connector Step

[dbwiddis]

Description

Adds a Delete Connector workflow step.

Issues Resolved

In support of #89

curl -i -XPOST "localhost:9200/_plugins/_flow_framework/workflow" -H "Content-Type:application/json" --data '
{
  "name": "create-connector-register-deploy-model",
  "description": "test case",
  "use_case": "TEST_CASE",
  "version": {
    "template": "1.0.0",
    "compatibility": [
      "2.11.0",
      "3.0.0"
    ]
  },
  "workflows": {
    "provision": {
      "nodes": [
        {
          "id": "workflow_step_1",
          "type": "create_connector",
          "user_inputs": {
            "name": "OpenAI Chat Connector",
            "description": "The connector to public OpenAI model service for GPT 3.5",
            "version": "1.0.0",
            "protocol": "http",
            "parameters": {
              "endpoint": "api.openai.com",
              "model": "gpt-3.5-turbo"
            },
            "credential": {
              "openAI_key": "12345"
            },
            "actions": [
              {
                "action_type": "predict",
                "method": "POST",
                "url": "https://${parameters.endpoint}/v1/chat/completions"
              }
            ]
          }
        },
        {
          "id": "workflow_step_2",
          "type": "delete_connector",
          "previous_node_inputs": {
            "workflow_step_1": "connector_id"
          }
        }
      ]
    }
  }
}'

curl -i -XPOST "localhost:9200/_plugins/_flow_framework/workflow/-f6-GIwBk0tyGR8GjdmT/_provision"

Output:

[2023-11-28T17:49:23,179][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] Queueing process [workflow_step_1]. Can start immediately!
[2023-11-28T17:49:23,181][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] Queueing process [workflow_step_2]. Must wait for [workflow_step_1] to complete first.
[2023-11-28T17:49:23,181][INFO ][o.o.f.w.ProcessNode      ] [88665a43e3fa] Starting workflow_step_1.
[2023-11-28T17:49:23,182][INFO ][o.o.m.a.c.TransportCreateConnectorAction] [88665a43e3fa] connector created, indexing into the connector system index
[2023-11-28T17:49:23,319][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] updated workflow -f6-GIwBk0tyGR8GjdmT state to PROVISIONING
[2023-11-28T17:49:23,340][INFO ][o.o.m.a.c.TransportCreateConnectorAction] [88665a43e3fa] Connector saved into index, result:CREATED, connector id: -_7EGIwBk0tyGR8GPdlu
[2023-11-28T17:49:23,359][INFO ][o.o.f.w.CreateConnectorStep] [88665a43e3fa] Created connector successfully
[2023-11-28T17:49:23,359][INFO ][o.o.f.w.ProcessNode      ] [88665a43e3fa] Finished workflow_step_1.
[2023-11-28T17:49:23,359][INFO ][o.o.f.w.ProcessNode      ] [88665a43e3fa] Starting workflow_step_2.
[2023-11-28T17:49:23,422][INFO ][o.o.m.a.c.DeleteConnectorTransportAction] [88665a43e3fa] Completed Delete Connector Request, connector id:-_7EGIwBk0tyGR8GPdlu deleted
[2023-11-28T17:49:23,442][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] Provisioning completed successfully for workflow -f6-GIwBk0tyGR8GjdmT
[2023-11-28T17:49:23,442][INFO ][o.o.f.w.ProcessNode      ] [88665a43e3fa] Finished workflow_step_2.
[2023-11-28T17:49:23,542][INFO ][o.o.f.w.CreateConnectorStep] [88665a43e3fa] updated resources created of -f6-GIwBk0tyGR8GjdmT
[2023-11-28T17:49:23,605][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] updated workflow -f6-GIwBk0tyGR8GjdmT state to COMPLETED

Output with invalid connector id:

[2023-11-28T17:39:38,768][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] Queueing process [workflow_step_1]. Can start immediately!
[2023-11-28T17:39:38,769][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] Queueing process [workflow_step_2]. Must wait for [workflow_step_1] to complete first.
[2023-11-28T17:39:38,769][INFO ][o.o.f.w.ProcessNode      ] [88665a43e3fa] Starting workflow_step_1.
[2023-11-28T17:39:38,856][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] updated workflow 67m7GIwB4pgpFwKmABmZ state to PROVISIONING
[2023-11-28T17:39:38,880][INFO ][o.o.m.a.c.TransportCreateConnectorAction] [88665a43e3fa] connector created, indexing into the connector system index
[2023-11-28T17:39:38,891][INFO ][o.o.p.PluginsService     ] [88665a43e3fa] PluginService:onIndexModule index:[.plugins-ml-connector/URKg7xTpT4uKPi1bJNwTLw]
[2023-11-28T17:39:38,899][INFO ][o.o.c.m.MetadataCreateIndexService] [88665a43e3fa] [.plugins-ml-connector] creating index, cause [api], templates [], shards [1]/[1]
[2023-11-28T17:39:38,900][INFO ][o.o.c.r.a.AllocationService] [88665a43e3fa] updating number_of_replicas to [0] for indices [.plugins-ml-connector]
[2023-11-28T17:39:38,974][INFO ][o.o.p.PluginsService     ] [88665a43e3fa] PluginService:onIndexModule index:[.plugins-ml-connector/URKg7xTpT4uKPi1bJNwTLw]
[2023-11-28T17:39:39,387][INFO ][o.o.c.r.a.AllocationService] [88665a43e3fa] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.plugins-ml-connector][0]]]).
[2023-11-28T17:39:39,482][INFO ][o.o.m.e.i.MLIndicesHandler] [88665a43e3fa] create index:.plugins-ml-connector
[2023-11-28T17:39:39,584][INFO ][o.o.m.a.c.TransportCreateConnectorAction] [88665a43e3fa] Connector saved into index, result:CREATED, connector id: 7Lm7GIwB4pgpFwKmVRlt
[2023-11-28T17:39:39,603][INFO ][o.o.f.w.CreateConnectorStep] [88665a43e3fa] Created connector successfully
[2023-11-28T17:39:39,603][INFO ][o.o.f.w.ProcessNode      ] [88665a43e3fa] Finished workflow_step_1.
[2023-11-28T17:39:39,603][INFO ][o.o.f.w.ProcessNode      ] [88665a43e3fa] Starting workflow_step_2.
[2023-11-28T17:39:39,648][INFO ][o.o.m.a.c.DeleteConnectorTransportAction] [88665a43e3fa] Connector id:test not found
[2023-11-28T17:39:39,668][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] Provisioning completed successfully for workflow 67m7GIwB4pgpFwKmABmZ

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[dbwiddis]

Converting to draft, I'd like #215 merged first, then I'll rebase and implement previous step fetching from maps.

[dbwiddis]

Re-tested after updating to "previous step" fetching of connector ID.

[2023-11-29T19:29:22,484][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] Queueing process [workflow_step_1]. Can start immediately!
[2023-11-29T19:29:22,486][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] Queueing process [workflow_step_2]. Must wait for [workflow_step_1] to complete first.
[2023-11-29T19:29:22,486][INFO ][o.o.f.w.ProcessNode      ] [88665a43e3fa] Starting workflow_step_1.
[2023-11-29T19:29:22,562][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] updated workflow i4NFHowBKn_YW9vi2HbQ state to PROVISIONING
[2023-11-29T19:29:22,645][INFO ][o.o.m.a.c.TransportCreateConnectorAction] [88665a43e3fa] connector created, indexing into the connector system index
[2023-11-29T19:29:22,713][INFO ][o.o.m.a.c.TransportCreateConnectorAction] [88665a43e3fa] Connector saved into index, result:CREATED, connector id: jINGHowBKn_YW9viJHbX
[2023-11-29T19:29:22,732][INFO ][o.o.f.w.CreateConnectorStep] [88665a43e3fa] Created connector successfully
[2023-11-29T19:29:22,732][INFO ][o.o.f.w.ProcessNode      ] [88665a43e3fa] Finished workflow_step_1.
[2023-11-29T19:29:22,732][INFO ][o.o.f.w.ProcessNode      ] [88665a43e3fa] Starting workflow_step_2.
[2023-11-29T19:29:22,796][INFO ][o.o.m.a.c.DeleteConnectorTransportAction] [88665a43e3fa] Completed Delete Connector Request, connector id:jINGHowBKn_YW9viJHbX deleted
[2023-11-29T19:29:22,815][INFO ][o.o.f.w.ProcessNode      ] [88665a43e3fa] Finished workflow_step_2.
[2023-11-29T19:29:22,815][INFO ][o.o.f.t.ProvisionWorkflowTransportAction] [88665a43e3fa] Provisioning completed successfully for workflow i4NFHowBKn_YW9vi2HbQ

[owaiskazi19]

Looks good to me overall with one comment. Thanks for addressing the CVE.

[joshpalis]

The implementation looks good! Just a few comments on the tests

[amitgalitz]

LGTM