[BUG] Inconsistency with generated model interfaces

[ohltyler]

I've run into inconsistencies in the documentation and functionality of the model interfaces.

The documentation states:

To simplify your workflow, you can register an externally hosted model using a connector in one of the connector blueprint formats. If you do so, a predefined model interface for this connector is generated automatically during model registration. The predefined model interface is generated based on the connector blueprint and the model's metadata, so you must strictly follow the blueprint when creating the connector in order to avoid errors.

The first thing I'm wondering, is what does it mean to "strictly follow the blueprint" and what can be changed/customized in the blueprint, in order to still trigger the automatic interface generation?

Secondly, is there a library of suggested/preset model interfaces?

An example of the ambiguity and issues I'm facing: the second connector example here has a request body containing parameters.prompt but not parameters.inputs anywhere. Yet the default interface generated from this requires inputs but not prompt.

Interface generated:

"interface": {
    "output": """{
    "type": "object",
    "properties": {
        "inference_results": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "output": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "name": {
                                    "type": "string"
                                },
                                "dataAsMap": {
                                    "type": "object",
                                    "properties": {
                                        "response": {
                                            "type": "string"
                                        }
                                    },
                                    "required": [
                                        "response"
                                    ]
                                }
                            },
                            "required": [
                                "name",
                                "dataAsMap"
                            ]
                        }
                    },
                    "status_code": {
                        "type": "integer"
                    }
                },
                "required": [
                    "output",
                    "status_code"
                ]
            }
        }
    },
    "required": [
        "inference_results"
    ]
}""",
    "input": """{
    "type": "object",
    "properties": {
        "parameters": {
            "type": "object",
            "properties": {
                "inputs": {
                    "type": "string"
                }
            },
            "required": [
                "inputs"
            ]
        }
    },
    "required": [
        "parameters"
    ]
}"""
  }

Error when executing predict (not surprising, since prompt is needed in the request body and not listed under the default parameters in the connector blueprint)

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Some parameter placeholder not filled in payload: prompt"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Some parameter placeholder not filled in payload: prompt"
  },
  "status": 400
}

The automatically generated interfaces should be consistent & dependent on the connector blueprints' required parameters, such as to not cause added confusion when inputs satisfy the model interface, yet fail when hitting the actual prediction.

[b4sjoo]

Nice catch, we will fix the blueprint accordingly