sysl openapi spec importer behaviour is nondeterministic if input open api spec defines multiple request bodies with differing content-types

[anz-rfc]

Description

sysl-go openapi spec importer behaviour is nondeterministic if input open api spec defines multiple request bodies with differing content-types

Steps to Reproduce

here's a self-contained smoking gun example.

Note: since the behaviour is nondeterministic you may need to run this for 100 or 1000 or 10,000 times on your machine to reproduce

$ cat specs/poc.sysl 
import banana.yaml as FruitShop :: BananaExchange

SomeEnterprise :: SomePlatform :: SomeService:
    @package="fruitorigination"

    FruitOClock:
        FruitShop :: BananaExchange <- POST /banana
        return ok <: string

$ cat specs/banana.yaml 
openapi: '3.0.2'
tags: 
  - name: "banana Exchange"

info:
  title: banana Exchange API
  version: '1.2'


paths:
  /banana:
    post:
      requestBody:
        required: true
        content:
          application/x-www-form-urlencoded:
            schema:
              $ref: '#/components/schemas/BananaRequest'
          application/x-www-form-urlencoded; charset=utf-8:
            schema:
              $ref: '#/components/schemas/BananaRequest'

      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/BananaResponse'

components:
  schemas:
    BananaRequest:
      required:
      - client_id
      properties:
        client_id:
          type: string
          description: The client identifier.
  
    BananaResponse:
      type: object
      properties:
        banana:
          type: string
          description: The banana issued.

sysl pb --mode json --root . specs/poc.sysl --output model0.json
sysl pb --mode json --root . specs/poc.sysl --output model1.json
sysl pb --mode json --root . specs/poc.sysl --output model2.json
sysl pb --mode json --root . specs/poc.sysl --output model3.json
sysl pb --mode json --root . specs/poc.sysl --output model4.json
sysl pb --mode json --root . specs/poc.sysl --output model5.json
sysl pb --mode json --root . specs/poc.sysl --output model6.json
sysl pb --mode json --root . specs/poc.sysl --output model7.json
sysl pb --mode json --root . specs/poc.sysl --output model8.json
sysl pb --mode json --root . specs/poc.sysl --output model9.json
sysl pb --mode json --root . specs/poc.sysl --output model10.json

$ md5sum model*.json
a9c7e95d84057fbc4bd9e6f263f1c769  model.json
a9c7e95d84057fbc4bd9e6f263f1c769  model1.json
a9c7e95d84057fbc4bd9e6f263f1c769  model10.json
a9c7e95d84057fbc4bd9e6f263f1c769  model2.json
a9c7e95d84057fbc4bd9e6f263f1c769  model3.json
a9c7e95d84057fbc4bd9e6f263f1c769  model4.json
a9c7e95d84057fbc4bd9e6f263f1c769  model5.json
a9c7e95d84057fbc4bd9e6f263f1c769  model6.json
2e5b4bb21a16ca7e99e5b00d9230aec4  model7.json
a9c7e95d84057fbc4bd9e6f263f1c769  model8.json
a9c7e95d84057fbc4bd9e6f263f1c769  model9.json

$ diff -U 10 --unified  model6.json  model7.json
--- model6.json 2021-03-03 15:41:22.000000000 +1100
+++ model7.json 2021-03-03 15:41:25.000000000 +1100
@@ -37,21 +37,21 @@
        "type": {
         "typeRef": {
          "ref": {
           "path": [
            "BananaRequest"
           ]
          }
         },
         "attrs": {
          "mediatype": {
-          "s": "application/x-www-form-urlencoded; charset=utf-8"
+          "s": "application/x-www-form-urlencoded"
          },
          "patterns": {
           "a": {
            "elt": [
             {
              "s": "body"
             }
            ]
           }
          }

Expected behavior

sysl openapi spec importer behaviour is deterministic function of input

Actual behavior

sysl openapi spec importer behaviour is not deterministic function of input

Your Environment

$ sysl info

Build:
  Version      : v0.254.0
  Git Commit   : a5a64cd82c67781f94fe5dd75b846b6446b2783c
  Date         : 2020-10-25T23:53:41Z
  Go Version   : go1.14.8 darwin/amd64
  OS           : darwin/amd64

note: this problem appears present with sysl-go 0.175.0 and sysl-go 0.180.0, which internally use sysl version 0.258.0

[anz-rfc]

Here's a guess of a possible root cause:

1. importer code iterates over map items -- the keys are request body media types, the values are schemas for that media type. note the go language reference says "The iteration order over maps is not specified and is not guaranteed to be the same from one iteration to the next." (https://golang.org/ref/spec)
2. a nameless Param value is created. the param Field is set by fieldForMediaType
3. the nameless Param is added into the Parameters structure and stored in a map keyed by the name (which appears to be the zero string for all Param values)

sysl/pkg/importer/openapi.go

Lines 439 to 445 in 065dccd

	if req := op.RequestBody; req != nil {
	for mediaType, obj := range req.Value.Content {
	param := Param{In: "body"}
	param.Field = o.fieldForMediaType(mediaType, obj.Schema, "Request")
	ep.Params.Add(param)
	}
	}

sysl/pkg/importer/parameters.go

Lines 13 to 21 in 20fa66e

	func (p *Parameters) Add(param Param) {
	if p.items == nil {
	p.items = map[string]Param{}
	}
	if _, found := p.items[param.Name]; !found {
	p.insertOrder = append(p.insertOrder, param.Name)
	}
	p.items[param.Name] = param
	}