Ability to config rule of modify response

[mildronize]

Problem Statement

Many may wonder why we don't use tools like Nginx, Traefik, or Kong, which already serve as gateways. From my observations, these options do not necessarily provide a configuration simplicity that end-users can easily manage. Additionally, it appears that, for the most part, these tools are designed for static values that can be transformed in the response, without the ability to incorporate additional conditions, such as based on the path or specific requests.

If anyone is familiar with a project that addresses similar concerns, I would appreciate some recommendations. Thank you.

References for API Gateways/Reverse Proxy Server:

• nginx: https://nginx.org/en/docs/http/ngx_http_proxy_module.html...
• traefik: https://plugins.traefik.io/.../628c9f.../replace-status-code
  https://github.com/traefik/plugin-rewritebody
  https://github.com/traefik/traefik/issues/2039
• Kong: https://docs.konghq.com/hub/kong-inc/response-transformer/
• YARP https://github.com/microsoft/reverse-proxy/blob/release/1.1/docs/docfx/articles/transforms.md
• lua-nginx-module: https://gist.github.com/ejlp12/b3949bb40e748ae8367e17c193fa9602
• Tyk: https://tyk.io/docs/advanced-configuration/transform-traffic/response-body/#response-body-transformation

References for Stubbing & Double Test:

Some tools can also be proxy or gateway:

• wiremock: https://wiremock.org/
• mountebank: https://www.mbtest.org/

How the spy work?

Modify HTTP Response Reverse Proxy

Store config rule in a data store

• Implement abstract class for generalize API to any data store
• Data Store:
  • Support in Azure Storage Table (For the first phase, the end user can use Azure Storage Explorer to config it.)
  • JSON local store files (extending ability of write-file-atomic,

Admin Route

• Serve the affected rules using path /srp (If pass expression evaluation) using Provide express middleware as external library data-viewer#6, to show the list for debuging
• Using env variables: SRP_ADMIN_ROOT_PATH : the root path of admin of the reverse proxy, default is /srp

Data Schema for Spy config rules

• ruleName
• upstreamUrl
• path
• condition
  • true / false (for early release) = default is true
• plugin
  • response_transformer
  • request_transformer (For Future Release)
• data
  • Expression syntax for **response_transformer**
    1. <action> = <value>, the value is JSON-compatibility data
    2. <action> = <key> : <value>, the value is JSON-compatibility data
    3. Multiple expression⇒
       1. Ex: <action> = <value> , <action> = <value>
       2. Ex: <action> = <value> , <action> = <key> : <value>
  • Allow Actions
    • add.headers:
      • add.headers={"Content-Type":"application/json"}
    • replace.status_code
      • replace.status_code=400
    • add.json
      • add.json={"data": "Hello World"}
    • replace.json — replace the whole json
      • replace.json={"data": "Hello"}
    • replace.key_json: — replace by the json key

      • replace.key_json=customers.info.phone:"01-1234-5678"

      • replace.key_json=customers.info:{"phone":"01-1234-5678"}

      • using syntax a[0].b.c=new payload,a[0].b="hey,xxx" , this will override 2 parts of the body shown below (Use " for wrapping comma (,) and use \" in the double quotes " ):

        • a[0].b.c ⇒ new payload
        • a[0].b.d ⇒ hey,xxx

      • Example Response Input:

        {
          "a": [
            {
              "b": {
                "c": 3
              }
            }
          ]
        }
        

      • Example Response Output:

        {
          "a": [
            {
              "b": {
                "c": "new payload",
                "d": "hey,xxx"
              }
            }
          ]
        }
        

      • use syntax from lodash.set() as https://lodash.com/docs/4.17.15#set

    • remove.key_json — remove the json key
      • remove.key_json=customers.info.phone

Environment Variables

• SRP_UPSTREAM_URL the target URL that want the spy server taken over it.
• SRP_DATA_AZURE_TABLE_CONNECTION_STRING, the connection string for Azure Table
• SRP_PORT the running port
• SRP_DISABLED if the value is true , ignore all spy config rules
• SRP_ADMIN_ROOT_PATH : the root path of admin of the reverse proxy, default is /srp

Running Mode

Single Upstream URL

Multiple Upstream URLs in single datastore

• It means that we can use it for mutli-tenant systems

Vocab

• Upstream URLs: config the spy server to pass the request to the target URL

[mildronize]

Redesign spec

Inspired from https://docs.konghq.com/hub/kong-inc/response-transformer/how-to/basic-example/#configuration

(Edited already updated the issue description)

[mildronize]

key_json better than nested_json

(Edited already updated the issue description)

[mildronize]

Serve the affected rules using path /srp (If pass expression evaluation) using thaitype/data-viewer#6, to show the list for debugging

(Edited already updated the issue description)