Provide an adaptor for localstack deployment

[tyge68]

Expected Behaviour

Deployment of the generated zip should be compatible with localstack development (which allow to run AWS locally)

Actual Behaviour

Current "lambda" handler is not working while running in localstack due to few differences in the event object structure.
It returns a message like "Request with GET/HEAD method cannot have body" for a simple GET request.

Reproduce Scenario (including but not limited to)

a bit complex, you need to use localstack in docker, and setup a few services (api gateway, s3, secresmanager etc.. as it would run in a real AWS)

version: '3.1'

services:

  localstack:
    image: localstack/localstack:latest
    environment:
      - AWS_DEFAULT_REGION=us-east-1
      - SERVICES=s3,apigateway,sqs,lambda,secretsmanager
    ports:
      - '4566:4566'
      - '4567:4567'
    volumes:
      - "${TEMPDIR:-/tmp/localstack}:/tmp/localstack"
      - ./test/aws:/etc/localstack/init/ready.d
      - ./dist:/dist
      - "/var/run/docker.sock:/var/run/docker.sock"

define a bootstrap script in test/aws/apigateway.sh (you will also need some for s3 and secretsmanager) but that one is the most important

#!/usr/bin/env bash
API_NAME=myapp
REGION=us-east-1
STAGE=dev

function fail() {
    echo $2
    exit $1
}

awslocal lambda create-function \
    --region ${REGION} \
    --function-name ${API_NAME} \
    --runtime nodejs16.x \
    --handler index.lambda \
    --memory-size 1024 \
    --zip-file fileb:///dist/helix-services/[email protected] \
    --role arn:aws:iam::123456123456:role/irrelevant

[ $? == 0 ] || fail 1 "Failed: AWS / lambda / create-function"

LAMBDA_ARN=$(awslocal lambda list-functions --query "Functions[?FunctionName==\`${API_NAME}\`].FunctionArn" --output text --region ${REGION})

echo LAMBDA_ARN=${LAMBDA_ARN}

awslocal apigateway create-rest-api \
    --region ${REGION} \
    --name ${API_NAME}

[ $? == 0 ] || fail 2 "Failed: AWS / apigateway / create-rest-api"

API_ID=$(awslocal apigateway get-rest-apis --query "items[?name==\`${API_NAME}\`].id" --output text --region ${REGION})
PARENT_RESOURCE_ID=$(awslocal apigateway get-resources --rest-api-id ${API_ID} --query 'items[?path==`/`].id' --output text --region ${REGION})

awslocal apigateway create-resource \
    --region ${REGION} \
    --rest-api-id ${API_ID} \
    --parent-id ${PARENT_RESOURCE_ID} \
    --path-part "endpoint"

[ $? == 0 ] || fail 2 "Failed: AWS / apigateway / create-resource /endpoint"

RESOURCE_ID=$(awslocal apigateway get-resources --rest-api-id ${API_ID} --query 'items[?path==`/endpoint`].id' --output text --region ${REGION})

awslocal apigateway put-method \
    --region ${REGION} \
    --rest-api-id ${API_ID} \
    --resource-id ${RESOURCE_ID} \
    --http-method ANY \
    --authorization-type "NONE"

[ $? == 0 ] || fail 4 "Failed: AWS / apigateway / put-method"

awslocal apigateway put-integration \
    --region ${REGION} \
    --rest-api-id ${API_ID} \
    --resource-id ${RESOURCE_ID} \
    --http-method ANY \
    --type AWS_PROXY \
    --integration-http-method POST \
    --uri arn:aws:apigateway:${REGION}:lambda:path/2015-03-31/functions/${LAMBDA_ARN}/invocations \
    --passthrough-behavior WHEN_NO_MATCH \

[ $? == 0 ] || fail 5 "Failed: AWS / apigateway / put-integration"

awslocal apigateway create-deployment \
    --region ${REGION} \
    --rest-api-id ${API_ID} \
    --stage-name ${STAGE} \

[ $? == 0 ] || fail 6 "Failed: AWS / apigateway / create-deployment"

ENDPOINT=http://localhost:4566/restapis/${API_ID}/${STAGE}/_user_request_/endpoint
echo "API available at: ${ENDPOINT}"

Steps to Reproduce

Once you setup to make the generated zip (dist/helix-services/[email protected]) as the lambda function to be used by the apigateway endpoint of your choice. Call the service url .i.e http://localhost:4566/restapis/hrbmpo1ifh/dev/_user_request_/endpoint, it will returns a 500 with "Request with GET/HEAD method cannot have body"

Platform and Version

AWS via Localstack

Sample Code that illustrates the problem

Logs taken while reproducing problem

The main reason seems that the main function expect some specific structure for the requestContent, which is not 100% the same in the localstack

main differences is that there is no event.requestContent.http.method, instead it is under event.requestContent.httpMethod
and second main differences is that event.body = "" instead of being null in case of "GET" request.

one fix could be to create a small adapter that fix the event object to be like the default "lambda" adapter expect (mainly copy what is needed under event.requestContext.http and fix the body value to be null in get of "GET|HEAD" request.

[dominique-pfister]

for completion, this is what a Lambda GET invocation via HTTP API Gateway looks like (irrelevant fields omitted):

{
  "event": {
    "version": "2.0",
    "rawPath": "/simple-package/dump",
    "rawQueryString": "",
    "headers": {
      "accept": "*/*",
      "content-length": "0",
      "host": "xxx.execute-api.us-east-1.amazonaws.com",
    },
    "requestContext": {
      "domainName": "xxx.execute-api.us-east-1.amazonaws.com",
      "domainPrefix": "xxx",
      "http": {
        "method": "GET",
        "path": "/simple-package/dump",
        "protocol": "HTTP/1.1",
        "sourceIp": "xxx.xxx.xxx.xxx",
        "userAgent": "curl/8.1.2"
      },
      "stage": "$default",
      "time": "28/Aug/2023:14:40:07 +0000",
      "timeEpoch": 1693233607826
    },
    "isBase64Encoded": false
  },
}

[tyge68]

this is also the event how it looks like using localstack

{
    "path": "/endpoint",
    "headers": {
         ....
    },
    "multiValueHeaders": {
        ....
    },
    "body": "",
    "isBase64Encoded": false,
    "httpMethod": "GET",
    "queryStringParameters": null,
    "multiValueQueryStringParameters": null,
    "pathParameters": {},
    "resource": "/endpoint",
    "requestContext": {
        "accountId": "000000000000",
        "apiId": "xxx",
        "resourcePath": "/endpoint",
        "domainPrefix": "localhost",
        "domainName": "localhost",
        "resourceId": "xxx",
        "requestId": "xxx",
        "identity": {
            "accountId": "000000000000",
            "sourceIp": "172.22.0.1",
            "userAgent": "xxx"
        },
        "httpMethod": "GET",
        "protocol": "HTTP/1.1",
        "requestTime": "28/Aug/2023:11:57:29 +0000",
        "requestTimeEpoch": 1693223849432,
        "authorizer": {},
        "path": "/dev/endpoint",
        "stage": "dev"
    },
    "stageVariables": {}
}

[tyge68]

Note locally I could fix it by unzipping the generated zip, and add / fix some lines in the generated index.js

For example adding my custom handler for localstack specifics (or what seems specific , although all docs I found about AWS suppose that the event match the same as the one generated by localstack...)

async function localstack(event, context) {
    if (event.requestContext) {
        const reqCopy = JSON.parse(JSON.stringify(event.requestContext));
        event.requestContext.http = reqCopy;
        event.requestContext.http.method = event.requestContext.httpMethod;
        const method = event.requestContext.http.method;
        if (method === "GET" || method === "HEAD") {
            delete event.body
        }
    }
    return await lambda(event, context);
}

module.exports = {
  main: openwhisk,
  lambda,
  localstack,
  google,
};

For some other issues (more related to the secretsmanager related code that read a secret, where it fails due to "UnrecognizedClientException" possible exception in the context of localstack

console.error(`unable to load function params from '${secretId}'`, e);
const error = new Error('unable to load function params');
if (e.code === 'ThrottlingException') {
  error.statusCode = 429;
}
if (e.code === 'ResourceNotFoundException' || e.code === 'UnrecognizedClientException') {
  return {};
}
throw error;