Up to this point, we've seen how we can leverage API Gateway to create REST APIs, the example used is just enough to get the bare minimum on how it works by using a MOCK integration. API Gateway can be integrated with Lambda, in fact, this is one of the most common service combinations to create serverless applications. There are two ways to achieve this integration: Lambda integration (AWS) and Proxy integration (AWS_PROXY).
In this part we'll see how to integrate a Lambda function with API Gateway using Lambda integration (AWS). This approach defines a clear separation between the API responsibilities and the Lambda function, the function doesn't know anything about how is going to be invoked and therefore is totally independent from any HTTP handling. The API, on the other hand, will carry out all the mappings from the HTTP request to the Lambda function input and all the way back from the Lambda result to the end user HTTP response.
In the following sections we will create a Lambda integration using three different implementations in order to better understand the details. First, using purely AWS CLI, then from an Open API file and finally with a CloudFormation template.
In this section we'll deploy a Lambda function which will be common to the first and second integration implementations. The code will be the same as in part 05 and can be copied from there to follow along. In a directory with files Function.cs and project.lambda.csproj we'll create and deploy the Lambda function by running the following commands (more details in part 05).
- abelperez-temp is the bucket of your choice to upload the code to S3.
- project-lambda/ is just a folder inside the bucket.
- the value for --role parameter is copied from the output of the aws iam create-role command.
- do take note of the function arn as we'll need it later for the integration.
$ dotnet lambda package -c Release -o project-lambda.zip
$ aws s3 cp project-lambda.zip s3://abelperez-temp/project-lambda/
$ cat > role-policy.json <<EOM
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
EOM
$ aws iam create-role --role-name HelloLambdaRole --assume-role-policy-document file://role-policy.json
$ aws iam attach-role-policy --policy-arn arn:aws:iam::aws:policy/AWSLambdaExecute --role-name HelloLambdaRole
$ aws lambda create-function --function-name HelloLambda \
--code S3Bucket=abelperez-temp,S3Key=project-lambda/project-lambda.zip \
--role arn:aws:iam::123123123123:role/HelloLambdaRole \
--handler project.lambda::project.lambda.Function::HelloHandler \
--runtime dotnetcore3.1 --timeout 30
Use dotnet lambda invoke-function
command for simplicity, here are two examples to cover both scenarios.
$ dotnet lambda invoke-function -fn HelloLambda -p "{\"Name\":\"Abel\",\"Age\":33}" --region eu-west-1
$ dotnet lambda invoke-function -fn HelloLambda -p "{\"Name\":\"Abel\",\"Age\":88}" --region eu-west-1
We need an API and at least one resource with a method, let's create them.
Following the same procedure as in part 06, we create a REST API with one resource listening on /hello and a GET Method. Each command's output returns ids used in the next command.
$ aws apigateway create-rest-api --name project-api-cli
$ aws apigateway get-resources --rest-api-id ryk941lawh
$ aws apigateway create-resource --rest-api-id ryk941lawh --parent-id cvzihh5l2d --path-part hello
$ aws apigateway put-method --rest-api-id ryk941lawh --resource-id bay12i --http-method GET --authorization-type "NONE"
$ aws apigateway put-method-response --rest-api-id ryk941lawh --resource-id bay12i --http-method GET --status-code 200
First, we need to set up a request mapping template, the same way it's done with MOCK integration. Create a JSON file with the template mapping, in this case, we define that query string / header parameters name and age will be passed respectively to Name and Age input properties in the Lambda function as part of the input. This JSON is escaped because it's the value expected as a string for every key as content type. $input.params() retrieves the parameters from query string and headers (in that order). These templates are created using Apache VTL.
$ cat > request-template.json <<EOM
{
"application/json": "{\"Name\":\"\$input.params('name')\",\"Age\":\$input.params('age')}"
}
EOM
Assuming the function arn from the steps above is 'arn:aws:lambda:eu-west-1:123123123123:function:HelloLambda', let's create the integration by using the aws apigateway put-integration
command.
$ aws apigateway put-integration --rest-api-id ryk941lawh \
--resource-id bay12i --http-method GET \
--type AWS \
--integration-http-method POST \
--uri arn:aws:apigateway:eu-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:eu-west-1:123123123123:function:HelloLambda/invocations \
--request-templates file://request-template.json
- --rest-api-id and --resource-id from the commands above.
- --type AWS indicates the integration with an AWS service, Lambda in this case.
- --http-method has to match one of the methods for the chosen resource, GET in this case is the only one.
- --integration-http-method is always POST for Lambda invocations.
- --uri is in the form of arn:aws:apigateway:{aws-region}:lambda:path/2015-03-31/functions/{function-arn}/invocations .
- --request-templates we use the file previously created.
Second, we need to set up a response mapping template, just like in the MOCK example. Similarly we create a JSON file which follows the same format.
$ cat > response-template.json <<EOM
{
"application/json": "#set (\$root=\$input.path('$'))\n{\n\"Message\": \"Dear \$root.Name, you are#if(!\$root.Old) not#end old.\"\n}"
}
EOM
This example is also escaped, it reads from the Lambda output and creates a new JSON with a Message property and a simple logic to adapt the content, this is the unescaped version:
#set ($root=$input.path('$'))
{
"Message": "Dear $root.Name, you are#if(!$root.Old) not#end old."
}
Notice the directives #if and #end. Now, add the integration response.
$ aws apigateway put-integration-response --rest-api-id ryk941lawh \
--resource-id bay12i --http-method GET \
--status-code 200 --selection-pattern "" \
--response-templates file://response-template.json
- --rest-api-id and --resource-id from the commands above.
- --http-method has to match one of the methods for the chosen resource, GET in this case is the only one.
- --status-code has to match one of the method responses created above, 200 is this case is the only one.
- --selection-pattern indicates the regular expression to identify the status code from the Lambda output, empty means it's the default one.
After this, we have closed the circuit from client API request to server API response. There is however, a missing piece in this puzzle.
So far we've given permission to Lambda function to access CloudWatch logs so it can log to a log stream. However, we haven't given permission to API Gateway to invoke our Lambda function, this is done by creating a Lambda permission.
$ aws lambda add-permission --function-name HelloLambda \
--statement-id project-api-cli-invoke-lambda \
--action lambda:InvokeFunction \
--principal apigateway.amazonaws.com \
--source-arn arn:aws:execute-api:eu-west-1:123123123123:ryk941lawh/*/GET/hello
- --statement-id is an id for the permission, api name + invoke lambda would make it more meaningful.
- --action should be lambda:InvokeFunction, that's all we need from the Lambda function, being able to invoke it.
- --principal should be apigateway.amazonaws.com , indicates that API Gateway service will be granted the permission.
- --source-arn should be in the form of arn:aws:execute-api:{aws-region}:{accountNumber}:{restApiId}/{stage}/{method}/{path}
Just like previously, we need a deployment to make all these resources available through an endpoint.
$ aws apigateway create-deployment \
--rest-api-id ryk941lawh \
--stage-name dev \
--stage-description "Development Stage" \
--description "Testing dev stage"
A Rest API endpoint url is the format https://{rest-api-id}.execute-api.{AWS-region}.amazonaws.com/{stage-name}{path-and-query}
In this example, Rest API Id is ryk941lawh and we've created a stage named dev in the region of Ireland (eu-west-1), to test the resource with path /hello we use the following curl command.
$ curl -X GET "https://ryk941lawh.execute-api.eu-west-1.amazonaws.com/dev/hello?name=Abel&age=33"
{
"Message": "Dear Abel, you are not old."
}
In the example above, both parameters have been supplied in order to get a good response from the Lambda function. Now we can test the other alternative using a value for age higher than 50.
$ curl -X GET "https://ryk941lawh.execute-api.eu-west-1.amazonaws.com/dev/hello?name=Abel&age=88"
{
"Message": "Dear Abel, you are old."
}
Delete the REST API
$ aws apigateway delete-rest-api --rest-api-id ryk941lawh
Delete the lambda function
$ aws lambda delete-function \
--function-name HelloLambda
Detach the policy.
$ aws iam detach-role-policy \
--role-name HelloLambdaRole \
--policy-arn arn:aws:iam::aws:policy/AWSLambdaExecute
Delete the role.
$ aws iam delete-role --role-name HelloLambdaRole
In this implementation we'll declare an Open API file where apart from the API definition, there will be the API Gateway integration extensions that are not part of the original specification. They are identified by x-amazon-apigateway prefix.
openapi: "3.0.1"
info:
title: Hello API
version: "1.0"
description: A simple Rest API written using OpenAPI Specification
paths:
Just like the previous example, the first section contains general metadata about the API.
/hello:
get:
summary: Simple Lambda integration
description: Simple Lambda integration with a function previously created.
parameters:
- name: name
in: query
required: true
description: The person's name
schema:
type: string
- name: age
in: query
required: true
description: The person's age
schema:
type: integer
responses:
200:
description: A normal output name / old
content:
application/json:
schema:
type: object
properties:
Name:
type: string
Old:
type: boolean
In this section, we declare the resource /hello and a GET method with two parameters: name and age as well as a response with status code 200.
x-amazon-apigateway-integration:
type: aws
uri: arn:aws:apigateway:eu-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:eu-west-1:123123123123:function:HelloLambda/invocations
httpMethod: POST
requestTemplates:
application/json: |
{
"Name": "$input.params('name')",
"Age": $input.params('age')
}
responses:
"default":
statusCode: 200
responseTemplates:
application/json: |
#set ($root=$input.path('$'))
{
"Message": "Dear $root.Name, you are#if(!$root.Old) not#end old."
}
Here we use the Open API extension x-amazon-apigateway-integration where we define what Lambda function will be invoked. The request template and response template are the same as above, just now taking advantage of YAML syntax that allows multi-line text.
With the Open API file ready, and assuming we still have the Lambda function deployed, we can now import the REST API.
$ aws apigateway import-rest-api --body fileb://openapi.yaml
{
"id": "8l5yjkbj2e",
"name": "Hello API",
"description": "A simple Rest API written using OpenAPI Specification",
"createdDate": "2020-05-01T17:18:56+01:00",
"version": "1.0",
"apiKeySource": "HEADER",
"endpointConfiguration": {
"types": [
"EDGE"
]
}
}
Just like the example using CLI, we need to grant the REST API permission to invoke Lambda function.
$ aws lambda add-permission --function-name HelloLambda \
--statement-id project-api-openapi-invoke-lambda \
--action lambda:InvokeFunction \
--principal apigateway.amazonaws.com \
--source-arn arn:aws:execute-api:eu-west-1:123123123123:fjw1w4q0q8/*/GET/hello
Following the same procedure, create a deployment to get an endpoint.
$ aws apigateway create-deployment \
--rest-api-id fjw1w4q0q8 \
--stage-name dev \
--stage-description "Development Stage" \
--description "Testing dev stage"
Now we can issue the same curl commands as before.
$ curl -X GET "https://fjw1w4q0q8.execute-api.eu-west-1.amazonaws.com/dev/hello?name=Abel&age=33"
$ curl -X GET "https://fjw1w4q0q8.execute-api.eu-west-1.amazonaws.com/dev/hello?name=Abel&age=88"
Delete the REST API
$ aws apigateway delete-rest-api --rest-api-id fjw1w4q0q8
Delete the lambda function
$ aws lambda delete-function \
--function-name HelloLambda
Detach the policy.
$ aws iam detach-role-policy \
--role-name HelloLambdaRole \
--policy-arn arn:aws:iam::aws:policy/AWSLambdaExecute
Delete the role.
$ aws iam delete-role --role-name HelloLambdaRole
In this implementation we'll create a CloudFormation template that contains all the resources we need, it will follow similar structure to the one explained in part 08.
Description: Template to create a serverless web api
Resources:
AWSLambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
- PolicyName: PermitLambda
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: arn:aws:logs:*:*:*
HelloLambda:
Type: AWS::Lambda::Function
Properties:
Handler: project.lambda::project.lambda.Function::HelloHandler
Role: !GetAtt AWSLambdaExecutionRole.Arn
Code:
S3Bucket: abelperez-temp
S3Key: project-lambda/project-lambda.zip
Runtime: dotnetcore3.1
Timeout: 30
In this section we declare the Lambda function and the IAM role associated with an inline policy that allows access to CloudWatch logs. Tha Lambda function uses the same compiled and zipped code in the above steps.
ApiGatewayRestApi:
Type: AWS::ApiGateway::RestApi
Properties:
Name: Serverless API
Description: Serverless API - Using CloudFormation
ApiGatewayResourceHello:
Type: AWS::ApiGateway::Resource
Properties:
ParentId: !GetAtt ApiGatewayRestApi.RootResourceId
PathPart: hello
RestApiId: !Ref ApiGatewayRestApi
Here we declare the REST API and the /hello resource associated with the root resource of the API.
ApiGatewayMethodHelloGet:
Type: AWS::ApiGateway::Method
Properties:
HttpMethod: GET
RequestParameters: {}
ResourceId: !Ref ApiGatewayResourceHello
RestApiId: !Ref ApiGatewayRestApi
ApiKeyRequired: false
AuthorizationType: NONE
Integration:
IntegrationHttpMethod: POST
Type: AWS
Uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${HelloLambda.Arn}/invocations
PassthroughBehavior: NEVER
RequestTemplates:
application/json: |
{
"Name": "$input.params('name')",
"Age": $input.params('age')
}
IntegrationResponses:
- StatusCode: '200'
SelectionPattern: ''
ResponseParameters: {}
ResponseTemplates:
application/json: |
#set ($root=$input.path('$'))
{
"Message": "Dear $root.Name, you are#if(!$root.Old) not#end old."
}
MethodResponses:
- ResponseParameters: {}
ResponseModels: {}
StatusCode: '200'
In this section we declare a GET method for the /hello resource. As part of this method definition, it's the integration, in this case, the Lambda function is also defined in the template and therefore we can use its reference as opposed to the hard-coded function ARN and AWS region.
ApiGatewayLambdaPermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !GetAtt HelloLambda.Arn
Action: lambda:InvokeFunction
Principal: apigateway.amazonaws.com
SourceArn: !Sub arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${ApiGatewayRestApi}/*/GET/hello
Once again, Lambda permission referencing both the Lambda function and the REST API.
ApiGatewayDeployment202005011747:
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId: !Ref ApiGatewayRestApi
StageName: dev
DependsOn:
- ApiGatewayMethodHelloGet
A Deployment resource which contains a timestamp in the name, just like in the previous part, so every time we deploy, it forces the creation of a new resource (and the deletion of the previous one). It's important not to forget to include the explicit dependency on the methods that should be included in the deployment.
Outputs:
ApiEndpoint:
Description: "Endpoint to communicate with the API"
Value: !Sub https://${ApiGatewayRestApi}.execute-api.${AWS::Region}.amazonaws.com/dev/hello
Finally, some output to get the endpoint to test the API.
$ aws cloudformation deploy \
--template-file cloudformation.yaml \
--stack-name project-api-cfn \
--capabilities CAPABILITY_IAM
Now get the outputs.
$ aws cloudformation describe-stacks --stack-name project-api-cfn --query Stacks[*].Outputs
[
[
{
"OutputKey": "ApiEndpoint",
"OutputValue": "https://t2kbgh6ps1.execute-api.eu-west-1.amazonaws.com/dev/hello",
"Description": "Endpoint to communicate with the API"
}
]
]
Issue curl command to test the urls with the above test cases.
$ curl -X GET "https://t2kbgh6ps1.execute-api.eu-west-1.amazonaws.com/dev/hello?name=Abel&age=33"
$ curl -X GET "https://t2kbgh6ps1.execute-api.eu-west-1.amazonaws.com/dev/hello?name=Abel&age=88"
$ aws cloudformation delete-stack --stack-name project-api-cfn
$ aws cloudformation wait stack-delete-complete --stack-name project-api-cfn
We have seen a basic example of Lambda integration with API Gateway using three different implementations: AWS CLI, Open API and CloudFormation. API Gateway is very flexible when it comes to mapping requests / responses from / to Lambda functions. API Gateway integration extensions can be used from an Open API spec file. CloudFormation allows to build a template containing all required resources for a fully functional REST API and Lambda back end.