Skip to content

voteflux/flux-ga-to-s3

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AWS Serverless Application

This is a sample template for AWS Serverless Application - Below is a brief explanation of what we have generated for you:

.
├── README.md                   <-- This instructions file
├── hello_world                 <-- Source code for a lambda function
│   ├── __init__.py
│   ├── app.py                  <-- Lambda function code
│   └── requirements.txt        <-- Python dependencies
├── template.yaml               <-- SAM Template
└── tests                       <-- Unit tests
    └── unit
        ├── __init__.py
        └── test_handler.py

Requirements

Setup process

Building the project

AWS Lambda requires a flat folder with the application as well as its dependencies. When you make changes to your source code or dependency manifest, run the following command to build your project local testing and deployment:

sam build

If your dependencies contain native modules that need to be compiled specifically for the operating system running on AWS Lambda, use this command to build inside a Lambda-like Docker container instead:

sam build --use-container

By default, this command writes built artifacts to .aws-sam/build folder.

Local development

Invoking function locally through local API Gateway

sam local start-api

If the previous command ran successfully you should now be able to hit the following local endpoint to invoke your function http://localhost:3000/hello

SAM CLI is used to emulate both Lambda and API Gateway locally and uses our template.yaml to understand how to bootstrap this environment (runtime, where the source code is, etc.) - The following excerpt is what the CLI will read in order to initialize an API and its routes:

...
Events:
    HelloWorld:
        Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
        Properties:
            Path: /hello
            Method: get

Packaging and deployment

AWS Lambda Python runtime requires a flat folder with all dependencies including the application. SAM will use CodeUri property to know where to look up for both application and dependencies:

...
    HelloWorldFunction:
        Type: AWS::Serverless::Function
        Properties:
            CodeUri: ga_to_s3
            ...

Firstly, we need a S3 bucket where we can upload our Lambda functions packaged as ZIP before we deploy anything - If you don't have a S3 bucket to store code artifacts then this is a good time to create one:

aws s3 mb s3://BUCKET_NAME

Next, run the following command to package our Lambda function to S3:

sam package \
    --output-template-file packaged.yaml \
    --s3-bucket REPLACE_THIS_WITH_YOUR_S3_BUCKET_NAME

Next, the following command will create a Cloudformation Stack and deploy your SAM resources.

sam deploy \
    --template-file packaged.yaml \
    --stack-name aws-serverless-application \
    --capabilities CAPABILITY_IAM

See Serverless Application Model (SAM) HOWTO Guide for more details in how to get started.

After deployment is complete you can run the following command to retrieve the API Gateway Endpoint URL:

aws cloudformation describe-stacks \
    --stack-name aws-serverless-application \
    --query 'Stacks[].Outputs'

Testing

We use Pytest and pytest-mock for testing our code and you can install it using pip: pip install pytest pytest-mock

Next, we run pytest against our tests folder to run our initial unit tests:

python -m pytest tests/ -v

NOTE: It is recommended to use a Python Virtual environment to separate your application development from your system Python installation.

Appendix

Python Virtual environment

In case you're new to this, python3 comes with virtualenv library by default so you can simply run the following:

  1. Create a new virtual environment
  2. Install dependencies in the new virtual environment
python3 -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt

NOTE: You can find more information about Virtual Environment at Python Official Docs here. Alternatively, you may want to look at Pipenv as the new way of setting up development workflows

AWS CLI commands

AWS CLI commands to package, deploy and describe outputs defined within the cloudformation stack:

sam package \
    --output-template-file packaged.yaml \
    --s3-bucket REPLACE_THIS_WITH_YOUR_S3_BUCKET_NAME

sam deploy \
    --template-file packaged.yaml \
    --stack-name aws-serverless-application \
    --capabilities CAPABILITY_IAM \
    --parameter-overrides MyParameterSample=MySampleValue

aws cloudformation describe-stacks \
    --stack-name aws-serverless-application --query 'Stacks[].Outputs'

Bringing to the next level

Here are a few ideas that you can use to get more acquainted as to how this overall process works:

  • Create an additional API resource (e.g. /hello/{proxy+}) and return the name requested through this new path
  • Update unit test to capture that
  • Package & Deploy

Next, you can use the following resources to know more about beyond hello world samples and how others structure their Serverless applications:

About

grab GA data and dump in S3 for graphs and stuff

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages