Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: copilot downgrade aurora to v1. #176

Merged
merged 5 commits into from
Jan 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 2 additions & 14 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,35 +6,23 @@ on:
jobs:
test:
timeout-minutes: 60
runs-on: macos-latest
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/[email protected]
- name: Install docker
run: |
brew install docker docker-compose
mkdir -p ~/.docker/cli-plugins
ln -sfn $(brew --prefix)/opt/docker-compose/bin/docker-compose ~/.docker/cli-plugins/docker-compose
colima start --arch x86_64
- name: Apt Pkgs
if: runner.os == 'ubuntu'
uses: awalsh128/cache-apt-pkgs-action@latest
with:
packages: libnss3-tools
version: 1.3.1
- name: Install mkcert
if: runner.os == 'ubuntu'
run: |
curl -JLO "https://dl.filippo.io/mkcert/latest?for=linux/amd64"
chmod +x mkcert-v*-linux-amd64
sudo cp mkcert-v*-linux-amd64 /usr/local/bin/mkcert
- name: Install mkcert (macos)
if: runner.os == 'macos'
run: |
brew install mkcert
brew install nss
- name: Install Certificates
run: |
mkcert -install
mkcert -install
mkcert -key-file busmap.localhost-key.pem -cert-file busmap.localhost.pem busmap.localhost localhost
cp *.pem packages/web/certs
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,72 +4,65 @@ Parameters:
Description: Your application's name.
Env:
Type: String
Description: The name of the environment being deployed.
Description: The environment name your service, job, or workflow is being deployed to.
Name:
Type: String
Description: Your workload's name.
# Customize your Aurora Serverless cluster by setting the default value of the following parameters.
bmclusterDBName:
Type: String
Description: The name of the initial database to be created in the Aurora Serverless v2 cluster.
Description: The name of the initial database to be created in the DB cluster.
Default: busmap
# Cannot have special characters
# Naming constraints: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Limits.html#RDS_Limits.Constraints
VPCID:
Type: String
Description: The ID of the VPC in which to create the Aurora Serverless v2 cluster.
Default: ''
PrivateSubnets:
Type: String
Description: The IDs of the private subnets in which to create the Aurora Serverless v2 cluster.
Default: ''

bmclusterDBAutoPauseSeconds:
Type: Number
Description: The duration in seconds before the cluster pauses.
Default: 1000
Mappings:
bmclusterEnvScalingConfigurationMap:
test:
'DBMinCapacity': 0.5 # AllowedValues: from 0.5 through 128
'DBMaxCapacity': 1 # AllowedValues: from 0.5 through 128
'DBMinCapacity': 2 # AllowedValues: [2, 4, 8, 16, 32, 64, 192, 384]
'DBMaxCapacity': 4 # AllowedValues: [2, 4, 8, 16, 32, 64, 192, 384]
Comment on lines +25 to +26
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather DBMinCapacity = DBMaxCapacity = 1.


All:
'DBMinCapacity': 0.5 # AllowedValues: from 0.5 through 128
'DBMaxCapacity': 1 # AllowedValues: from 0.5 through 128
'DBMinCapacity': 2 # AllowedValues: [2, 4, 8, 16, 32, 64, 192, 384]
'DBMaxCapacity': 4 # AllowedValues: [2, 4, 8, 16, 32, 64, 192, 384]

Resources:
bmclusterDBSubnetGroup:
Type: 'AWS::RDS::DBSubnetGroup'
Properties:
DBSubnetGroupDescription: Group of private subnets for Aurora Serverless v2 cluster.
SubnetIds: !Split [',', !Ref PrivateSubnets]

bmclusterWorkloadSecurityGroup:
DBSubnetGroupDescription: Group of Copilot private subnets for Aurora cluster.
SubnetIds: !Split [',', { 'Fn::ImportValue': !Sub '${App}-${Env}-PrivateSubnets' }]
bmclusterSecurityGroup:
Metadata:
'aws:copilot:description': 'A security group for one or more workloads to access the Aurora Serverless v2 cluster bmcluster'
'aws:copilot:description': 'A security group for your workload to access the DB cluster bmcluster'
Type: 'AWS::EC2::SecurityGroup'
Properties:
GroupDescription: 'The Security Group to access Aurora Serverless v2 cluster bmcluster.'
VpcId: !Ref VPCID
GroupDescription: !Sub 'The Security Group for ${Name} to access DB cluster bmcluster.'
VpcId:
Fn::ImportValue: !Sub '${App}-${Env}-VpcId'
Tags:
- Key: Name
Value: !Sub 'copilot-${App}-${Env}-Aurora'

Value: !Sub 'copilot-${App}-${Env}-${Name}-Aurora'
bmclusterDBClusterSecurityGroup:
Metadata:
'aws:copilot:description': 'A security group for your Aurora Serverless v2 cluster bmcluster'
'aws:copilot:description': 'A security group for your DB cluster bmcluster'
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: The Security Group for the Aurora Serverless v2 cluster.
VpcId: !Ref VPCID
GroupDescription: The Security Group for the database cluster.
SecurityGroupIngress:
- ToPort: 5432
FromPort: 5432
IpProtocol: tcp
Description: !Sub 'From the Aurora Security Group of the workload ${Name}.'
SourceSecurityGroupId: !Ref bmclusterSecurityGroup
VpcId:
Fn::ImportValue: !Sub '${App}-${Env}-VpcId'
Tags:
- Key: Name
Value: !Sub 'copilot-${App}-${Env}-Aurora'

bmclusterDBClusterSecurityGroupIngressFromWorkload:
Type: AWS::EC2::SecurityGroupIngress
Properties:
Description: Ingress from one or more workloads in the environment.
GroupId: !Ref bmclusterDBClusterSecurityGroup
IpProtocol: tcp
ToPort: 5432
FromPort: 5432
SourceSecurityGroupId: !Ref bmclusterWorkloadSecurityGroup

Value: !Sub 'copilot-${App}-${Env}-${Name}-Aurora'
bmclusterAuroraSecret:
Metadata:
'aws:copilot:description': 'A Secrets Manager secret to store your DB credentials'
Expand All @@ -88,13 +81,12 @@ Resources:
Type: 'AWS::RDS::DBClusterParameterGroup'
Properties:
Description: !Ref 'AWS::StackName'
Family: 'aurora-postgresql15'
Family: 'aurora-postgresql13'
Parameters:
client_encoding: 'UTF8'

bmclusterDBCluster:
Metadata:
'aws:copilot:description': 'The bmcluster Aurora Serverless v2 database cluster'
'aws:copilot:description': 'The bmcluster Aurora Serverless database cluster'
Type: 'AWS::RDS::DBCluster'
Properties:
MasterUsername:
Expand All @@ -117,48 +109,30 @@ Resources:
]
DatabaseName: !Ref bmclusterDBName
Engine: 'aurora-postgresql'
EngineVersion: '15.5'
EngineVersion: '13.9' # LTS versions of PostgreSQL for Aurora Serverless v1 are v13.9 and v11.9
EngineMode: serverless
DBClusterParameterGroupName: !Ref bmclusterDBClusterParameterGroup
DBSubnetGroupName: !Ref bmclusterDBSubnetGroup
Port: 5432
VpcSecurityGroupIds:
- !Ref bmclusterDBClusterSecurityGroup
ServerlessV2ScalingConfiguration:
ScalingConfiguration:
AutoPause: true
# Replace "All" below with "!Ref Env" to set different autoscaling limits per environment.
MinCapacity:
!FindInMap [bmclusterEnvScalingConfigurationMap, !Ref Env, DBMinCapacity]
MaxCapacity:
!FindInMap [bmclusterEnvScalingConfigurationMap, !Ref Env, DBMaxCapacity]

bmclusterDBWriterInstance:
Metadata:
'aws:copilot:description': 'The bmcluster Aurora Serverless v2 writer instance'
Type: 'AWS::RDS::DBInstance'
Properties:
DBClusterIdentifier: !Ref bmclusterDBCluster
DBInstanceClass: db.serverless
Engine: 'aurora-postgresql'
PromotionTier: 1
AvailabilityZone: !Select
- 0
- !GetAZs
Ref: AWS::Region

SecondsUntilAutoPause: !Ref bmclusterDBAutoPauseSeconds
bmclusterSecretAuroraClusterAttachment:
Type: AWS::SecretsManager::SecretTargetAttachment
Properties:
SecretId: !Ref bmclusterAuroraSecret
TargetId: !Ref bmclusterDBCluster
TargetType: AWS::RDS::DBCluster

Outputs:
bmclusterSecret:
bmclusterSecret: # injected as BMCLUSTER_SECRET environment variable by Copilot.
Description: "The JSON secret that holds the database username and password. Fields are 'host', 'port', 'dbname', 'username', 'password', 'dbClusterIdentifier' and 'engine'"
Value: !Ref bmclusterAuroraSecret
Export:
Name: !Sub ${App}-${Env}-bmclusterAuroraSecret
bmclusterSecurityGroup:
Description: 'The security group to attach to the workload.'
Value: !Ref bmclusterWorkloadSecurityGroup
Export:
Name: !Sub ${App}-${Env}-bmclusterSecurityGroup
Value: !Ref bmclusterSecurityGroup
5 changes: 0 additions & 5 deletions copilot/api/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,6 @@ command: ['node', 'packages/api/dist/index.js']

network:
connect: true
vpc:
security_groups:
- from_cfn: ${COPILOT_APPLICATION_NAME}-${COPILOT_ENVIRONMENT_NAME}-bmclusterSecurityGroup

sidecars:
nginx:
Expand Down Expand Up @@ -61,8 +58,6 @@ secrets:
POSTGRES_PASSWORD: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/BM_POSTGRES_PASSWORD
SSO_GOOG_CLIENT_SECRET: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/SSO_GOOG_CLIENT_SECRET
BM_COOKIE_SECRET: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/BM_COOKIE_SECRET
DB_SECRET:
from_cfn: ${COPILOT_APPLICATION_NAME}-${COPILOT_ENVIRONMENT_NAME}-bmclusterAuroraSecret

environments:
test:
Expand Down
3 changes: 0 additions & 3 deletions copilot/environments/addons/addons.parameters.yml

This file was deleted.

4 changes: 2 additions & 2 deletions packages/api/src/db.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ let password = env.POSTGRES_PASSWORD
/**
* Check for environment variable injected by AWS copilot.
*/
if (env.DB_SECRET) {
const secrets = JSON.parse(env.DB_SECRET) as AuroraCluster
if (env.BMCLUSTER_SECRET) {
const secrets = JSON.parse(env.BMCLUSTER_SECRET) as AuroraCluster

host = secrets.host
port = secrets.port
Expand Down
2 changes: 1 addition & 1 deletion playwright.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ export default defineConfig({
ignoreHTTPSErrors: true,
url: 'https://localhost/healthcheck',
reuseExistingServer: !process.env.CI,
timeout: 60_000 * 5,
timeout: 60_000 * 7,
stdout: 'pipe'
}
})