Skip to content

Commit

Permalink
QoL update: pulumi + readme + harbor
Browse files Browse the repository at this point in the history
  • Loading branch information
ozhyhin-nikita committed Oct 13, 2023
1 parent 864564d commit 68c6246
Show file tree
Hide file tree
Showing 8 changed files with 219 additions and 43 deletions.
15 changes: 14 additions & 1 deletion .github/workflows/bootstrap.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
uses: bn-digital/vault/.github/workflows/import-secrets.yml@latest
secrets: inherit

database:
bootstrap:
needs: [secrets]
runs-on: self-hosted
steps:
Expand All @@ -24,6 +24,19 @@ jobs:
infrastructure/data/digitalocean token | DIGITALOCEAN_TOKEN ;
infrastructure/data/kubernetes cluster | KUBERNETES_CLUSTER ;
infrastructure/data/postgresql password | PGPASSWORD;
accounts/data/dcr.bndigital.dev api_token | HARBOR_TOKEN;
- name: Create project in Harbor
run: |
curl -X 'POST' \
'https://dcr.bndigital.dev/api/v2.0/projects' \
-H 'authorization: Basic ${{ secrets.HARBOR_TOKEN }}' \
-H 'Content-Type: application/json' \
-d '{
"project_name": "${{ github.event.repository.name }}",
"public": false,
"storage_limit": 10737418240
}'
- name: Setup DigitalOcean cli
uses: digitalocean/action-doctl@v2
with:
Expand Down
19 changes: 14 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,23 +13,32 @@
To ensure that WFs are working properly, add these GH Secrets right after project creation:
- [ ] Secrets for `VAULT_TOKEN` and `VAULT_ENDPOINT` (copy `github-token` and `url` from [https://vault.bndigital.dev](https://vault.bndigital.dev/ui/vault/secrets/infrastructure/show/vault))
- [ ] Secret for `GH_TOKEN` (copy `github-token` from [https://vault.bndigital.dev](https://vault.bndigital.dev/ui/vault/secrets/accounts/show/github/bn-enginseer))
- [ ] Secret for `HARBOR_TOKEN` (copy `api-token` from [https://vault.bndigital.dev](https://vault.bndigital.dev/ui/vault/secrets/accounts/show/dcr.bndigital.dev))

## Staging Rollout

After setting up secrets, perform these actions:
- [ ] Create new project in [https://dcr.bndigital.dev/] (click `+ New project`, in pop-up window only enter `Project Name`, left other fields unchanged and click `OK` (credentials are here: [https://vault.bndigital.dev](https://vault.bndigital.dev/ui/vault/secrets/accounts/show/dcr.bndigital.dev)))
After setting up **_all_**(!) required secrets, perform these actions:
- [ ] In the `Actions` tab, find WF named `Bootstrap` in the left pane
- [ ] Click on the button `Run workflow` in the top-right corner and in pop-up window click green button `Run workflow`
- [ ] After that replace all occurrences of `project-templates` with the name of your project
- [ ] After that replace all occurrences of `"project-templates"` in all `package.json` files with the name of your project, run `yarn install` from root, then commit and push these changes
- [ ] Wait for the `Staging Deployment` workflow to run and check your staging at `REPO_NAME.bndigital.dev`
- [ ] Go to `REPO_NAME.bndigital.dev/admin` and set up Strapi Admin with [these](https://vault.bndigital.dev/ui/vault/secrets/templates/show/project/staging/strapi) credentials
- [ ] Click on the `Settings` option in the menu on the left and on the page that appeared click `Transfer tokens` and then `+ Create new Transfer Token` button inn the top-right corner
- [ ] In the appeared window enter these values:
- For `Name` any name you like (e.g staging_token)
- For `Token duration` select `30 days`
- For `Description` don't put anything
- For `Token type` select `Full access`
- [ ] Then copy appeared token, press `Save` and save it as a GH secret named `STRAPI_STAGING_TRANSFER_TOKEN`
- [ ] Finally, run workflow named `Transfer` the same way you ran `Bootstrap`
- [ ] Say a few kind words to yourself after completing all steps 🙃

## Documentation & References

- [React Documentation](https://react.dev/reference/react)
- [Strapi CMS Documentation](https://docs.strapi.io/developer-docs/latest/getting-started/introduction.html)
- [Ant Design Components](https://ant.design/components/overview/)
- [Github Actions](https://docs.github.com/en/actions)
- [GitHub Actions](https://docs.github.com/en/actions)
- [Build and run as Docker container](docs/docker.md)
- [Environment variables](docs/env-variables.md)
- [Backend object storage](docs/storage.md)
Expand All @@ -40,7 +49,7 @@ After setting up secrets, perform these actions:

[Run in sandbox](https://codesandbox.io/p/github/bn-digital/project-templates/latest)

### Operations tools
### Official documentation for tools in use

- [docker](https://docs.docker.com/)
- [kubectl](https://github.com/kubernetes/kubectl)
Expand Down
19 changes: 0 additions & 19 deletions packages/cloud/Pulumi.staging.yaml

This file was deleted.

21 changes: 13 additions & 8 deletions packages/cloud/package.json
Original file line number Diff line number Diff line change
@@ -1,18 +1,23 @@
{
"name": "@project-templates/cloud",
"version": "2023.6.8",
"version": "1.0.0",
"main": "src/index.ts",
"eslintConfig": {
"extends": "@bn-digital/eslint-config/typescript"
},
"scripts": {
"pulumi": "pulumi",
"test": "pulumi up --policy-pack=tests",
"provision": "yarn pulumi up --yes"
"provision": "pulumi up --non-interactive --stack=production --yes"
},
"dependencies": {
"@bn-digital/pulumi": "1.7.51",
"@pulumi/pulumi": "3.69.x"
"@pulumi/digitalocean": "^4.17.0",
"@pulumi/kubernetes": "^3.23.1",
"@pulumi/pulumi": "^3.62.0",
"@pulumi/vault": "^5.8.0",
"dotenv": "^16.0.3"
},
"devDependencies": {
"@bn-digital/typescript-config": "^1.3.6",
"@pulumi/policy": "^1.5.0"
"@bn-digital/typescript-config": "^1.3.1",
"@types/node": "^18.15.11",
"typescript": "^5.0.4"
}
}
165 changes: 165 additions & 0 deletions packages/cloud/src/digitalocean.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
import {
Cdn,
Certificate,
DnsRecord,
Domain,
DropletSlug,
KubernetesCluster,
Project,
Region,
SpacesBucket,
SpacesBucketPolicy,
getKubernetesVersions,
CdnArgs,
DnsRecordArgs,
DomainArgs,
KubernetesClusterArgs,
ProjectArgs,
SpacesBucketArgs,
} from "@pulumi/digitalocean"
import { Config, Input, Output } from "@pulumi/pulumi"

const config = new Config()
const name = config.name
const {
region,
domain: dns = "",
cdn = false,
} = config.requireObject<{ region: Region; cdn?: boolean; domain?: string }>("digitalocean")
const environment = "production"
const tags = ["provisioner:pulumi", `environment:${environment}`, `app:${name}`]

type ResourceID = "provider" | "cluster" | "storage" | "cdn" | "dns" | "project" | "certificate" | "policy"

function urn(resource: ResourceID, ...tags: string[]) {
return [["bn", name, environment].join(":"), [resource as string].concat(tags).filter(Boolean).join(":")].join("/")
}

function getCmsPolicy(bucket: string | Input<string>) {
return {
Version: "2012-10-17",
Statement: [
{
Sid: "CmsPublicAccess",
Effect: "Allow",
Action: ["s3:PutObject", "s3:GetObject", "s3:ListBucket", "s3:DeleteObject", "s3:PutObjectAcl"],
Resource: [`arn:aws:s3:::${bucket}`, `arn:aws:s3:::${bucket}/*`],
},
],
}
}

/**
* Create a DigitalOcean Spaces bucket required for CMS uploads and assets
* @param {Domain} domain
*/
function createBucket(domain?: string): SpacesBucket {
const bucket = new SpacesBucket(
urn("storage", "cms"),
{
acl: "public-read",
name: `${name}-cms`,
region,
versioning: { enabled: false },
forceDestroy: true,
},
{ ignoreChanges: ["name", "region"] as (keyof SpacesBucketArgs)[] }
)
new SpacesBucketPolicy(
urn("storage", "cms", "policy"),
{ policy: JSON.stringify(getCmsPolicy(`${name}-cms`)), region, bucket: `${name}-cms` },
{
dependsOn: [bucket],
}
)
if (cdn) {
const cdnRecord = new DnsRecord(
urn("storage", "cms", "cdn", "dns"),
{ type: "CNAME", name: "cdn", value: bucket.bucketDomainName.apply(fqdn => `${fqdn}.`), domain: dns },
{
ignoreChanges: [] as (keyof DnsRecordArgs)[],
}
)
const certificate = new Certificate(
urn("storage", "cms", "cdn", "certificate"),
{
domains: [`cdn.${dns}`],
name,
type: "lets_encrypt",
},
{ ignoreChanges: ["name"] as (keyof DomainArgs)[], dependsOn: [cdnRecord] }
)
new Cdn(
urn("storage", "cms", "cdn"),
{
origin: bucket.bucketDomainName,
customDomain: cdnRecord.fqdn,
certificateName: certificate.name,
},
{ ignoreChanges: [] as (keyof CdnArgs)[], dependsOn: [certificate, cdnRecord, bucket] }
)
}

return bucket
}

function createDomain(name: string): Domain {
return new Domain(urn("dns"), {
name,
})
}

/**
* @param name
* @param version
*/
function createCluster(name: string, version: string): KubernetesCluster {
return new KubernetesCluster(
urn("cluster"),
{
ha: false,
surgeUpgrade: true,
autoUpgrade: false,
name,
nodePool: {
name: "production",
size: DropletSlug.DropletS2VCPU4GB_INTEL,
minNodes: 1,
maxNodes: 2,
autoScale: true,
},
region,
version,
tags,
},
{ ignoreChanges: ["name", "version", "region"] as (keyof KubernetesClusterArgs)[] }
)
}

function createProject(resources: Output<string>[]): Project {
return new Project(
urn("project"),
{
environment,
name,
purpose: "Web Application",
resources,
},
{ ignoreChanges: [] as (keyof ProjectArgs)[] }
)
}

export async function run() {
const name = config.name
const version = await getKubernetesVersions().then(versions => versions.latestVersion)
const cluster = createCluster(name, version)
const resources = [cluster.clusterUrn]
let domain = undefined
if (dns) {
domain = createDomain(dns)
resources.push(domain.domainUrn)
}
const bucket = domain?.name.apply(createBucket) || createBucket()
resources.push(bucket.bucketUrn)
return createProject(resources)
}
10 changes: 5 additions & 5 deletions packages/cloud/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { configure, providers, stacks } from "@bn-digital/pulumi"
import { Config } from "@pulumi/pulumi"
import "./types"
import { config } from "dotenv"

const spec = new Config("project").requireObject<ConfigSpec>("spec")
import { run } from "./digitalocean"

configure(spec).cloud(stacks.digitalocean.CloudNativeWebApp).deploy(providers.helm.WebAppDeployment).release()
config()

run().then()
2 changes: 1 addition & 1 deletion packages/cloud/src/types/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ declare global {
type ConfigSpec = stacks.InfrastructureConfig & providers.helm.AppConfig
}

export {}
export {}
11 changes: 7 additions & 4 deletions packages/cloud/tsconfig.json
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
{
"extends": "@bn-digital/typescript-config/pulumi",
"extends": "@bn-digital/typescript-config/pulumi.json",
"compilerOptions": {
"baseUrl": ".",
"outDir": "bin",
"lib": ["esnext", "dom"]
"esModuleInterop": true,
"outDir": "build",
"noEmit": true,
"target": "ESNext",
"resolveJsonModule": true
},
"include": ["./src", "./package.json"]
"include": ["src", "./package.json"]
}

0 comments on commit 68c6246

Please sign in to comment.