CI/CD: Research + Design

[cyri113]

Description

The existing CI/CD process has gaps and bottlenecks that are stoping the team from effectively delivering quality code into production.

Pain Points

• There is no linting, automated testing, and security review throughout the CI/CD process.
• There are no gate (code coverage, maintainability)
• UAT only happens in staging, where features are no longer isolated. This creates bottlenecks as we are unable to push to production unless all problems are resolved.
• We do not have the ability to run E2E tests for a specific feature

Considerations

• More repositories will be created overtime
• We should aim to make configurations reusable, so that it is easy to add a new repo
• We should have linting, automated testing (unit + integration), and a security review (need to pick a tool)
• We are using CodeClimate for test coverage + code review (maintenance) feedback
• We should not allow a merge if the test coverage is reduced.
• We want to be able to run automated E2E tests for each feature
• We want to be able to complete user acceptance testing (UAT) for each feature

Current Repositories

• https://github.com/RnDAO/tc-botComm (Node)
• https://github.com/RnDAO/tc-landing (Node)
• https://github.com/RnDAO/tc-DAOlytics (Python)
• https://github.com/RnDAO/tc-uiComm (Node)
• https://github.com/RnDAO/tc-serverComm (Node)

Tasks

• Research best practices for CI/CD for our micro-service architecture use case
• Design a CI/CD pipeline diagram that demonstrates different stages and planned technology

Documentation

• https://www.youtube.com/watch?v=TAP8vVbsBXQ
• https://www.redhat.com/en/topics/devops/what-is-blue-green-deployment
• https://miro.com/app/board/uXjVMPos_JU=/
• https://kubernetes.io/docs/tutorials/kubernetes-basics/
• https://discord.com/channels/915914985140531240/1101582686839111831/1104061920694247575

[covendev]

Approach that can be used:

1. AWS ECR+AWS CodePipeline+ AWS EC2 / Serverless
2. Kubernetes+ Github workflows(for pipelines) + AWS EC2/serverless
   Both approaches use GitHub actions in workflow.

Github actions in workflows

• CodeClimate - Code Coverage GitHub action : https://github.com/marketplace/actions/code-climate-coverage-action-test
• Lint action with multiple language support(mainly py and js which we are presently using): https://github.com/wearerequired/lint-action
  example workflow: https://github.com/Uniswap/routing-api/blob/main/.github/workflows/lint.yml
• E2E - https://github.com/cypress-io/github-action
  example workflow : https://github.com/Uniswap/interface/blob/main/.github/workflows/test.yml
• Release tag action - mathieudutour/[email protected] (creates release tag)

[cyri113]

@covendev do we have a pipeline design out yet for review yet?

Personally, I rather stay as platform agnostic as possible. Therefore I would lean towards option 2.

[covendev]

@covendev do we have a pipeline design out yet for review yet?

Personally, I rather stay as platform agnostic as possible. Therefore I would lean towards option 2.

agree with that approach too. Have updated workflow design in existing miro and also added comments to make it more readable

[covendev]

PR title checker : https://github.com/marketplace/actions/pr-title-checker