Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Infra features #3

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
COMPOSE_FILE=docker-compose.yml:docker-compose.monitoring.yml
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.terraform/**/*.x5 filter=lfs diff=lfs merge=lfs -text
16 changes: 16 additions & 0 deletions .github/workflows/cd-application.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: CD - Deploy Application Stack
on:
push:
branches:
- deployment

jobs:
deploy-stack:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Run Docker Compose
run: |
docker compose pull
docker compose up -d
52 changes: 52 additions & 0 deletions .github/workflows/ci-application.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: CI - Build and Push Docker Images
on:
push:
branches:
- integration

jobs:
build-and-push:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Log in to Docker Hub
run: echo "${{ secrets.DOCKER_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_USERNAME }}" --password-stdin

- name: Build Docker Images
run: |
sudo docker compose build frontend
sudo docker compose build backend

- name: Pull Dependencies
run: |
sudo docker compose pull db adminer proxy nginx prometheus grafana loki promtail cadvisor

- name: Tag Docker Images
run: |
sudo docker tag cv-challenge01-frontend:latest otowosamuel/frontend:latest
sudo docker tag cv-challenge01-backend:latest otowosamuel/backend:latest
sudo docker tag postgres:latest otowosamuel/postgres:latest
sudo docker tag adminer:latest otowosamuel/adminer:latest
sudo docker tag jc21/nginx-proxy-manager:latest otowosamuel/nginx-proxy-manager:latest
sudo docker tag nginx:latest otowosamuel/nginx:latest
sudo docker tag prom/prometheus:latest otowosamuel/prometheus:latest
sudo docker tag grafana/grafana:latest otowosamuel/grafana:latest
sudo docker tag grafana/loki:2.8.2 otowosamuel/loki:2.8.2
sudo docker tag grafana/promtail:2.8.2 otowosamuel/promtail:2.8.2
sudo docker tag gcr.io/cadvisor/cadvisor:latest otowosamuel/cadvisor:latest

- name: Push Docker Images
run: |
sudo docker push otowosamuel/frontend:latest
sudo docker push otowosamuel/backend:latest
sudo docker push otowosamuel/postgres:latest
sudo docker push otowosamuel/adminer:latest
sudo docker push otowosamuel/nginx-proxy-manager:latest
sudo docker push otowosamuel/nginx:latest
sudo docker push otowosamuel/prometheus:latest
sudo docker push otowosamuel/grafana:latest
sudo docker push otowosamuel/loki:2.8.2
sudo docker push otowosamuel/promtail:2.8.2
sudo docker push otowosamuel/cadvisor:latest
26 changes: 26 additions & 0 deletions .github/workflows/terraform-apply.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Apply Terraform which triggers ansible

on:
push:
branches:
- infra_main

jobs:
apply:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Google Cloud auth
env:
GOOGLE_APPLICATION_CREDENTIALS: /tmp/account.json
run: |
echo '${{ secrets.GCP_SERVICE_ACCOUNT_KEY }}' > /tmp/account.json
gcloud auth activate-service-account --key-file=/tmp/account.json

- name: Set up Terraform
uses: hashicorp/setup-terraform@v2

- name: Terraform Apply
run: terraform apply -auto-approve
39 changes: 39 additions & 0 deletions .github/workflows/terraform-plan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Terraform Plan and Cost Estimation

on:
pull_request:
branches:
- infra_main

jobs:
plan:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Google Cloud auth
env:
GOOGLE_APPLICATION_CREDENTIALS: /tmp/account.json
run: |
echo '${{ secrets.GCP_SERVICE_ACCOUNT_KEY }}' > /tmp/account.json
gcloud auth activate-service-account --key-file=/tmp/account.json

- name: Set up Terraform
uses: hashicorp/setup-terraform@v2

- name: Terraform Init
run: terraform init

- name: Terraform Plan
run: terraform plan -out=tfplan

- name: InfraCost
uses: infracost/infracost-gh-action@v2
with:
path: .
plan: tfplan
post_condition: always()

- name: Cleanup
run: rm -f /tmp/account.json
33 changes: 33 additions & 0 deletions .github/workflows/terraform-validate.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Terraform Validate

on:
push:
branches:
- infra_features

jobs:
validate:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Google Cloud auth
env:
GOOGLE_APPLICATION_CREDENTIALS: /tmp/account.json
run: |
echo '${{ secrets.GCP_SERVICE_ACCOUNT_KEY }}' > /tmp/account.json
gcloud auth activate-service-account --key-file=/tmp/account.json

- name: Set up Terraform
uses: hashicorp/setup-terraform@v2
with:
terraform_version: 1.5.6

- name: Terraform Init
run: terraform init

- name: Validate Terraform configuration
run: terraform validate


7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Ignore virtual environment
venv/

# Ignore system files
*.pyc
__pycache__/
.DS_Store
10 changes: 0 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
Here’s the updated and fully detailed README with all the additional instructions incorporated:

---

# Full-Stack FastAPI and React Template - DevOps November CV Challenge

Welcome to the **Full-Stack FastAPI and React Template** repository! This project serves as a demo application for the **DevOps November Resume Challenge**, where participants will deploy a full-stack application with a **FastAPI** backend and **ReactJS** frontend using **ChakraUI**. Additionally, participants will set up monitoring and logging tools, configure a reverse proxy, and deploy the application to a cloud platform, showcasing their end-to-end DevOps skills.
Expand Down Expand Up @@ -159,13 +155,7 @@ Your hosted application will be evaluated based on the following criteria:
- Metrics and logs displayed in Grafana.
- Reverse proxy routes and configurations.

3. Use the provided [submission form](#) to share your work.

---

**EXPO**
- To ensure that the services in the Monitoring Stack (e.g., Prometheus, Grafana, cAdvisor, and Loki) are properly routed by the reverse proxy in the Application Stack, both docker-compose.yml files should share the same Docker network. This will enable the reverse proxy in the Application Stack to route traffic to the appropriate services in the Monitoring Stack, despite them being in separate Docker Compose files.
Additionally, having both stacks on the same Docker network allows cAdvisor to collect container metrics and Promtail to gather logs from the containers in the Application Stack.


Good luck with this Challenge! This is your opportunity to showcase your end-to-end DevOps skills and stand out to potential employers.
86 changes: 86 additions & 0 deletions ansible_files/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
- hosts: web_servers
become: true
tasks:
- name: Create Docker network named monitoring-network
docker_network:
name: monitoring-network
driver: bridge

- name: Copy Promtail config
copy:
content: |
server:
http_listen_port: 9080
grpc_listen_port: 0

clients:
- url: http://loki:3100/loki/api/v1/push

positions:
filename: /tmp/positions.yaml

scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/*.log
dest: ./promtail-config.yml

- name: Copy Loki config
copy:
content: |
server:
http_listen_port: 9080
grpc_listen_port: 0

positions:
filename: /tmp/positions.yaml

clients:
- url: http://loki:3100/loki/api/v1/push

scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/*log

- job_name: docker
static_configs:
- targets:
- localhost
labels:
job: docker_logs
__path__: /var/lib/docker/containers/*/*-json.log
dest: ./loki-config.yaml

- name: Copy Prometheus config
copy:
content: |
global:
scrape_interval: 15s

scrape_configs:
- job_name: 'prometheus'
metrics_path: "/prometheus/metrics"
static_configs:
- targets: ['prometheus:9090']

- job_name: 'grafana'
static_configs:
- targets: ['grafana:3000']

- job_name: 'cadvisor'
static_configs:
- targets: ['cadvisor:8080']

- job_name: 'loki'
static_configs:
- targets: ['loki:3100']
dest: ./prometheus.yml
Loading