Initial Deployment Docs

infrastructure-documenation/deployment/README.md

@@ -0,0 +1,89 @@
+# Eclipse PASS Deployment Guide
+
+## Table of Contents
+1. [Summary](#summary)
+2. [AWS Infrastructure Components](#aws-infrastructure-components)
+3. [Deep Dive: Deployment & Release](#deep-dive-deployment--release)
+4. [PASS Deployment Process](#pass-deployment-process)
+5. [PASS Release Process](#pass-release-process)
+6. [Related Information](#related-information)
+
+## Summary
+
+The Public Access Submission System (PASS) is an open-source platform designed to streamline compliance with funder and institutional open access policies. This guide outlines the deployment process for PASS, which is adaptable to various architectures including cloud, hybrid, or on-premises environments.
+
+> **Note**: PASS is transitioning towards a cloud-native version. Expect ongoing changes to the architecture, infrastructure, and deployment process, such as moving from Docker Compose to Kubernetes or implementing Infrastructure as Code with Terraform. See (roadmap)[./roadmap.md] for more information.
+
+## AWS Infrastructure Components
+
+The current PASS infrastructure in AWS includes:
+
+- **EC2**: Hosts Docker Compose
+- **ECS**: Hosts auxiliary microservices
+- **RDS**: Stores metadata
+- **S3**: Stores binary data (managed by OCFL)
+- **ALB**: Provides SSL for the frontend
+- **WAF**: Protects the frontend
+
+## Deep Dive: Deployment & Release 
+
+### Prerequisites
+- Docker and Docker Compose
+- Git
+
+### PASS Deployment Process
+
+1. Install dependencies:
+   ```bash
+   apt-get -y update
+   apt-get install -y gnupg2 pass docker-compose
+   ```
+
+2. Clone the repository:
+   ```bash
+   mkdir -p /src
+   cd /src
+   git clone [email protected]:eclipse-pass/pass-docker.git
+   cd pass-docker
+   git checkout minimal-assets
+   ```
+
+3. Run PASS:
+   ```bash
+   cd /src/pass-docker && \
+     docker-compose pull && \
+     docker-compose up
+   ```
+
+### PASS Release Process
+
+PASS uses semantic versioning (`MAJOR.MINOR.PATCH`). The release process includes:
+
+1. Code contribution
+2. CI/CD via GitHub Actions
+3. Building and testing
+4. Generating release artifacts (Java artifacts and Docker images)
+5. Publishing artifacts to repositories
+6. Triggering deployment via AWS SQS
+7. Updating infrastructure with new artifacts
+8. Generating release notes
+
+#### GitHub Actions Release Workflow
+
+The "Publish: Release All" workflow automates the release process:
+
+1. Builds and tests components
+2. Publishes Java artifacts
+3. Builds and pushes Docker images
+4. Creates GitHub Releases
+
+For detailed configuration, refer to the [pass-complete-release.yml](https://github.com/eclipse-pass/main/blob/main/.github/workflows/pass-complete-release.yml) Actions workflow.
+
+Please refer to [github-cicd.md](./github-cicd.md) for further information on its useage.
+
+## Related Information
+
+- [PASS main repository](https://github.com/eclipse-pass/main)
+- [PASS Docker repository](https://github.com/eclipse-pass/pass-docker)
+
+For further assistance or questions, please open an issue in the [PASS main repository](https://github.com/eclipse-pass/main/issues) or find us in the [PASS Slack](https://eclipse-pass.slack.com).

infrastructure-documenation/deployment/github-cicd.md

@@ -0,0 +1,142 @@
+# GitHub CI/CD Guide
+
+## Summary
+
+GitHub Actions is a powerful [CI/CD](./github-cicd.md) platform integrated directly into GitHub repositories. It allows you to automate various software development workflows, including building, testing, and [deploying](./README.md) your code.
+
+### Key Concepts
+
+1. **Workflows**: YAML files that define a set of jobs to be executed when triggered by an event.
+2. **Jobs**: A set of steps that execute on the same runner.
+3. **Steps**: Individual tasks that can run commands or actions.
+4. **Actions**: Reusable units of code that can be shared across workflows.
+5. **Events**: Specific activities that trigger a workflow run.
+
+### Workflow Action Structure
+
+```yaml
+name: CI
+
+on: [push, pull_request]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Run a script
+        run: echo Hello, world!
+```
+
+This simple workflow runs on push and pull request events, checks out the repository, and runs a simple command.
+
+## GitHub Secrets
+
+GitHub secrets are encrypted environment variables used to store sensitive information securely. They are crucial for handling authentication and other confidential data in your workflows.
+
+### Types of Secrets
+
+1. **Organization Secrets**: Available to all repositories in the `eclipse-pass` organization.
+2. **Repository Secrets**: Specific to a single repository.
+3. **Environment Secrets**: Tied to a specific environment within a repository.
+
+### Creating Secrets
+
+Due to permission restrictions, PASS project members should use the provided Python script to create repository or environment secrets:
+
+```bash
+python github_secrets.py -u <username> -t <token> -r <repo> -n <name> -v <value> [-e <environment>]
+```
+
+For organization secrets, open a ticket with the [Eclipse Help Desk](https://gitlab.eclipse.org/eclipsefdn/helpdesk).
+
+### Using Secrets in Workflows
+
+Reference secrets in your workflows like this:
+
+```yaml
+${{ secrets.SECRET_NAME }}
+```
+
+For reusable workflows, pass secrets explicitly:
+
+```yaml
+jobs:
+  call-publish-docker:
+    uses: ./.github/workflows/docker-publish.yml
+    secrets:
+      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
+      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+```
+
+In the called workflow, declare expected secrets:
+
+```yaml
+on:
+  workflow_call:
+    secrets:
+      AWS_ACCESS_KEY_ID:
+        required: true
+      AWS_SECRET_ACCESS_KEY:
+        required: true
+```
+### Best Practices
+
+1. Use reusable workflows for common tasks to maintain DRY principles.
+2. Leverage GitHub-hosted runners when possible to reduce maintenance overhead.
+3. Use environment protection rules for sensitive deployments.
+4. Regularly audit and rotate your secrets.
+5. Use GitHub Actions marketplace for pre-built actions to speed up development.
+
+## AWS Integration
+
+To interact with AWS services, including ECR (Elastic Container Registry), you'll need to set up appropriate secrets and use AWS-specific actions in your workflows.
+
+### Setting up AWS Credentials
+
+Store your AWS credentials as secrets:
+
+1. `AWS_ACCESS_KEY_ID`
+2. `AWS_SECRET_ACCESS_KEY`
+
+### Example Workflow for AWS Deployment
+
+```yaml
+name: Deploy to ECR
+
+on:
+  push:
+    branches: [main]
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+
+      - name: Configure AWS credentials
+        uses: aws-actions/configure-aws-credentials@v1
+        with:
+          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+          aws-region: us-east-1
+
+      - name: Login to Amazon ECR
+        id: login-ecr
+        uses: aws-actions/amazon-ecr-login@v1
+
+      - name: Build, tag, and push image to Amazon ECR
+        env:
+          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
+          ECR_REPOSITORY: my-ecr-repo
+          IMAGE_TAG: ${{ github.sha }}
+        run: |
+          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
+          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
+```
+
+This workflow builds a Docker image and pushes it to Amazon ECR.
+
+## Related Information
+
+For more detailed information, refer to the [GitHub Actions documentation](https://docs.github.com/en/actions).

infrastructure-documenation/deployment/roadmap.md

@@ -0,0 +1,96 @@
+# Eclipse PASS Cloud-Native Kubernetes Deployment Roadmap
+
+## Table of Contents
+1. [Summary](#summary)
+2. [Current State](#current-state)
+3. [Target Architecture](#target-architecture)
+4. [Roadmap Phases](#roadmap-phases)
+5. [Key Milestones](#key-milestones)
+6. [Challenges and Considerations](#challenges-and-considerations)
+
+## Summary
+
+This document outlines the roadmap for transitioning the Eclipse Public Access Submission System (PASS) from its current Docker Compose-based deployment to a cloud-native Kubernetes deployment. This transition aims to improve scalability, resilience, and manageability of the PASS platform.
+
+## Current State
+
+- Deployment primarily uses Docker Compose
+- Infrastructure hosted on AWS (EC2, ECS, RDS, S3, ALB, WAF)
+- Manual scaling and management of components
+
+## Target Architecture
+
+The target architecture will leverage Kubernetes for orchestration and cloud-native principles:
+
+- Kubernetes cluster (e.g., Amazon EKS, self-managed K8s on EC2)
+- Containerized microservices
+- Helm charts for deployment management
+
+### Future Possibilities
+
+- Prometheus and Grafana for monitoring
+- Istio for service mesh
+- CI/CD pipeline for Kubernetes deployment
+
+## Roadmap Phases
+
+### Phase 1: Preparation and Planning
+- Analyze current architecture and identify components for containerization
+- Define Kubernetes resource requirements
+- Plan data migration strategy
+- Identify necessary changes to application code for Kubernetes compatibility
+
+### Phase 2: Containerization and Kubernetes Basics
+- Containerize all PASS components
+- Create initial Kubernetes manifests (Deployments, Services, ConfigMaps)
+- Set up a test Kubernetes cluster
+- Implement basic health checks and readiness probes
+
+### Phase 3: Kubernetes Integration and Testing
+- Develop Helm charts for PASS components
+- Integrate persistent storage solutions (e.g., persistent volumes for databases)
+- Implement secrets management
+- Set up ingress controllers and configure routing
+- Conduct thorough testing of the Kubernetes deployment
+
+### Phase 4: Advanced Kubernetes Features and Optimization
+- Implement horizontal pod autoscaling
+- Set up cluster autoscaling
+- Optimize resource requests and limits
+- Implement pod disruption budgets for high availability
+- Configure liveness probes for self-healing
+
+### Phase 5: Monitoring, Logging, and Security
+- Set up Prometheus and Grafana for monitoring
+- Implement centralized logging (e.g., ELK stack)
+- Configure network policies for enhanced security
+- Implement pod security policies
+
+### Phase 6: CI/CD Pipeline Integration
+- Adapt CI/CD pipelines for Kubernetes deployment
+- Implement GitOps practices with tools like ArgoCD or Flux
+- Set up canary deployments and rollback mechanisms
+
+### Phase 7: Production Migration and Optimization
+- Perform staged migration to production Kubernetes environment
+- Optimize performance and resource utilization
+- Conduct load testing and fine-tuning
+- Document new deployment and management processes
+
+## Key Milestones
+
+1. All PASS components successfully containerized
+2. Basic Kubernetes deployment functional in test environment
+3. Helm charts created and tested for all components
+4. Monitoring and logging solutions implemented
+5. CI/CD pipeline fully integrated with Kubernetes deployment
+6. Production environment migrated to Kubernetes
+7. Performance optimization and fine-tuning completed
+
+## Challenges and Considerations
+
+- Ensuring data integrity during migration
+- Managing stateful applications in Kubernetes (e.g., databases)
+- Balancing resource allocation and cost optimization
+- Training team members on Kubernetes and cloud-native technologies
+- Ensuring security and compliance in the new architecture