doc: major doc refactoring

docs/book.toml

@@ -11,3 +11,5 @@ create-missing = false
 [output.html]
 git-repository-url = "https://github.com/PierreBeucher/novops"
 edit-url-template = "https://github.com/PierreBeucher/novops/edit/main/docs/{path}"
+
+# [output.linkcheck]

docs/src/SUMMARY.md

@@ -3,32 +3,32 @@
 - [Introduction](intro.md)
 - [Installation](install.md)
 - [Getting started](getting-started.md)
+- [Security Model](security.md)
 
 - [Configuration and Modules](config/README.md)
   - [Configuration](config/config.md)
     - [Configuration schema](config/schema.md)
-  - [Files and Variables](config/files-variables.md)
+  - [Variables & Files](config/files-variables.md)
   - [Hashicorp Vault](config/hashicorp-vault.md)
   - [AWS](config/aws.md)
   - [Google Cloud](config/google-cloud.md)
   - [Microsoft Azure](config/microsoft-azure.md)
   - [BitWarden](config/bitwarden.md)
-- [Integrate Novops in your workflow](integrate/README.md)
-  - [Plain shell (sh, bash, zsh...)](integrate/plain-shell.md)
-  - [Nix](integrate/nix.md)
-  - [Docker](integrate/docker.md)
-  - [GitLab CI](integrate/gitlab-ci.md) 
-  - [GitHub Action](integrate/github-action.md)
-  - [Jenkins](integrate/jenkins.md)
 - [Examples and Use Cases](examples/README.md)
-  - [Ansible setup](examples/ansible.md)
-  - [Terraform setup](examples/terraform.md)
-  - [Pulumi setup](examples/pulumi.md)
-  - [Authenticate with Hashicorp Vault before loading Novops](examples/hvault-auth.md)
-  - [Impersonate IAM Role before loading Novops](examples/aws-role.md)
+  - [Shell - sh, bash, zsh...](examples/shell.md)
+  - [Docker](examples/docker.md)
+  - [Nix](examples/nix.md)
+  - [CI/CD Platforms](examples/cicd/README.md)
+    - [GitLab CI](examples/cicd/gitlab-ci.md) 
+    - [GitHub Action](examples/cicd/github-action.md)
+    - [Jenkins](examples/cicd/jenkins.md)
+  - [Infra as Code](examples/iac/README.md)
+    - [Ansible](examples/iac/ansible.md)
+    - [Terraform](examples/iac/terraform.md)
+    - [Pulumi](examples/iac/pulumi.md)
 - [Advanced](advanced/README.md)
-  - [Security - how are secrets safely loaded?](advanced/security.md)
   - [Debugging and log verbosity](advanced/debugging.md)
   - [Internal architecture: Inputs, Outputs and resolving](advanced/architecture.md)
-[Contributing](contributing/README.md)
-- [Development guide](contributing/development.md)
+- [Contributing](contributing/README.md)
+  - [Development guide](contributing/development.md)
+  - [Implementing a module](contributing/add-module.md)

docs/src/advanced/README.md

@@ -1,5 +1,4 @@
 # Advanced
 
-- [Security - how are secrets safely loaded?](security.md)
 - [Debugging and log verbosity](debugging.md)
 - [Internal architecture: Inputs, Outputs and resolving](architecture.md)

docs/src/advanced/architecture.md

@@ -82,7 +82,7 @@ impl ResolveTo<String> for HashiVaultKeyValueV2 {
 }
 ```
 
-See [`src/core.rs`](src/core.rs) for details.
+See [`src/core.rs`](https://github.com/PierreBeucher/novops/blob/main/src/core.rs) for details.
 
 ## Novops config schema and internal structure
 

docs/src/config/README.md

@@ -1,9 +1,18 @@
-# Load secrets and configs
+# Configuration and modules
+
+Everything you can use within `.novops.yml`
 
 - [`.novops.yml` configuration schema](config.md)
 - [Files and Variables](files-variables.md)
 - [Hashicorp Vault](hashicorp-vault.md)
+  - Key Value v1/v2
+  - AWS temporary credentials
 - [AWS](aws.md)
+  - Secrets Manager
+  - STS Assume Role for temporary IAM credentials
+  - SSM Parameter Store
 - [Google Cloud](google-cloud.md)
-- [Microsoft Azure](microsoft-azure.md)
+ - Secret Manager
+- [Azure](microsoft-azure.md)
+  - Key Vault
 - [BitWarden](bitwarden.md)

docs/src/config/aws.md

@@ -1,36 +1,33 @@
 # AWS
 
-- [AWS](#aws)
-  - [Authentication & Configuration](#authentication--configuration)
-  - [STS Assume Role](#sts-assume-role)
-  - [Systems Manager (SSM) Parameter Store](#systems-manager-ssm-parameter-store)
-  - [Secrets Manager](#secrets-manager)
+- [Authentication & Configuration](#authentication--configuration)
+- [STS Assume Role](#sts-assume-role)
+- [Systems Manager (SSM) Parameter Store](#systems-manager-ssm-parameter-store)
+- [Secrets Manager](#secrets-manager)
 
 ## Authentication & Configuration
 
-See [AWS Examples](../examples/aws-role.md) for authentication methods you can use CI or for local development environments.
+Authenticating with `aws` CLI is enough, Novops will use locally available credentials. Specify your AWS credentials as usual (see [AWS Programmatic access](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys) or [Credentials quickstart](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html#cli-configure-quickstart-creds)):
 
-Specify your AWS credentials as usual (see [AWS Programmatic access](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys) or [Credentials quickstart](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html#cli-configure-quickstart-creds)):
+Credentials are loaded in order of priority:
 
 - Environment variables `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, etc.
 - Config file `.aws/config` and `.aws/credentials`
 - Use IAM Role attached from ECS or EC2 instance
 
-You can also set global AWS configuration to override certains configs (such as AWS endpoint), for example:
+You can also use `config` root element override certains configs (such as AWS endpoint), for example:
 
 ```yaml
-environments:
-  # ...
-
-aws:
-  endpoint: "http://localhost:4566/" # Use LocalStack endpoint
+config:
+  aws:
+    endpoint: "http://localhost:4566/" # Use LocalStack endpoint
 ```
 
 ## STS Assume Role
 
-Generate temporary [IAM Role credentials with AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html):
+Generate temporary [IAM Role credentials with STS AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html):
 
-Note that `aws` is an `environment` key rather than a `files` or `variables`. That's because it will output multiple variables.
+Note that `aws` is an `environment` sub-key, not a `files` or `variables` sub-key as it will output multiple variables `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_SESSION_TOKEN`
 
 ```yaml
 environments:

docs/src/config/bitwarden.md

@@ -1,21 +1,24 @@
 # BitWarden
 
-Using BitWarden requires BitWarden CLI `bw` to be installed locally and `BW_SESSION_TOKEN` variable set. 
+## Authentication & Configuration
+
+To use BitWarden module:
+
+- Ensure BitWarden CLI `bw` is available in the same context `novops` runs in
+- Set environment variable `BW_SESSION_TOKEN`
 
 ```yaml
 environments:
   dev:
     files: 
-    - name: ssh-key
-      content:
-        bitwarden:
-          # Name of the entry to load
-          entry: Some SSH Key entry
-          # Field to read from BitWarden objects. Maps directly to JSON field from 'bw get item' command
-          # See below for details
-          field: notes
-
-
+      - name: ssh-key
+        content:
+          bitwarden:
+            # Name of the entry to load
+            entry: Some SSH Key entry
+            # Field to read from BitWarden objects. Maps directly to JSON field from 'bw get item' command
+            # See below for details
+            field: notes
 ```
 
 Novops will load items using `bw get item` as JSON. `field` must be set to expected field. Separate sub-field with `.`. Examples:

docs/src/config/config.md

@@ -1,21 +1,17 @@
 # `.novops.yml` configuration schema
 
-`novops` will load configuration from current directory `.novops.yml` by default. 
-
-See modules documentation and [full configuration schema](./schema.html) for detailed schema.
+`novops` uses `.novops.yml` to load secrets. This doc details how this file can be used for various use cases. You can use another config with `novops [load|run] -c PATH`, though this doc will refer to `.novops.yml` for config file.
 
 ## Configuration: Environments, Modules, Inputs and Outputs
 
-Novops config is the source of truth for variables and files to be loaded. Novops config defines:
+`.novops.yml` defines:
 
-- **Environments** which can be loaded with `novops load`
+- **Environments** for which secrets can be loaded
 - Environments define **Inputs** (`files`, `variables`, `aws`...)
-- When loaded, Inputs are **resolved** to provide **Outputs** resulting into files and environment variables
+- Inputs are **resolved** into **Environment Variables** and **Files** (and other Outputs constructs internally with files and variables)
 - Inputs can also use other Inputs, such as an Hashicorp Vault `hvault_kv2` Inputs used by a `variable` Input to resolve a secret into an environment variable (see below for example)
 
-Example below defines two environments (`dev` & `prod`), each using multiple inputs: `files`, `variables` and `hvault_kv2`. 
-
-`files` and `variables` modules are the most common Novops modules. Their values/content can be defined using plain strings or other Inputs.
+Example: environments `dev` and `prod` with inputs `files`, `variables` and `hvault_kv2`. 
 
 ```yaml
 name: some-app
@@ -25,31 +21,29 @@ environments:
   # Environment name
   dev:    
 
-   # "variables" Input (mind the plural) defines one or more variables for our environment
-   # it's a list of "variable" Input 
-   # Loading these inputs will result in envionment variables Outputs
+   # "variables" is a list of "variable" inputs for environment 
+   # Loading these inputs will result in envionment variables outputs
    variables:
 
-      # "variable" Input has two arguments: 
-      # - name: the name of the result environment variable
-      # - value: define the variable value, can be a plain string or another Input
+      # - name: environment variable name
+      # - value: variable value, can be a plain string or another input
       - name: MY_APP_HOST
         value: "localhost:8080"
 
-      # variable value can be another Input which resolves to a string
-      # novops will read the referenced value, in this case from Hashicorp Vault server
+      # here variable value is another Input resolving to a string
+      # novops will read the referenced value
+      # in this case from Hashicorp Vault server
       - name: MY_APP_PASSWORD
         value:
           hvault_kv2:
             path: crafteo/app/dev
             key: password
 
-    # "files" another input taking a lise of "file" inputs as arguments
+    # "files" is a list of "file" inputs
     files:
 
-      # "file" Input arguments are:
-      # - content: Input resolving to a string. Can be a plain string or another input resolving to a string
-      # - variable: Optionally, a variable name which will point to generated file
+      # - content: input resolving to a string. Can be a plain string or another input resolving to a string
+      # - variable: a variable name which will point to generated file
       # - dest: Optionally, the final destination where file will be generate. By default Novops create a file in a secure directory.
       #
       # This file input will resolve to two Outputs:
@@ -69,40 +63,9 @@ environments:
             key: token
 ```
 
-## Generic configuration schema
-
-More generally, a novops config looks like:
-
-```yaml
-# Required, name of the application
-# should be unique, loading multiple Novops config with the same name may cause conflicts
-name: crafteo-app
-
-# Each environments represent a specific environments
-# with its own variables and files output
-# derived from modules config
-environments:
-
-  environment_A:
-
-    # Each environments defines inputs and their config
-    input_1: 
-      input_config:
-        # Other inputs can be used within inputs
-        input_4:
-
-    input_2: 
-      some_config: # ...
-
-  environment_B:
-    input_3:
-
-# Optional, additional configurations
-config:
-  # Additional config...
-```
+## Root `config` keyword
 
-`config` is used to specifhy internal Novops config and some module configs:
+Root `config` is used to specifhy global configurations for Novops and its modules:
 
 ```yaml
 config:

docs/src/config/files-variables.md

@@ -2,7 +2,7 @@
 
 `files` and `variables` are primay way to configure Novops
 - Each element in `variables` will generate a single environment variable loaded from `value`
-- Each element in `files` will generate a [secured temporary file](../advanced/security.md) loaded from `content`
+- Each element in `files` will generate a [secured temporary file](../security.md) loaded from `content`
 
 ```yaml
 environments:

docs/src/config/google-cloud.md

@@ -6,7 +6,7 @@
 
 ## Authentication
 
-Provide credentials using [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials):
+Authenticating with `gcloud` CLI is enough. Otherwise, provide credentials using [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials):
 
 - Set `GOOGLE_APPLICATION_CREDENTIALS` to a credential JSON file
 - Setup creds using `gcloud` CLI
@@ -20,15 +20,15 @@ Retrieve secrets from [GCloud Secret Manager](https://cloud.google.com/secret-ma
 environments:
   dev:
     variables:
-    - name: SECRETMANAGER_VAR_STRING
-      value:
-        gcloud_secret:
-          name: projects/my-project/secrets/SomeSecret/versions/latest
-          # validate_crc32c: true
+      - name: SECRETMANAGER_VAR_STRING
+        value:
+          gcloud_secret:
+            name: projects/my-project/secrets/SomeSecret/versions/latest
+            # validate_crc32c: true
 
     files:
-    - name: SECRETMANAGER_VAR_FILE
-      content:
-        gcloud_secret:
-          name: projects/my-project/secrets/SomeSecret/versions/latest
+      - name: SECRETMANAGER_VAR_FILE
+        content:
+          gcloud_secret:
+            name: projects/my-project/secrets/SomeSecret/versions/latest
 ```