docs: update README and add arch docs.

in-toto · Feb 4, 2022 · 48c2dcd · 48c2dcd
1 parent 8524008
commit 48c2dcd
Show file tree

Hide file tree

Showing 4 changed files with 271 additions and 22 deletions.
diff --git a/README.md b/README.md
@@ -1,49 +1,303 @@
 ![witness](/docs/assets/logo.png)
 
+[![asciicast](https://asciinema.org/a/2DZRRh8uzrzHcUVL8md86Zj4D.svg)](https://asciinema.org/a/2DZRRh8uzrzHcUVL8md86Zj4D)
+
 ## Witness is a pluggable framework for supply chain security
 
-Witness prevents tampering of build materials and verifies the integrity of the build process from source to target.  It works by wrapping commands executed in a continuous integration process.  Its attestation system is pluggable and offers support out of the box for most major CI and infrastructure providers.  Verification of Witness metadata and a secure PKI distribution system will mitigate against many supply chain attack vectors.
+Witness prevents tampering of build materials and verifies the integrity of the build process from source to target. It works by wrapping commands executed in a continuous integration process. Its attestation system is pluggable and offers support out of the box for most major CI and infrastructure providers. Verification of Witness metadata and a secure PKI distribution system will mitigate against many supply chain attack vectors and can be used as a framework for automated governance.
+
+Witness is an implementation of the in-toto spec including ITE-5, ITE-6, ITE-7 with an embedded rego policy engine.
 
+- Does **NOT** require elevated privileges.
+- Can run in a containerized or non-containerized environment
 - Records secure hashes of materials, artifacts, and events occurring during the CI process
 - Integrations with cloud identity services
 - Keyless signing with SPIFFE/SPIRE
 - Support for uploading attestation evidence to rekor server (sigstore)
 - Build policy enforcement with Open Policy Agent.
+- Alpha support for tracing and process tampering prevention
+- Verifies file integrity between CI steps, and across air gap.
+- Experimental Windows and ARM Support
+
+## Usage
+
+- [Run](docs/witness_run.md) - Runs the provided command and records attestations about the execution.
+- [Sign](docs/witness_sign.md) - Signs the provided file with the provided key.
+- [Verify](docs/witness_verify.md) - Verifies a witness policy.
+
+## TOC
+
+- [Witness Attestors](#witness-attestors)
+  - [What is a witness attestor?](#what-is-a-witness-attestor)
+  - [Attestor Security Model](#attestor-security-model)
+  - [Attestor Life Cycle](#attestor-life-cycle)
+    - [Attestation Lifecycle](#attestation-lifecycle)
+  - [Attestor Types](#attestor-types)
+    - [Pre Run Attestors](#pre-run-attestors)
+    - [Internal Attestors](#internal-attestors)
+    - [Post Run Attestors](#post-run-attestors)
+    - [AttestationCollection](#attestationcollection)
+    - [Attestor Subjects](#attestor-subjects)
+  - [Witness Policy](#witness-policy)
+    - [What is a witness policy?](#what-is-a-witness-policy)
+  - [Witness Verification](#witness-verification)
+    - [Verification Lifecycle](#verification-lifecycle)
+  - [Using SPIRE for Keyless Signing](#using-spire-for-keyless-signing)
+  - [Witness Examples](#witness-examples)
+  - [Media](#media)
+  - [Roadmap](#roadmap)
+  - [Support](#support)
 
 ## Getting Started
 
+### Download the Binary
+[Releases](https://github.com/testifysec/witness/releases)
 ```
 curl -LO https://github.com/testifysec/witness/releases/download/${VERSION}/witness_${VERSION}_${ARCH}.tar.gz
 tar -xzf witness_${VERSION}_${ARCH}.tar.gz
+```
+
+### Create a Keypair
+
+> Witness supports keyless signing!
 
+- Link: Using SPIRE with witness for keyless signing.
+
+```
 openssl genpkey -algorithm ed25519 -outform PEM -out testkey.pem
+openssl pkey -in testkey.pem -pubout > testpub.pem
+```
 
-./witness run -s build -k testkey.pem -o attestation.json -- \
-  go build .
+### Create a Witness configuration
 
-cat attestation.json | jq -r .payload | base64 -d | jq
+> - This file generally resides in your source code repository along with the public keys generated above.
+> - `.witness yaml` is the default location for the configuration file
+> - `witness help` will show all configuration options
+> - command-line arguments overrides configuration file values.
 
 ```
+## .witness.yaml
 
-## Usage
+run:
+    key: testkey.pem
+    trace: false
+verify:
+    attestations:
+        - "test-att.json"
+    policy: policy-signed.json
+    publickey: testpub.pem
+```
 
-- [Run](docs/witness_run.md) - Runs the provided command and records attestations about the execution.
-- [Sign](docs/witness_sign.md) - Signs the provided file with the provided key.
-- [Verify](docs/witness_verify.md) - Verifies a witness policy.
+### Record attestations for a build step
+
+> - The `-a {attestor}` flag allows you to define which attestors run
+> - ex. `-a maven -a was -a gitlab` would be used for a maven build running on a GitLab runner on GCP.
+> - Defining step names is important, these will be used in the policy.
+> - This should happen as a part of a CI step
+
+```
+witness run --step build -o test-att.json -- go build -o=testapp .
+```
 
-## Attestors
+### View the attestation data in the signed DSSE Envelope
+
+> - This data can be stored and retrieved from rekor!
+> - This is the data that is evaluated against the Rego policy
+> - Link example attestation data
+
+```
+cat test-att.json | jq -r .payload | base64 -d | jq
+```
+
+### Create a Policy File
+
+> - Make sure to replace the keys in this file with the ones from the step above.
+> - Rego policies should be base64 encoded
+> - Steps are bound to keys. Policy can be written to check the certificate data. For example, we can require a step is signed by a key with a specific `CN` attribute.
+> - Witness will require all attestations to succeed
+> - Witness will evaluate the rego policy against the JSON object in the corresponding attestor
+
+- Link: Witness Policy
+- Link: Embedding Rego in Witness Policies
+- Link: Witness Functionaries
+
+```
+## policy.json
+
+{
+  "expires": "2023-12-17T23:57:40-05:00",
+  "steps": {
+    "build": {
+      "name": "build",
+      "attestations": [
+        {
+          "type": "https://witness.testifysec.com/attestations/material/v0.1",
+          "regopolicies": []
+        },
+        {
+          "type": "https://witness.testifysec.com/attestations/command-run/v0.1",
+          "regopolicies": []
+        },
+        {
+          "type": "https://witness.testifysec.com/attestations/product/v0.1",
+          "regopolicies": []
+        }
+      ],
+      "functionaries": [
+        {
+          "publickeyid": "{{PUBLIC_KEY_ID}}"
+        }
+      ]
+    }
+  },
+  "publickeys": {
+    "{{PUBLIC_KEY_ID}}": {
+      "keyid": "{{PUBLIC_KEY_ID}}",
+      "key": "{{B64_PUBLIC_KEY}}"
+    }
+  }
+}
+```
+
+### Replace the variables in the policy
+
+```
+id=`sha256sum testpub.pem | awk '{print $1}'` && sed -i "s/{{PUBLIC_KEY_ID}}/$id/g" policy.json
+pubb64=`cat testpub.pem | base64 -w 0` && sed -i "s/{{B64_PUBLIC_KEY}}/$pubb64/g" policy.json
+```
+
+### Sign The Policy File
+
+Keep this key safe, its owner will control the policy gates.
+
+```
+witness sign -f policy.json --key testkey.pem --outfile policy-signed.json
+```
+
+### Verify the Binary Meets Policy Requirements
+
+> This process works across air-gap as long as you have the signed policy file, correct binary, and public key or certificate authority corresponding to the private key that signed the policy.
+> `witness verify` will return a `non-zero` exit and reason in the case of failure. Success will be silent with a `0` exit status
+> for policies that require multiple steps, multiple attestations are required.
+
+```
+witness verify -f testapp -a test-att.json -p policy-signed.json -k testpub.pem
+```
+
+# Witness Attestors
+
+## What is a witness attestor?
+
+Witness attestors are pieces of code that assert facts about a system and store those facts in a versioned schema. Each attestor has a `Name`, `Type`, and `RunType`. The `Type` is a versioned string corresponding to the JSON schema of the attestation. For example, the AWS attestor is defined as follows:
+
+```
+  Name    = "aws"
+  Type    = "https://witness.testifysec.com/attestation/aws/v0.1"
+  RunType = attestation.PreRunType
+```
+
+The attestation types are used when we write policy against these attestations.
+
+## Attestor Security Model
+
+Attestations are only as secure as the data that feeds them. Where possible cyrtographic material should be validated, evidence of validation should be included in the attestation for out-of-band validation.
+
+Examples of cryptographic validation is found in the GCP, AWS, and GitLab attestors.
+
+## Attestor Life Cycle
+
+- **PreRun:** `PreRun` attestors run before the `material` attestor and `commandRun` attestors. These attestors generally collect information about the environment.
+
+- **Material Attestor:** The `material` attestor is an internal attestor and runs immediately after
+
+- **CommandRun Attestor:** The CommandRun attestor is an internal attestor. It has experimental tracing support that can be enabled with the `--trace` flag
+
+- **Product Attestor:** The Product attestor collects the products produced by the `commandRun` attestor and calculates the secure hash, and makes the file descriptor available to the `postRun` attestors.
+
+### Attestation Lifecycle
+
+![](arch.drawio.png?page=1)
+
+## Attestor Types
+
+### Pre Run Attestors
 
 - [AWS](docs/attestors/aws-iid.md) - Attestor for AWS Instance Metadata
-- [CommandRun](docs/attestors/commandrun.md) - Attestor for running a command
-- [Environment](docs/attesors/environment.md) - Attestor for environment variables
 - [GCP](docs/attestors/gcp-iit.md) - Attestor for GCP Instance Idenity Service
-- [Git](docs/attestors/git.md) - Attestor for Git Repository
 - [GitLab](docs/attestors/gitlab.md) - Attestor for GitLab Pipelines
-- [GitHub](docs/attestor.md#github) - Attestor for GitHub Actions
-- [JWT](docs/attestors/jwt.md) - Attestor of JSON Web Tokens
-- [Material](docs/attestors/material.md) - Attesor of materials
-- [Maven](docs/attestors/maven.md) - Attestor of Maven pom.xml
-- [Product](docs/attestors/product.md) - Attestor of products
+- [Git](docs/attestors/git.md) - Attestor for Git Repository
+- [Maven](docs/attestors/maven.md) Attestor for Maven Projects
+- [Environment](docs/attestors/environment.md) - Attestor for environment variables (**_be careful with this - there is no way to mask values yet_**)
+- [JWT](docs/attestors/jwt.md) - Attestor for JWT Tokens
+
+### Internal Attestors
+
+- [CommandRun](docs/attestors/commandrun.md) - Records traces and metadata about the actual process being run
+- [Material](docs/attestors/material.md) - Records secure hashes of files in current working directory
+- [Product](docs/attestors/product.md) - Records secure hashes of files produced by commandrun attestor (only detects new files)
+
+### Post Run Attestors
+
+PostRun attestors collect have access to the files discovered by the product attestor. The purpose of PostRun attestors is to select metadata from the products. For example, in the OCI attestor the attestor examines the tar file and extracts OCI container meta-data.
+
+- [OCI](docs/attestors/oci.md) - Attestor for tar'd OCI images
+
+### AttestationCollection
+
+An `attestationCollection` is a collection of attestations that are cryptographically bound together. Because the attestations are bound together, we can trust that they all happened as part of the same attesation life cycle. Witness policy defines which attestations are required.
+
+### Attestor Subjects
+
+Attestors define subjects that act as lookup indexes. The attestationCollection can be looked up by any of the subjects defined by the attestors.
+
+## Witness Policy
+
+### What is a witness policy?
+
+A witness policy is a signed document that encodes the requirements for an artifact to be validated. A witness policy includes public keys for trusted functionaries, which attestations must be found, and rego policy to evaluate against the attestation meta-data.
+
+I witness policy allowers administrators trace the compliance status of an artifact at any point during it's lifecycle.
+
+![](https://hackmd.io/_uploads/HkpIDY5CY.jpg)
+
+## Witness Verification
+
+### Verification Lifecycle
+
+![](arch.drawio.png?page=2)
+
+## Using SPIRE for Keyless Signing
+
+Witness can consume ephemeral keys from a SPIRE node agent. Configure witness with the flag `--spiffe-socket` to enable keyless signing.
+
+During the verification, process witness will use the Rekor integrated time to make a determination on certificate validity. The SPIRE certificate only needs to remain valid long enough for the attestation to be integrated into the Rekor log.
+
+## Witness Examples
+
+- [Using Witness To Prevent SolarWinds Type Attacks](examples/solarwinds/README.md)
+- [Using Witness To Find Artifacts With Hidden Vulnerable Log4j Dependencies](examples/log4shell/README.md)
+
+## Media
+
+- [Blog - What is a supply chain attestation, and why do I need it?](https://www.testifysec.com/blog/what-is-a-supply-chain-attestation/)
+- [Talk - Securing the Software Supply Chain with the in-toto & SPIRE projects](https://www.youtube.com/watch?v=4lFbdkB62QI)
+- [Talk - Securing the Software Supply Chain with SBOM and Attestation](https://www.youtube.com/watch?v=wX6aTZfpJv0)
+
+## Roadmap
+
+- Attestors for all major platforms
+- CaC Card Attestor
+- GovCloud Attestor
+- OIDC Attestor
+- FIDO Attestor
+- Vault Key Provider
+- Cloud KMS Support
+- Kubernetes Admission Controller
+- EDR Agent
+- Cosign Signature Validation
+- Notary v2 Signature Validation
+- Zarf Integration
+- IronBank Attestor
 
 ## Support
 

diff --git a/arch.drawio.png b/arch.drawio.png
diff --git a/go.mod b/go.mod
@@ -82,7 +82,6 @@ require (
 	github.com/spf13/afero v1.6.0 // indirect
 	github.com/spf13/cast v1.4.1 // indirect
 	github.com/spf13/jwalterweatherman v1.1.0 // indirect
-	github.com/spiffe/spire v1.1.3
 	github.com/subosito/gotenv v1.2.0 // indirect
 	github.com/theupdateframework/go-tuf v0.0.0-20211203210025-7ded50136bf9 // indirect
 	github.com/xanzy/ssh-agent v0.3.0 // indirect

diff --git a/go.sum b/go.sum
@@ -962,7 +962,6 @@ github.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA
 github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
 github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
 github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
-github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
 github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
 github.com/hashicorp/hcl v1.0.1-0.20190430135223-99e2f22d1c94 h1:LaH4JWe6Q7ICdxL5raxQjSRw7Pj8uTtAENrjejIYZIg=
 github.com/hashicorp/hcl v1.0.1-0.20190430135223-99e2f22d1c94/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
@@ -1347,7 +1346,6 @@ github.com/pseudomuto/protoc-gen-doc v1.5.0/go.mod h1:exDTOVwqpp30eV/EDPFLZy3Pwr
 github.com/pseudomuto/protokit v0.2.0/go.mod h1:2PdH30hxVHsup8KpBTOXTBeMVhJZVio3Q8ViKSAXT0Q=
 github.com/qur/ar v0.0.0-20130629153254-282534b91770/go.mod h1:SjlYv2m9lpV0UW6K7lDqVJwEIIvSjaHbGk7nIfY8Hxw=
 github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
-github.com/rcrowley/go-metrics v0.0.0-20200313005456-10cdbea86bc0 h1:MkV+77GLUNo5oJ0jf870itWm3D0Sjh7+Za9gazKc5LQ=
 github.com/rcrowley/go-metrics v0.0.0-20200313005456-10cdbea86bc0/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
 github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 h1:N/ElC8H3+5XpJzTSTfLsJV/mx9Q9g7kxmchpfZyxgzM=
 github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
@@ -1453,8 +1451,6 @@ github.com/spf13/viper v1.10.1 h1:nuJZuYpG7gTj/XqiUwg8bA0cp1+M2mC3J4g5luUYBKk=
 github.com/spf13/viper v1.10.1/go.mod h1:IGlFPqhNAPKRxohIzWpI5QEy4kuI7tcl5WvR+8qy1rU=
 github.com/spiffe/go-spiffe/v2 v2.0.0-beta.10 h1:UXfGMp27MlQcYCAVRl21+cZrbKXMLsFmMXam5W3qBIA=
 github.com/spiffe/go-spiffe/v2 v2.0.0-beta.10/go.mod h1:TEfgrEcyFhuSuvqohJt6IxENUNeHfndWCCV1EX7UaVk=
-github.com/spiffe/spire v1.1.3 h1:JoaRctkcJ+778YziOp/rmBCCTwkBM1dLatwlu5hqOUA=
-github.com/spiffe/spire v1.1.3/go.mod h1:eocLm+hrh7uL4LBwJ97vVwX9egWOkmzG8z80nvAe/jg=
 github.com/src-d/gcfg v1.4.0/go.mod h1:p/UMsR43ujA89BJY9duynAwIpvqEujIH/jFlfL7jWoI=
 github.com/stefanberger/go-pkcs11uri v0.0.0-20201008174630-78d3cae3a980/go.mod h1:AO3tvPzVZ/ayst6UlUKUv6rcPQInYe3IknH3jYhAKu8=
 github.com/streadway/amqp v0.0.0-20190404075320-75d898a42a94/go.mod h1:AZpEONHx3DKn8O/DFsRAY58/XVQiIPMTMB1SddzLXVw=