Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ref/docs #450

Merged
merged 25 commits into from
Aug 14, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/docs/architecture/marbles.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Marbles

Marbles are MarbleRun's data planes and run your application code in secure enclaves within otherwise regular Docker containers.
Marbles are MarbleRun's data planes and run your application code in secure enclaves.
Marbles communicate with the Coordinator via gRPC over TLS.
MarbleRun [supports multiple enclave runtimes](../features/runtimes.md) as the data plane.
See the [Add a Service](../workflows/add-service.md) section on how to build and configure an enclave as Marble.
6 changes: 4 additions & 2 deletions docs/docs/architecture/security.md
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The [Coordinator](coordinator.md) and the [Marbles](marbles.md) run inside SGX E
MarbleRun uses cryptography for the following tasks.

* Authentication and authorization
* Establishing a public key infrastructure (PKI) for MarbleRun
* Establishing a public critical infrastructure (PKI) for MarbleRun
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved
* Encrypting network traffic via mutual TLS between enclaves
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved
* Encrypting persistent state
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved

Expand Down Expand Up @@ -69,7 +69,9 @@ Depending on the Marble's runtime, the certificate can be used [manually](../wor
## Attested TLS (aTLS)

In a CC environment, attested TLS (aTLS) can establish secure connections between two parties using the remote attestation features of the CC components.
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved
aTLS modifies the TLS handshake by embedding an attestation statement into the TLS certificate. Instead of relying on a certificate authority, aTLS uses this attestation statement to establish trust in the certificate.
With aTLS, the party to be authenticated somehow binds its TLS certificate to an attestation statement.
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved
For example, it embeds the certificate's public key into an attestation statement.
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved
Instead of relying on a certificate authority, aTLS uses this attestation statement to establish trust in the certificate.
The protocol can be used by clients to verify a server certificate, by a server to verify a client certificate, or for mutual verification (mutual aTLS).


Expand Down
14 changes: 7 additions & 7 deletions docs/docs/building-enclaves/occlum.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@
Running an Occlum app with MarbleRun requires some changes to its manifest.

## Requirements
Set up an environment to create Occlum images. For an easy start, we recommend that you use either the official [Occlum Docker image](https://hub.docker.com/r/occlum/occlum) or use [our provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). For a working DCAP remote attestation environment, we recommend [our platform deployment guide](../deployment/platforms/platforms.md).
Set up an environment to create Occlum images. For an easy start, we recommend using either the official [Occlum Docker image](https://hub.docker.com/r/occlum/occlum) or [our provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). For a working DCAP remote attestation environment, we recommend [our platform deployment guide](../deployment/platforms/platforms.md).

Check warning on line 5 in docs/docs/building-enclaves/occlum.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/docs/building-enclaves/occlum.md#L5

[Microsoft.We] Try to avoid using first-person plural like 'we'.
Raw output
{"message": "[Microsoft.We] Try to avoid using first-person plural like 'we'.", "location": {"path": "docs/docs/building-enclaves/occlum.md", "range": {"start": {"line": 5, "column": 67}}}, "severity": "WARNING"}

Check warning on line 5 in docs/docs/building-enclaves/occlum.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/docs/building-enclaves/occlum.md#L5

[Microsoft.We] Try to avoid using first-person plural like 'we'.
Raw output
{"message": "[Microsoft.We] Try to avoid using first-person plural like 'we'.", "location": {"path": "docs/docs/building-enclaves/occlum.md", "range": {"start": {"line": 5, "column": 335}}}, "severity": "WARNING"}

To build your service, you can start with [Occlum's Introduction](https://github.com/occlum/occlum#introduction) to get your application up and running, and then come back here to adapt it for use with MarbleRun.
To build your service, you can start with [Occlum's Introduction](https://github.com/occlum/occlum#introduction) to get your application up and running and then come back here to adapt it for use with MarbleRun.

## Configuration
### Premain executable
Add our pre-built [premain-libos](https://github.com/edgelesssys/marblerun/releases/latest/download/premain-libos) executable to your Occlum image, e.g., by copying it to `image/bin/premain-libos`. By default, Occlum restricts executable files to the `/bin` directory. If you placed the `premain-libos` binary to a different path, you need to adjust this setting accordingly.
Add our pre-built [premain-libos](https://github.com/edgelesssys/marblerun/releases/latest/download/premain-libos) executable to your Occlum image, e.g., by copying it to `image/bin/premain-libos`. By default, Occlum restricts executable files to the `/bin` directory. If you placed the `premain-libos` binary at a different path, you must adjust this setting accordingly.

Check warning on line 11 in docs/docs/building-enclaves/occlum.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/docs/building-enclaves/occlum.md#L11

[Microsoft.We] Try to avoid using first-person plural like 'our'.
Raw output
{"message": "[Microsoft.We] Try to avoid using first-person plural like 'our'.", "location": {"path": "docs/docs/building-enclaves/occlum.md", "range": {"start": {"line": 11, "column": 5}}}, "severity": "WARNING"}

Finally, define the original entry point for your Occlum instance as the first `Argv` parameter for your Marble in MarbleRun's `manifest.json`. See [Defining a manifest](../workflows/define-manifest.md) for more information on how to define the `Argv` parameters. This lets MarbleRun launch your application after it succeeded in authenticating with the Coordinator and provides entrypoint pinning similar to the one offered in `Occlum.json`.

### Environment variables
The Marble needs to retrieve the MarbleRun specific configuration parameters via environment variables, as [described under Step 3 in "Adding a service."](../workflows/add-service.md)
The Marble needs to retrieve the MarbleRun-specific configuration parameters via environment variables, as [described under Step 3 in "Adding a service."](../workflows/add-service.md)

To pass environment variables to the enclave, Occlum requires them to be specified in the `env` section in `Occlum.json`.

Expand Down Expand Up @@ -41,9 +41,9 @@
This will allow you to embed the expected default values during build time. It also lets the user/host system change them during run time when a non-default Coordinator configuration is used.

### Resource limits
The premain process is written in Go. The enclave needs to have enough resources for the Go runtime, plus additional memory to launch your application.
The premain process is written in Go. The enclave must have enough resources for the Go runtime and additional memory to launch your application.

We recommend starting with the following values which should work fine for light-weight to medium memory demanding applications:
We recommend starting with the following values, which should work fine for light-weight to medium memory-demanding applications:

Check warning on line 46 in docs/docs/building-enclaves/occlum.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/docs/building-enclaves/occlum.md#L46

[Microsoft.We] Try to avoid using first-person plural like 'We'.
Raw output
{"message": "[Microsoft.We] Try to avoid using first-person plural like 'We'.", "location": {"path": "docs/docs/building-enclaves/occlum.md", "range": {"start": {"line": 46, "column": 1}}}, "severity": "WARNING"}
```javascript
"user_space_size": "2048MB",
"default_mmap_size": "900MB"
Expand All @@ -68,7 +68,7 @@
right: `SGX_QL_NETWORK_ERROR`: fail to launch QE', src/util/sgx/dcap/quote_generator.rs:22:13
```

You might need to check the DCAP configuration on your system. Note that when using the Docker image, the local Intel DCAP configuration needs to be correctly set from **inside the container.**
You might need to check the DCAP configuration on your system. Note that when using the Docker image, the local Intel DCAP configuration must be correctly set **inside the container.****
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved

If you use an Azure Confidential Computing machine, you can use our [provided Dockerfile](https://github.com/edgelesssys/marblerun/blob/master/samples/occlum-hello/Dockerfile). It patches the official Occlum image to use the Azure DCAP client, which handles the configuration automatically.

Expand Down
13 changes: 6 additions & 7 deletions docs/docs/deployment/platforms/alibaba.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,15 @@ Alibaba Cloud Container Service for Kubernetes (ACK) offers a popular deployment
[ACK hosts Kubernetes pods in SGX-capable Alibaba VMs](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/tee-based-confidential-computing) and exposes the underlying SGX hardware.

### Prerequisites
* Follow the instructions on the [ACK Confidential Computing Quick Start guide](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/create-an-ack-managed-cluster-for-confidential-computing) to provision an ACK cluster with Intel SGX enabled worker nodes.
* Follow the instructions on the [ACK Confidential Computing Quick Start guide](https://www.alibabacloud.com/help/en/ack/ack-managed-and-ack-dedicated/user-guide/create-an-ack-managed-cluster-for-confidential-computing) to provision an ACK cluster with Intel SGX-enabled worker nodes.

### Deploy MarbleRun

See our [Kubernetes guide](../kubernetes.md) on installing MarbleRun in your ACK cluster.
See the [Kubernetes guide](../kubernetes.md) on installing MarbleRun in your ACK cluster.


## Alibaba Cloud Elastic Compute Service

With 7th generation [security-enhanced ECS instances](https://www.alibabacloud.com/help/en/ecs/user-guide/overview-25), users can use Intel SGX on Alibaba Cloud.
With 7th-generation [security-enhanced ECS](https://www.alibabacloud.com/help/en/ecs/user-guide/overview-25) instances](https://www.alibabacloud.com/help/en/ecs/user-guide/overview-25), users can use Intel SGX on Alibaba Cloud.
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved
You can follow the guide for creating a [g7t, c7t, or r7t](https://www.alibabacloud.com/help/en/elastic-compute-service/latest/create-security-enhanced-instances) instance.

The description below uses a VM running Ubuntu 18.04.
Expand All @@ -38,7 +37,7 @@ The description below uses a VM running Ubuntu 18.04.

1. Set configuration for Alibaba Cloud SGX remote attestation service

Alibaba Cloud provides a PCCS for remote attestation, deployed per-region. For optimal stability, accessing the service in your instance's region is recommended.
Alibaba Cloud provides a PCCS for remote attestation deployed per region. For optimal stability, accessing the service in your instance's region is recommended.
The configuration is set in `/etc/sgx_default_qcnl.conf`.

* If your instance is assigned a public IP address, change the configuration to the following, where `[Region-ID]` is the ID of your instance's region:
Expand All @@ -63,5 +62,5 @@ The description below uses a VM running Ubuntu 18.04.

### Deploy MarbleRun

You can run MarbleRun standalone on your Alibaba Cloud ECS VM, see our [standalone guide](../standalone.md).
Alternatively, you can install a Kubernetes cluster. Probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/). See our [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in minikube.
You can run MarbleRun standalone on your Alibaba Cloud ECS VM. See the [standalone](../standalone.md) guide](../standalone.md).
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved
Alternatively, you can install a Kubernetes cluster. Probably the simplest option would be [minikube](https://minikube.sigs.k8s.io/docs/start/). See the [Kubernetes guide](../kubernetes.md) on how to install MarbleRun in minikube.
15 changes: 7 additions & 8 deletions docs/docs/deployment/platforms/on-prem.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,16 @@ This guide walks you through setting up MarbleRun for your on-premises deploymen
To deploy MarbleRun with Intel SGX, the machine or VM has to support Intel SGX.
Particularly, MarbleRun requires support for the SGX Data Center Attestation Primitives (DCAP).
You can verify [if your CPU supports DCAP](https://www.intel.com/content/www/us/en/support/articles/000057420/software/intel-security-products.html).

For more information read this article on [detecting Intel Software Guard Extensions](https://software.intel.com/content/www/us/en/develop/articles/properly-detecting-intel-software-guard-extensions-in-your-applications.html) in your applications.
For more information, read this article on [detecting Intel Software Guard Extensions](https://software.intel.com/content/www/us/en/develop/articles/properly-detecting-intel-software-guard-extensions-in-your-applications.html) in your applications.

#### BIOS

BIOS support is required for Intel SGX to provide the capability to enable and configure the Intel SGX feature in the system.
Currently, most of the SGX capable systems have SGX disabled by default in the BIOS. This default setting might change but for now, you need to manually enable it if it's not already enabled.
Currently, most of the SGX-capable systems have SGX disabled by default in the BIOS. This default setting might change, but for now, you need to manually enable it if it's not already enabled.

#### Updates

As with any modern technology, Intel SGX has been affected by security vulnerabilities. Intel addresses these vulnerabilities by updating the microcode of CPUs, changing the hardware of new CPUs, and updating the system software. Each microcode update that patches an SGX vulnerability requires a BIOS update. During remote attestation, it's checked that the microcode of the CPU which is deployed by the BIOS is up to date. The microcode and platform enclaves are commonly called the platform `Trusted Computing Base (TCB)`.
As with any modern technology, Intel SGX has been affected by security vulnerabilities. Intel addresses these vulnerabilities by updating the microcode of CPUs, changing the hardware of new CPUs, and updating the system software. Each microcode update that patches an SGX vulnerability requires a BIOS update. During remote attestation, it's checked that the microcode of the CPU, which is deployed by the BIOS, is up to date. The microcode and platform enclaves are commonly called the platform `Trusted Computing Base (TCB)`.

If your BIOS/firmware is outdated, you will see errors as `Platform TCB (2) is not up-to-date (oe_result_t=OE_TCB_LEVEL_INVALID)` during remote attestation procedures.

Expand All @@ -39,19 +38,19 @@ Most of the popular hypervisors support SGX:
#### Driver

You need to install the [DCAP SGX Driver](https://download.01.org/intel-sgx/sgx-dcap/1.11/linux/docs/Intel_SGX_SW_Installation_Guide_for_Linux.pdf).
Azure provides the instructions on [how to install this driver](https://docs.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#2-install-the-intel-sgx-dcap-driver) that you can use for your on-premises machines.
Azure provides instructions on [how](https://docs.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#2-install-the-intel-sgx-dcap-driver) to install this driver](https://docs.microsoft.com/en-us/azure/confidential-computing/quick-create-portal#2-install-the-intel-sgx-dcap-driver) that you can use for your on-premises machines.
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved

### SGX Data Center Attestation Primitives (DCAP)

DCAP is the new attestation mechanism for SGX [replacing EPID](https://software.intel.com/content/www/us/en/develop/blogs/an-update-on-3rd-party-attestation.html).
DCAP is the new attestation mechanism for SGX, [replacing](https://software.intel.com/content/www/us/en/develop/blogs/an-update-on-3rd-party-attestation.html)[ EPID](https://software.intel.com/content/www/us/en/develop/blogs/an-update-on-3rd-party-attestation.html).
m1ghtym0 marked this conversation as resolved.
Show resolved Hide resolved
You can find an overview of DCAP in the [official Intel docs](https://download.01.org/intel-sgx/sgx-dcap/1.11/linux/docs/DCAP_ECDSA_Orientation.pdf).
MarbleRun only supports DCAP and requires DCAP libraries installed and configured on your system.

From the perspective of MarbleRun and your workloads DCAP is accessed with a [Quote Generation Library (QGL)](https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteGeneration/README.md) and a [Quote Verification Library (QVL)](https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteVerification/README.md) for generating and verifying quotes respectively.
From the perspective of MarbleRun and your workloads, DCAP is accessed with a [Quote Generation Library (QGL)](https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteGeneration/README.md) and a [Quote Verification Library (QVL)](https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteVerification/README.md) for generating and verifying quotes respectively.
The QGL and QVL libraries need to be configured to talk to a [Provisioning Certificate Caching Service (PCCS)](https://download.01.org/intel-sgx/sgx-dcap/1.11/linux/docs/DCAP_ECDSA_Orientation.pdf).
You currently have two options regarding PCCS for your on-premises machines and clusters:

1. Use a public PCCS service by configuring your QGL and QVL to point to the public endpoints. Currently, Azure and Alibaba Cloud provide such a service, but require using infrastructure by these providers to make full use of the service.
1. Use a public PCCS service by configuring your QGL and QVL to point to the public endpoints. Currently, Azure and Alibaba Cloud provide such a service but require using infrastructure by these providers to make full use of the service.

1. Run your own PCCS and expose it to your machine or cluster. See [Intel's demo reference implementation](https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/master/QuoteGeneration/pccs/README.md) and [design guide](https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/SGX_DCAP_Caching_Service_Design_Guide.pdf) for more information.

Expand Down
10 changes: 5 additions & 5 deletions docs/docs/features/attestation.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ However, MarbleRun works with any DCAP service complying with the SGX specificat

Initially, the MarbleRun Coordinator is deployed to the cluster.
The Coordinator generates an X.509 certificate chain with a root and intermediate certificate authority (CA).
It generates a remote attestation quote which contains the hash of the root certificate (as `report_data`) as well as a hardware measurement of its enclave (`MRENCLAVE` of the report).
It generates a remote attestation quote that contains the hash of the root certificate (as `report_data`) as well as a hardware measurement of its enclave (`MRENCLAVE` of the report).

The admin receives the quote and verifies the Coordinator's integrity via the measurement. Next, they verify that the quote contains the root certificate’s hash. This ensures that the certificate was indeed generated by this Coordinator and wasn't manipulated during transport.

Expand All @@ -22,7 +22,7 @@ The [manifest](../workflows/define-manifest.md) describes which Marbles (service
The Coordinator enforces the manifest and only admits the desired Marbles to the cluster.
Each Marble registers itself with a quote via gRPC to the Coordinator. This quote is tied to the TLS session of the gRPC call. It contains the Marble's measurements. The Coordinator compares them to the manifest to ensure the Marble's identity and integrity.
The [Marble's TLS credentials](../features/secrets-management.md) are generated by the Coordinator and signed with the intermediate certificate.
The Coordinator's certificate chain as well as the freshly generated Marble certificate are then delivered to the Marble.
The Coordinator's certificate chain and the freshly generated Marble certificate are then delivered to the Marble.

![Marble deployment](../_media/marble_deployment.svg)

Expand All @@ -35,17 +35,17 @@ If a client wants to connect to a Marble, the Coordinator acts as a trusted CA t

![service mesh](../_media/service_mesh.svg)

When the Coordinator receives an update to the manifest, e.g. a Marble is updated to a newer security version, a new intermediate certificate is generated. The communication between Marbles relies on the intermediate certificate as root of trust. This ensures that Marbles which were deployed before the update can't communicate with Marbles that have the newer security version.
When the Coordinator receives an update to the manifest, e.g., a Marble is updated to a newer security version, a new intermediate certificate is generated. The communication between Marbles relies on the intermediate certificate as the root of trust. This ensures that Marbles which were deployed before the update can't communicate with Marbles that have the newer security version.

## Cluster attestation

The Coordinator issues one concise remote-attestation statement for your whole distributed app.
The Coordinator issues one concise remote attestation statement for your whole distributed app.
The Coordinator's build-time measurement is distributed to the relying party (this must be done by the admin or operator).
The relying party requests the Coordinator's quote and certificate chain.
The quote contains the hash of the Coordinator's root CA certificate, which is verified against the received certificate chain.
The quote also contains the Coordinator's measurement, which is verified against the build-time measurement.

The relying party then requests the manifest from the Coordinator and ensures it contains the expected Marbles and their expected measurements.
The steps required on the client-side are described in our [verification hands-on](../workflows/verification.md).
The steps required on the client side are described in the [verification hands-on](../workflows/verification.md).

![cluster attestation](../_media/verify_cluster.svg)
Loading
Loading