Inclavare Containers is an innovation of container runtime with the novel approach for launching protected containers in hardware-assisted Trusted Execution Environment (TEE) technology, aka Enclave, which can prevent the untrusted entity, such as Cloud Service Provider (CSP), from accessing the sensitive and confidential assets in use.

Inclavare Containers has the following salient features:

• Confidential computing oriented. Inclavare Containers provides a general design for the protection of tenant’s workload.
  • Create the hardware-enforced isolation between tenant’s workload and privileged software controlled by CSP.
  • Remove CSP from the Trusted Computing Base (TCB) of tenant in untrusted cloud.
  • Construct the general attestation infrastructure to convince users to trust the workloads running inside TEE based on hardware assisted enclave technology.
• OCI-compliant. The component rune is fully compliant with OCI Runtime specification.
• Cloud platform agnostic. It can be deployed in any public cloud Kubernetes platform.

Please refer to Terminology for more technical expressions used in Inclavare Containers.

Audience

Inclavare Containers is helping to keep tenants' confidential data secure so they feel confident that their data is not being exposed to CSP or their own insiders, and they can easily move their trusted applications to the cloud.

Architecture

Inclavare Containers follows the classic container runtime design. It takes the adaption to containerd as first class, and uses dedicated shim-rune to interface with OCI Runtime rune. In the downstrem, init-runelet employs a novel approach of launching enclave runtime and trusted application in hardware-enforced enclave.

The major components of Inclavare Containers are:

• rune
  rune is a CLI tool for spawning and running enclaves in containers according to the OCI specification. rune is already written into OCI Runtime implementation list.

• shim-rune
  shim-rune resides in between containerd and rune, conducting enclave signing and management beyond the normal shim basis. In particular shim-rune and rune can compose a basic enclave containerization stack for confidential computing, providing low barrier to the use of confidential computing and the same experience as ordinary container. Please refer to this doc for the details.

• enclave runtime
  The backend of rune is a component called enclave runtime, which is responsible for loading and running trusted and protected applications inside enclaves. The interface between rune and enclave runtime is Enclave Runtime PAL API, which allows invoking enclave runtime through well-defined functions. The softwares for confidential computing may benefit from this interface to interact with cloud-native ecosystem.

  One typical class of enclave runtime implementations is based on Library OSes. Currently, the recommended enclave runtime interacting with rune is Occlum, a memory-safe, multi-process Library OS for Intel SGX. And another typical class of enclave runtime is WebAssembly Micro Runtime (WAMR) with Intel SGX, a standalone WebAssembly (WASM) runtime with a small footprint, including a VM core, an application framework and a dynamic management for WASM applications.

  In addition, you can write your own enclave runtime with any programming language and SDK (e.g, Intel SGX SDK) you prefer as long as it implements Enclave Runtime PAL API.

Non-core components

• sgx-tools
  sgx-tools is a CLI tool, used to interact Intel SGX AESM service to retrieve various materials such as launch token, quoting enclave's target information, enclave quote and remote attestation report from IAS. Refer to this tutorial for the details about its usage.

• epm
  epm is a service that is used to manage the cache pools to optimize the startup time of enclave. Refer to this tutorial for the details about its usage.

Roadmap

Please refer to Inclavare Containers Roadmap for the details. This document outlines the development roadmap for the Inclavare Containers project.

Building

Please follow the command to build Inclavare Containers from the latested source code on your system.

1. Download the latest source code of Inclavare Containers

mkdir -p "$WORKSPACE"
cd "$WORKSPACE"
git clone https://github.com/alibaba/inclavare-containers

2. Prepare the tools required by Inclavare Containers

cd inclavare-containers

# install Go protobuf plugin for protobuf3
go get github.com/golang/protobuf/[email protected]

3. Build Inclavare Containers

# build rune, shim-rune, epm and sgx-tools
make

Installing

After build Inclavare Containers on your system, you can use the following command to install Inclavare Containers on your system.

sudo make install

rune will be installed to /usr/local/bin/rune on your system. shim-rune will be installed to /usr/local/bin/containerd-shim-rune-v2. epm will be installed to /usr/local/bin/epm. sgx-tools will be installed to /usr/local/bin/sgx-tools.

If you don't want to build and install Inclavare Containers from latest source code. We also provide RPM/DEB repository to help you install Inclavare Containers quickly. Please see the steps about how to configure repository firstly. Then you can run the following command to install Inclavare Containers on your system.

• On CentOS 8.1

sudo yum install rune shim-rune epm sgx-tools

• On Ubuntu 18.04 server

sudo apt-get install rune shim-rune epm sgx-tools

Integrating

Inclavare Containers can be integrated with dockerd, containerd, and pouchd.

The former targets using docker to deploy Inclavare Containers. Specifically, you need to install the preferred enclave runtime when building container images, and then launch the enclave runtime through rune and enclave runtime specific PAL.

The latter targets using K8s to deploy Inclavare Containers. In this scenario, shim-rune and rune can compose an enclave containerization stack, so enclave runtime is not required and installed when building container images, providing with the same experience as ordinary containers.

dockerd

Add the assocated configurations for rune in dockerd config file, e.g, /etc/docker/daemon.json, on your system.

{
        "runtimes": {
                "rune": {
                        "path": "/usr/local/bin/rune",
                        "runtimeArgs": []
                }
        }
}

then restart dockerd on your system.

You can check whether rune is correctly enabled or not with:

docker info | grep rune

containerd

Add the assocated configurations for shim-rune in containerd config file, e.g, /etc/containerd/config.toml, on your system.

        [plugins.cri.containerd]
          ...
          [plugins.cri.containerd.runtimes.rune]
            runtime_type = "io.containerd.rune.v2"

then restart containerd on your system.

pouchd

Add the assocated configurations in pouchd config file, e.g, /etc/pouch/config.json, on your system.

	"add-runtime": {
		"rune": {
            		"path": "/usr/local/bin/rune",
            		"runtimeArgs": null,
            		"type": "io.containerd.runtime.v1.linux"
        	},
		...
	}	

where:

• @path: specify the path of OCI Runtime, such as the pach of rune.
• @runtimeArgs: specify the arguments of the pouchd runtime, such as --platform, --network.
• @type: specify the shim template from the following candidates:
  • io.containerd.runtime.v1.linux: correspond to containerd-shim
  • io.containerd.rune.v2: correspond to shim-rune
  • io.containerd.runc.v1/2: correspond to containerd-shim-runc-v1/2

then restart pouchd on your system.

You can check whether rune is correctly enabled or not with:

pouch info | grep rune

Deployment

Inclavare Containers can be deployed with dockerd, containerd, and pouchd.

dockerd

Occlum LibOS

Please refer to this guide to run Occlum with rune and docker.

WebAssembly Micro Runtime (WAMR)

Please refer to this guide to run WAMR with rune.

containerd

Please refer to this guide to deploy an enclave container in a Kubernetes cluster.

The reference container images are available for the demonstration purpose to show how a Confidential Computing Kubernetes Cluster with Inclavare Containers works. Currently, web application demos based on OpenJDK 11, Dragonwell, and Golang are provided.

All reference container images don't contain enclave runtime. They are only used for the deployment with containerd.

pouchd

Please refer to this guide to install pouch and refer to this guide to configure the runtime of pouchd.

pouch + pouchd + rune

Occlum LibOS

Please refer to this guide to build your Occlum container image. Then run pouch with Occlum container images refer to

pouch run -it --rm --runtime=rune \
  -e ENCLAVE_TYPE=intelSgx \
  -e ENCLAVE_RUNTIME_PATH=/opt/occlum/build/lib/libocclum-pal.so \
  -e ENCLAVE_RUNTIME_ARGS=occlum_instance \
  occlum-app

In addition, pouch supports to configure annotation` options to run container image. You can run pouch with annotations instead of environment variables.

pouch run -it --rm --runtime=rune \
  --annotation "enclave.type=intelSgx" \
  --annotation "enclave.runtime.path=/opt/occlum/build/lib/libocclum-pal.so" \
  --annotation "enclave.runtime.args=occlum_instance" \
  occlum-app