Add compartment_alloc "side compartment" example

Adds an example allocator which manages "side compartments", which are
introduced in this example's README. There are two functions here which
are relevant to creating compartments and allocating memory within them.

1. `init_compartment` `mmap`s an area of memory, creates a "compartment"
   (see `README` for a high-level definition, or `compartment_alloc.h`
   for the definition of the struct itself) and returns a sealed
   capability which can be used to identify a compartment when
   allocating memory.
2. `malloc_compartment` allocates memory within a compartment's `mmap`'d
   buffer. It accepts two arguments: a number of bytes to allocate, and
   a capability identifying a compartment (i.e. one returned by
   `init_compartment`). It returns a capability which points to memory
   within the compartment, assuming it's able to allocate (errors such
   as running our of space cause the program to exit with RC 1, because
   this is an example and it doesn't have to handle things particularly
   gracefully).

There's also a function to free a compartment — `free_compartment` —
which takes a compartment's identifying capability and frees the mmap'd
buffer associated with it. This is naive and not a complete
implementation; capabilities pointing to somewhere in that now-free
buffer are still valid capabilities, but will be unsafe to use.

example_allocators/compartment_alloc/Makefile.morello-hybrid

@@ -0,0 +1,6 @@
+# Copyright (c) 2021 The CapableVMs "CHERI Examples" Contributors.
+# SPDX-License-Identifier: MIT OR Apache-2.0
+
+include ../../build/Makefile.vars.morello-hybrid
+include ../../build/Makefile.vars.common
+include ../../build/Makefile.simple

example_allocators/compartment_alloc/README.md

@@ -0,0 +1,84 @@
+# Compartmentalising Allocator
+
+A modification of the "bump allocator" in this repo. This allocator creates
+compartments on the heap, and returns capabilities to allocations within the
+compartment.
+
+Because allocations are made against a compartment's (previously `mmap`'d)
+buffer, we need to identify the compartment being used when we call `malloc`.
+`init_compartment` returns a capability used to identify the compartment it
+creates; calls to `malloc_compartment` accept this identifier as a parameter.
+
+`main.c` shows a simple use, with allocations against two different
+compartments.
+
+Example is run using `make` in the typical fashion for the repo (i.e. along the
+lines of `make -f Makefile.morello-hybrid run-main`).
+
+## What are "compartments" here?
+
+A "compartment" as created by this allocator is different conceptually to the
+compartments created by the other examples in this repo. It's worth explaining
+what "compartment" means in this context.
+
+Here, the "compartment" is some heap-allocated space that only capabilities are
+used to access. There are obviously ways to abuse "just some space on the heap"
+if the DDC isn't also restricted. The easiest way to explain this is to draw it
+out.
+
+Assuming only the DDC is being used for compartmentalisation (so we're
+constraining, say, dereferencing of addresses for data but not jump points etc)
+and it's being used to bound which addresses can be legally dereferenced, we
+could draw memory out as a "number line" of the address space and mark points
+where a compartment's bounds begin and end. So we could diagram compartments
+like so:
+
+```
+|--------------------|
+0  g   f      F   G  ♾️
+```
+
+…where a compartment's lower bound is a lowercase letter and its upper bound
+the corresponding upper case letter (here ♾️ is the maximum address, and the `G`
+compartment's lower bound is marked by `g`, and the upper case marked by `G`;
+there's also a compartment, `F`, which is more restricted than `G` and is a
+strict subset of `G`). Clearly, this just defines compartment bounds relative
+to each other, and not in any kind of detail, but all we're interested in is
+the relative positioning of compartment boundaries here.
+
+The compartments this example constructs are intended to sit _outwith_ all
+other compartments, so that any access performed against them _must_ be done
+through the capabilities returned by `malloc_compartment`. For this to work,
+the DDC needs to be restricted when a program first runs, so that no future
+compartment's bounds will overlap with the ones we construct on the heap. Then,
+the ordinary compartments we create (derived from the now-restricted initial
+DDC) can't overlap with these separated heap-y compartments.
+
+I've taken to thinking of them as _"side compartments"_ to differentiate them
+from ordinary compartments.
+
+To illustrate what I mean, we can create a special space adjacent to our
+ordinary compartments and allocations like this:
+
+```
+|---------------------|
+0              ><     ♾️ 
+```
+
+…where `>` marks the upper bound of a DDC we install right when a program
+launches, and `<` marks the lower bound of possible "side" compartments. To
+extend the first diagram, we might make a side compartment that looks like…:
+
+```
+|------------------------------|
+0  g   f    F   G  ><  p   P   ♾️
+```
+
+…and the "side" compartment `P` is what `compartment` means in this instance.
+Those compartments are similar to the buffers created in the `bump-allocator`
+example in their structure.
+
+The intended use of these compartments is to create an isolated area of memory
+which is only accessible via capabilities, so that a process can't just
+calculate an address within the isolated area and dereference it (because the
+limited DDC prevents this).

example_allocators/compartment_alloc/compartment_alloc.c

@@ -0,0 +1,128 @@
+#include "compartment_alloc.h"
+#include "utils.h"
+#include <cheriintrin.h>
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/mman.h>
+
+heap_compartment compartments[maxCompartments];
+int numCompartments = 0;
+
+/*
+ * Initialise a compartment with `size_in_bytes` space, where all future
+ * allocations will be constrained by `dc` as if it was the DDC when allocation
+ * occurred.
+ */
+void *__capability init_compartment(size_t size_in_bytes, void *__capability dc)
+{
+    void *buf;
+    void *__capability compartment_id;
+
+    if (numCompartments + 1 == maxCompartments)
+    {
+        perror("Too many compartments requested.");
+        exit(1);
+    }
+
+    // Allocate memory for this compartment
+    buf = mmap(NULL, size_in_bytes, PROT_READ | PROT_WRITE, MAP_ANON | MAP_PRIVATE, -1, 0);
+    if (buf == MAP_FAILED)
+    {
+        perror("error in initial mem allocation");
+        exit(-1);
+    }
+
+    // Create the new compartment
+    compartment_id = sealed_reference(buf);
+    compartments[numCompartments] = (heap_compartment){
+        buf,           // Space on the heap
+        0,             // bytes allocated (currently none!)
+        size_in_bytes, // Our maximum size, to check when allocating
+        dc,            // The data capability to allocate against (like a DDC)
+        compartment_id // An identifier which allows a holder to allocate in this component
+    };
+    numCompartments++;
+
+    // Return the sealed capability which acts as a token of authentication for a component.
+    return compartment_id;
+}
+
+/*
+ * Allocate `len` bytes in the compartment identified by `compartment_id`.
+ * if `compartment_id` doesn't exist, returns NULL
+ */
+void *__capability malloc_compartment(size_t len, void *__capability compartment_id)
+{
+    size_t rounded_length;
+    size_t new_allocated;
+    heap_compartment compartment = compartments[0];
+    void *__capability allocated;
+    void *addr;
+    int i;
+
+    // Search for a compartment with the given identifier
+    for (i = 0; i < maxCompartments && compartments[i].identifier != compartment_id; i++)
+    {
+    }
+    if (i == maxCompartments)
+    {
+        perror("Given an ID for a non-existent compartment");
+        exit(1);
+    }
+
+    compartment = compartments[i];
+
+    // Try to "bump-allocate" some space in the compartment's buffer
+    addr = compartment.buffer + compartment.bytes_allocated;
+
+    addr = __builtin_align_up(addr, ~cheri_representable_alignment_mask(len) + 1);
+    rounded_length = cheri_representable_length(len);
+
+    new_allocated = (addr + rounded_length) - compartment.buffer;
+    if (new_allocated > compartment.max_allocated)
+    {
+        perror("Maximum # bytes in compartment exceeded.");
+        exit(1);
+    }
+
+    compartment.bytes_allocated = new_allocated;
+
+    // We allocated some space!
+    // Create a capability pointing to it and return that.
+    // The capability inherits the metadata of data capability for this compartment.
+    allocated = cheri_address_set(compartment.datacap, (long) addr);
+    allocated = cheri_bounds_set_exact(allocated, rounded_length);
+
+    return allocated;
+}
+
+/*
+ * Unmaps the compartment identified by the `compartment_id` argument,
+ * effectively freeing it.
+ *
+ * Note that this is a naive implementation for demo purposes. The capabilities
+ * previously returned by `malloc_compartment` now point to de-allocated memory.
+ */
+void free_compartment(void *__capability compartment_id)
+{
+    int munmap_rc;
+    int i;
+
+    // Search for a compartment with the given identifier
+    for (i = 0; i < maxCompartments && compartments[i].identifier != compartment_id; i++)
+    {
+    }
+    if (i == maxCompartments)
+    {
+        perror("Given an ID for a non-existent compartment");
+        exit(1);
+    }
+
+    munmap_rc = munmap(compartments[i].buffer, compartments[i].max_allocated);
+    if (munmap_rc != 0)
+    {
+        perror("Attempted to deallocate a compartment, but munmap errored");
+        exit(1);
+    }
+}

example_allocators/compartment_alloc/compartment_alloc.h

@@ -0,0 +1,14 @@
+#define maxCompartments 5
+
+typedef struct _heap_compartment
+{
+    void *buffer;
+    size_t bytes_allocated;
+    size_t max_allocated;
+    void *__capability datacap;
+    void *__capability identifier; // NOTE: THIS IS SEALED
+} heap_compartment;
+
+void *__capability init_compartment(size_t size_in_bytes, void *__capability dc);
+void *__capability malloc_compartment(size_t len, void *__capability component_id);
+void free_compartment(void *__capability compartment_id);

example_allocators/compartment_alloc/main.c

@@ -0,0 +1,37 @@
+#include "../../include/common.h"
+#include "compartment_alloc.c"
+
+int main()
+{
+    // Create capabilities which the heap compartments will construct capabilities against
+    // (as if they were the DDC when the allocation is made).
+    // One will be read-only, the other write-only, so we can differentiate when printing.
+    void *__capability dc1 = cheri_perms_and(cheri_ddc_get(), CHERI_PERM_STORE);
+    void *__capability dc2 = cheri_perms_and(cheri_ddc_get(), CHERI_PERM_LOAD);
+
+    // Create two compartments, and receive the capability that gives us authority to use them
+    void *__capability compartment1 = init_compartment(4096, dc1);
+    void *__capability compartment2 = init_compartment(4096, dc2);
+
+    // Allocate memory in each compartment
+    void *__capability c1_allocated_memory = malloc_compartment(64, compartment1);
+    void *__capability c2_allocated_memory = malloc_compartment(256, compartment2);
+
+    // Print an explainer of each allocation (and the capability itself)
+    printf("\n\n\tThis capability is heap-allocated within our first compartment, which is "
+           "write-only.\n\tWe allocated 64 bytes.\n");
+    pp_cap(c1_allocated_memory);
+
+    printf("\n\n\tThis capability is heap-allocated within our second compartment, which is "
+           "read-only.\n\tWe allocated 256 bytes.\n");
+    pp_cap(c2_allocated_memory);
+
+    // Clean up compartments
+    free_compartment(compartment1);
+    free_compartment(compartment2);
+
+    printf("\nFreed compartments.\n");
+
+    printf("\nCompleted successfully.\n");
+    return 0;
+}

example_allocators/compartment_alloc/utils.h

@@ -0,0 +1,17 @@
+#include <stdlib.h>
+#include <sys/sysctl.h>
+
+// A shortcut to turn a pointer into a sealed capability
+#define sealed_reference(ptr) cheri_seal((void *__capability) ptr, cheri_get_sealcap())
+
+void *__capability cheri_get_sealcap()
+{
+    void *__capability sealcap;
+    size_t sealcap_size = sizeof(sealcap);
+    if (sysctlbyname("security.cheri.sealcap", &sealcap, &sealcap_size, NULL, 0) < 0)
+    {
+        printf("Fatal error. Cannot get `security.cheri.sealcap`.");
+        exit(1);
+    }
+    return sealcap;
+}

tests/run_tests.sh

@@ -87,6 +87,7 @@ elif [ "$1" = "morello-hybrid" ]; then
     run OK hybrid/compartment_examples/inter_comp_call/base main
     run OK hybrid/compartment_examples/inter_comp_call/malicious_compartments inter_comp_call-secure
     run OK syscall-restrict syscall-restrict
+    run OK example_allocators/compartment_alloc main
 else
     echo "$1 not recognised."
     exit 1