Skip to content

Latest commit

 

History

History
executable file
·
374 lines (243 loc) · 21.2 KB

README.md

File metadata and controls

executable file
·
374 lines (243 loc) · 21.2 KB
Raw

The Container Ecosystem Project

The ecosystem of awesome new technologies emerging around containers and microservices can be a little overwhelming, to say the least. We thought we might be able to help: welcome to the Container Ecosystem Project. The goals of this project are (1) to clearly lay out the different types technologies that make up the growing container ecosystem and the microservices technology stack – starting from the lowest levels of core container technology, and rising up through layers of abstraction to full-blown container platforms and support tools – and (2) to put forth the latest and greatest examples of each type of technology.

This project is a living document - please submit edits/pull requests for suggested changes.

View this project as a webpage here:
http://www.sysdig.com/the-container-ecosystem-project/

Table of Contents

Core Container Technologies

Use these tools to run a small number of containers on a single host

Container specifications

An abstract definition of a standard "container", allowing an ecosystem of technologies to support a standard container with potentially multiple, interchangeable runtime implementations

  • Docker open source

  • CoreOS open source

    • AppC (deprecated): CoreOS is now supporting the OCI

Container runtimes

This is your actual running container (essentially an abstraction of Linux kernel components like namespaces and cgroups that allow virtualization on top of a shared kernel)

  • Docker open source

    • runc: Docker's container runtime, now donated to the OCI as the initial implementation of the standard; essentially a repackaging of libcontainer
    • libcontainer: a Linux container library; enables and abstracts interactions with Linux kernel components to create and control containers

  • CoreOS open source

    • rkt: CoreOS's container runtime; initially an implementation of the AppC specification, which is now being rolled into the OCI spec

  • Other open source

    • LXC: a Linux container library; originally utilized by runc until release of libcontainer
    • OpenVZ: a Linux container library

Container management

These tools abstract low level control of your container runtime adding further functionality and usability

  • Docker open source

    • Docker Engine (aka "Docker"): the core of Docker and its primary interface; creates and runs Docker containers; includes:
      • Docker daemon: runs as a process on the host machine and provides an API that abstracts basic container control functions
      • Docker client: a CLI for interacting with the Docker daemon

  • CoreOS open source

    • rkt CLI: rkt's container management functionality is delivered on-demand by a binary, rather than a daemon background process

  • Other open source

    • LXD: daemon and UI for LXC
    • libvirt: container and virtualization mgmt library that supports LXC, OpenVZ, and a variety of hypervisor technologies

Container definition

These tools allow you to define specific containers, so they can be saved, shared and reproduced

  • Docker open source

    • Docker image: a template representing a fully configured container; Docker container runtimes are created from these images; images are created with Dockerfiles and shared over registries
    • Dockerfile: text file containing all the commands needed to build a Docker image

  • CoreOS open source

Registries

Repositories for storing and sharing container images

Operating systems

OS's that are designed for hosting containers

  • Docker open source

    • boot2docker (basically deprecated by Docker Machine): minimalist Linux for running Docker on PC and Mac in a VM; now used by Docker Machine in certain environments

  • CoreOS open source

    • CoreOS: minimalist OS built for running distributed, containerized apps; includes etcd and fleet

  • Other open source

VM management

These tools help you manage the host virtual environments in which you run your containers

  • Docker open source

    • Docker Machine: creates and manages host VMs running Docker, including local VMs (eg. VirtualBox) and cloud VMs (eg. Amazon AWS, Google GCP)

  • Other open source

    • Hashicorp Vagrant: creates pre-configured VMs for dev environments based on a variety of "Providers" (virtualization technologies) including Docker containers
    • Hashicorp Otto: extends Vagrant to deploy and manage VMs across many platforms

Distributed Container Technologies

Use these technologies to run applications on a distributed cluster of containers

Scheduling

These tools manage placement of new containers across abstracted underlying resources

  • Docker open source

    • Docker Swarm: designed to extend Docker API to a cluster; includes scheduling and service discovery

  • CoreOS open source

    • fleet: low level orchestration included in CoreOS; supports basic scheduling; can be used to bootstrap Kubernetes for higher level orchestration

  • Other open source

    • Chronos: framework for scheduling on Mesos

Cluster definition

These tools allow you to define and manage a cluster of dependent containers as a single composable entity

  • Docker open source

    • Docker Compose: text files used to define and configure a distributed application across a cluster of Docker containers

  • CoreOS open source

    • fleet unit file: fleet uses a specialized version of systemd unit files to define a distributed application across containers

Service discovery / Distributed configuration storage

These tools allow applications within different containers to discover each other and share configuration information (eg. IP addresses or application settings); usually implemented as a globally distributed key-value store

  • Docker open source

    • Docker Swarm comes with built in service discovery, but can also use etcd, Consul, Zookeeper

  • CoreOS open source

    • etcd: globally distributed key-value store; included with CoreOS for service discovery

  • Other open source

    • Marathon: framework for initializing long running jobs on Mesos; includes service discovery and cluster management functionality
    • Hashicorp Consul: service discovery, key/value store, and cluster health checking; uses Serf
    • Apache ZooKeeper: globally distributed key-value store

Dynamic configuration management

These tools let you dynamically update application settings based on changes to your distributed key-value store in applications that don't natively support this

  • CoreOS open source

    • confd: originally built for etcd, but now supports Consul and ZooKeeper

  • Other open source

Container Platform Technologies

Use these technologies as complete platforms for running distributed applications across clusters of containers

Container orchestration platforms

These platforms include or abstract away all of the core functionality (listed above) needed for container cluster management ("orchestration"), including container management, scheduling, cluster definition, and service discovery

  • Docker open source

    • Docker Swarm, Compose, and Machine can all run together to create a complete orchestration platform (still beta); Docker Swarm can also support more advanced orchestration tools like Kubernetes

  • Other open source

    • Apache Mesos: mature, highly scalable service that abstracts a pool of underlying resources and distributes "tasks" (including Docker images) from various application frameworks; uses Marathon and Chronos to add cluster management, scheduling, and service discovery; also can support Kubernetes
    • Kubernetes: orchestration platform designed specifically for running microservices on clusters of containers; includes scheduling, cluster management and service discovery through abstractions such as "pods", "replication controllers (RCs)", and "services"; originally from Google, now donated to the CNCF
    • Hashicorp Nomad: uses Consul

Hosted container platforms

These platforms offer container hosting and orchestration as a service

Container platform management

These technologies add further abstracted management and control layers to distributed container environments, often through GUIs

Container-based PaaS

These platforms further abstract container-based infrastructures by managing application code deployment and offering PaaS-like user experiences

Container-Native Support Technologies

Use these additional container-native tools to support your container-based infrastructure

Networking

  • Docker open source

    • Docker port expose: Docker feature that links a container port to a host port
    • Docker linking: Docker feature offering a basic connection between containers on the same host
    • libnetwork: advanced container networking library (still "under heavy development")

  • CoreOS open source

    • flannel: overlay network built using etcd that gives each host a separate subnet for its containers

  • Other open source

    • Weave: overlay network that puts all containers in a distributed system onto a single virtual network; also includes service discovery functionality
    • Calico: layer 3 virtual network that provides each container with an IP address

Monitoring / Visibility

  • Docker open source

    • Docker ps/top/stats: runtime commands
    • Docker stats API: remote API for streaming basic container metrics; utilized by the Docker Ecosystem Technology Partners for Monitoring

  • Other open source

    • sysdig: CLI for deep system/containers visibility; includes curses-based "csysdig" interface
    • cAdvisor: basic container metrics exporter from Google; includes web GUI; Heapster adds Kubernetes support
    • Weave Scope: container network topologies

  • Commercial

    • Sysdig Cloud: uses sysdig; includes web-based UI, application topologies, and support for all major container formats and orchestration platforms

Data layer

  • Other open source
    • CusterHQ Flocker: data volume manager for running stateful services like databases in containers

Log management

  • Docker open source

  • Other open source

    • logspout: log router for Docker containers

CI/CD

Security

Getting started aides

  • Docker open source

About the Container Ecosystem Project

Here at Sysdig, the container-native visibility company, we talk to a lot of people in the container ecosystem: both consumers and producers of technology. And wow, there is a LOT of cool technology out there – and so much more coming out all the time. It can be hard to keep up with, even if you're a seasoned expert, much less as a curious newcomer just trying to figure out where to start. There are plenty of great guides out there for various container technologies and use cases (see below for some links). But we had yet to find a clearly organized survey of the different core technologies that make up the container ecosystem and the typical microservices stack. So we decided to make one: the Container Ecosystem Project.

For each type of technology, we've tried to provide a brief description, as well as list examples currently available for that technology. We've separated out open source solutions from commercial offerings. Two of the leading open source container technology producers, Docker and CoreOS, each get their own sections. Throughout the doc, we've tried to mark beta technologies and parent technologies accordingly.

This framework is not, of course, a perfect science, and this will be a work in progress. If you have any suggested edits, please tweet us or submit edits/pull requests. We'll do our best to keep this document up to date and prune off deprecated or abandoned technologies as the ecosystem evolves.

Further Reading

Project created and maintained by Sysdig, the container-native visibility company.
http://www.sysdig.com/
http://www.sysdig.org/