Orchestrating MicroVMs, from first principles: Introduction

At Valyent, our mission is to spare you the pain of deploying your applications. To do this, we build an OCI image from your code and we run it on our infrastructure.

We convey the right image to the right machine with the right compute resources for each of our customers. This is called orchestration.

Orchestration is the automation of the deployment, scaling, and operation of applications. It manages resources allocation, health monitoring, ensures proper networking, handles load balancing, and oversees application updates and rollbacks.

At Valyent we build a PaaS which could be partially summarized as a managed orchestration with good UX and good DX.

Here are the specifics needs of a PaaS:

1. Orchestrating compute resources: A PaaS must accommodate a large number of users and their computing needs, so an orchestration system is needed to support the workload efficiently.
2. Networking: A PaaS need a robust networking system to accomade internal networking between apps of an organization and external networking to allow traffic from internet.
3. Isolation & multi-tenancy: Each tenant have security and privacy needs. It is unthinkable that one tenant could access the resources of another. The PaaS must ensure the isolation between each tenant.

State of the art of orchestration

The market itself presents a pantheon of tools, each with its merits and challenges :

• Kubernetes, for instance is the catch-all tool offering a lot of features but not without dragging users through intricate labyrinths. Achieving multi-tenancy in Kubernetes is hard, with its namespaces providing limited isolation, potentially leading to security and resource concerns.
• Then there's Nomad, open-source, albeit with asterisks (BSL). Its single-responsibility design necessitates external tools for functionalities like service discovery and load balancing (Consul), secrets management (Vault) etc. But most of multi-tenancy features are only included in Enterprise plan which is expensive and this for each product.
• While they are ready for multi tenancy, platforms like OpenStack and OpenNebula were primarily designed to manage giants clusters and weren’t precisely created for apps orchestration. Their focus on diverse workloads can make them seem like a maze with a lot of complex components.

Multi-Tenant Distributed System Bottleneck

As you have certainly understood, building a distributed system ready for multitenancy is not easy. The isolation requirements of a multi-tenant orchestrator can be divided into two main parts:

• Network: Ensuring isolated and secure internal networking between the apps of a tenant is crucial.
• Execution: We need to execute multiple customer, potentially unsafe, code on the same host. The application of one tenant must not interfere in any way with that of another and must only use the computing resources to which it is allocated.

In the scope of this article we'll focus on the execution part. The technology most currently used to run applications in isolation are containers, popularized by tools like Docker and widely used by Kubernetes, Nomad, etc.

The problem with containers

Containers are offering numerous benefits in terms of efficiency, portability, and modularity. However, when it comes to security, containers have intrinsic challenges that set them apart from traditional virtual machines (VMs).

Let's explore why containers might not be as secure as one might think, and how they differ from VMs in this regard:

• Shared Kernel: Containers, unlike VMs, share the host's operating system kernel. This means that if a malicious actor manages to exploit a vulnerability in the kernel, all containers running on that host could potentially be compromised. In contrast, VMs have their own full-fledged OS and kernel, providing a stronger isolation boundary and thiner attack surface.

• Weak Isolation: By design, containers are less isolated from each other than VMs. If an attacker can break out of a container, they might gain access to other containers or even the host system. Such container escape vulnerabilities can be catastrophic in a shared environment.

• Resources management: Containers sharing a host can impact each other's performance, leading to the 'noisy neighbor' phenomenon. Memory allocation can be tricky; if a container surpasses its limit, it may be terminated. Shared kernel resources, like open file descriptors, can become exhausted by one container, affecting others. Resource isolation isn't foolproof; shared resources, such as CPU cache, can be exploited.

While containers offer a plethora of benefits like speed, they intrinsically possess vulnerabilities and challenges not present in the VM realm.

Firecracker

Enter Amazon Firecracker, a cloud-native alternative. Designed for lightweight, multi-tenant environments, Firecracker offers speed, agility, and efficiency, setting it apart from counterparts like QEMU.

So it's magic, just replace the containers with Firecracker microVMs? Not really.

First, Firecracker was designed to execute serverless tasks and is the underlying engine for Amazon Lambdas. By default, it's not really made for long-running task. Second, orchestrating firecracker microVMs is not easy. There is not as much tooling as for containers. Moreover, by default, Firecracker does not natively allow you to execute OCI images.

So what do we do next ?

Introducing Ravel, an Open-Source Firecracker Orchestrator

This post is the first iteration of a long series detailing our journey to build our own microVMs orchestrator: Ravel. Here are the main characteristics expected from Ravel:

• Distributed Firecracker microVMs orchestration
• OCI images compatible out of the box
• Multi-tenant by default: secured private internal networking included
• Easy to use and to install
• Open-source

The next article will cover the implementation of the first building block of Ravel: A small golang programm to manage multiple firecracker microVMs easily.

If you're interested by Valyent and Ravel you can join the waitlist on www.valyent.dev

Thank's for reading & let’s embark on this journey together.