Introducing Project Raksh
Securing data and code have been an area of focus across CPU architectures. For example:
- Intel provides SGX and Total Memory Encryption (TME/MKTME)
- AMD provides Secure Memory Encryption and Secure Encrypted Virtualization (SEV)
- IBM Power Secure virtual machine (SVM) and Protected Execution Facility (PEF)
Recently our focus has been to secure containerised workload by leveraging VM based Trusted Execution Environment with the aim of protecting in-use data and code without changing application code. Our team has a long history in working towards isolating and securing container workloads for our customers. The picture below gives a quick timeline overview.
You can read more about our work in this space by following the links in the reference section.
In this blog, I’ll focus on project Raksh (रक्ष) which means protect. We created this project with the aim to secure Kubernetes deployed workload along with its specification (POD or Deployment YAML) by leveraging VM based Trusted Execution Environment (TEE).
Simply put Raksh makes it easier to use VM based TEE with containers in a Kubernetes cluster.
Some of the key aspects of Raksh are:
- Introduces Secure Containers which are containers protected by VM based TEEs (eg Power PEF, AMD SEV, Intel MKTME).
- Introduces encryption for Kubernetes application spec (pod.yaml, deployment.yaml etc), thereby ensuring that the container details (image name, command, arguments etc) are protected from unauthorised entities including admin users on the host.
- No changes to Kubernetes application deployment workflow.
- Built on the Kubernetes Operator pattern.
- Leverages Kata VM container runtime.
The remaining part of the article goes into specific details by taking the example of a VM based TEE as provided by IBM Power processors.
Protected Execution Facility (PEF) provides the ability to secure data-in-use by protecting access to specific memory regions. It’s built-up on secure and trusted boot. Each system has a public/private key pair where the private key is protected by a TPM and is useable only if the correct and verified firmware has been launched. PEF introduces the concept of secure virtual machine (SVM) whereby anything running inside the SVM is protected. The SVM is the VM based TEE. The secure container runs inside this SVM.
Here are few key aspects of SVM and secure containers:
- An SVM can run only on PEF capable systems.
- Each system that supports PEF has a public/private key pair where the private key is known only to the system.
- An SVM image consists of bootloader, lockboxes and encrypted rootfs.
- A lockbox is a blob, which is self-protecting and contains required keys, secrets and other metadata encrypted with the public key of the target systems
A high level overview of the components involved can be seen in the figure below
More details on SVM and PEF is available from the following links:
Our goal was to integrate VM based TEEs with Kubernetes and make it consumable for end users. Since the protection and isolation is provided by the virtualisation layer (KVM) in conjunction with the hardware, the natural choice was to leverage Kata containers as the basis. There are already examples of Kata integration with different virtualisation technologies for improved security and isolation (firecracker etc).
The following figure shows the components in the Kubernetes worker node when running secure containers with Raksh.
Securing a containerised app with Raksh is broadly a two-step process:
- Create VM (Kata VM) initrd image
- Modify the existing deployment YAML to use encrypted container specification.
We use a modified Kata agent with the following functionalities to ensure all container life-cycle operation gets handled inside the VM:
- Support for decrypting the spec inside the VM
- Creating the containers based on the decrypted spec
We have made it easy to try the overall workflow without the need for a TEE capable system. You can run it on any KVM system.
Here is a short demo of executing the workflow on Intel KVM system .
Please give it a try and share your feedback. PRs welcome ☺
I would like to conclude by thanking the team especially Abhishek, Harshal, Manjunath, Nitesh, Sudipta, Suhail, community members and many others in our long journey towards isolating and securing containers.
- Raksh architecture details — https://github.com/IBM/raksh/blob/master/docs/architecture.md
- What you define is what you deploy — http://bit.ly/wydiwud