Picking up from where we left in our previous post, we will now install AWS Firecracker and configure Kata Containers to use it as their hypervisor.
Build Firecracker Link to heading
Kata Containers only support AWS Firecracker v0.23.1 (yet). To build Firecracker, we will clone the Github repo and checkout to the 0.23.1 version:
1git clone https://github.com/firecracker-microvm/firecracker.git -b v0.23.1 --depth 1 &&\
2 cd firecracker &&\
3 git submodule update --init
UPDATE: Tue Feb 21 16:43:47 UTC 2023: As of kata-containers#4735, the kata go runtime supports AWS Firecracker v1.1.0. So you can use the following command:
1git clone https://github.com/firecracker-microvm/firecracker.git -b v1.1.0 --depth 1 &&\
2 cd firecracker &&\
3 git submodule update --init
Now we can build the binaries:
Note AWS Firecracker uses docker to build the image, so make sure your user can access the docker daemon, or just run with sudo.
1sudo ./tools/devtool -y build --release
2toolchain="$(uname -m)-unknown-linux-musl"
3sudo cp build/cargo_target/${toolchain}/release/firecracker /opt/kata/bin/firecracker &&\
4sudo cp build/cargo_target/${toolchain}/release/jailer /opt/kata/bin/jailer
devmapper snapshotter Link to heading
AWS Firecracker requires a block device as the backing store for a VM. To interact with containerd and kata we use the devmapper snapshotter. To check support for your containerd installation, you can run:
1ctr plugins ls |grep devmapper
if the output of the above command is:
1io.containerd.snapshotter.v1 devmapper linux/amd64 ok
then you can skip this section and move on to Configure Kata Containers to use Firecracker
If the output of the above command is:
1io.containerd.snapshotter.v1 devmapper linux/amd64 error
then we need to setup devmapper snapshotter. Based on a very useful guide from docker, we can set it up using the following scripts:
1#!/bin/bash
2set -ex
3
4DATA_DIR=/var/lib/containerd/io.containerd.snapshotter.v1.devmapper
5POOL_NAME=containerd-pool
6
7mkdir -p ${DATA_DIR}
8
9# Create data file
10sudo touch "${DATA_DIR}/data"
11sudo truncate -s 100G "${DATA_DIR}/data"
12
13# Create metadata file
14sudo touch "${DATA_DIR}/meta"
15sudo truncate -s 10G "${DATA_DIR}/meta"
16
17# Allocate loop devices
18DATA_DEV=$(sudo losetup --find --show "${DATA_DIR}/data")
19META_DEV=$(sudo losetup --find --show "${DATA_DIR}/meta")
20
21# Define thin-pool parameters.
22# See https://www.kernel.org/doc/Documentation/device-mapper/thin-provisioning.txt for details.
23SECTOR_SIZE=512
24DATA_SIZE="$(sudo blockdev --getsize64 -q ${DATA_DEV})"
25LENGTH_IN_SECTORS=$(bc <<< "${DATA_SIZE}/${SECTOR_SIZE}")
26DATA_BLOCK_SIZE=128
27LOW_WATER_MARK=32768
28
29# Create a thin-pool device
30sudo dmsetup create "${POOL_NAME}" \
31 --table "0 ${LENGTH_IN_SECTORS} thin-pool ${META_DEV} ${DATA_DEV} ${DATA_BLOCK_SIZE} ${LOW_WATER_MARK}"
32
33cat << EOF
34#
35# Add this to your config.toml configuration file and restart containerd daemon
36#
37[plugins]
38 [plugins.devmapper]
39 pool_name = "${POOL_NAME}"
40 root_path = "${DATA_DIR}"
41 base_image_size = "10GB"
42 discard_blocks = true
43EOF
Make it executable and run it:
1sudo chmod +x ~/scripts/devmapper/create.sh && \
2 cd ~/scripts/devmapper/ && \
3 sudo ./create.sh
Now, we can add the devmapper configuration provided from the script to /etc/containerd/config.toml and restart containerd.
1sudo systemctl restart containerd
We can use dmsetup to verify that the thin-pool was created successfully. We should also check that devmapper is registered and running:
1sudo dmsetup ls
2# devpool (253:0)
3sudo ctr plugins ls | grep devmapper
4# io.containerd.snapshotter.v1 devmapper linux/amd64 ok
This script needs to be run only once, while setting up the devmapper snapshotter for containerd. Afterwards, make sure that on each reboot, the thin-pool is initialized from the same data dir. Otherwise, all the fetched containers (or the ones that you’ve created) will be re-initialized. A simple script that re-creates the thin-pool from the same data dir is shown below:
1#!/bin/bash
2set -ex
3
4DATA_DIR=/var/lib/containerd/io.containerd.snapshotter.v1.devmapper
5POOL_NAME=containerd-pool
6
7# Allocate loop devices
8DATA_DEV=$(sudo losetup --find --show "${DATA_DIR}/data")
9META_DEV=$(sudo losetup --find --show "${DATA_DIR}/meta")
10
11# Define thin-pool parameters.
12# See https://www.kernel.org/doc/Documentation/device-mapper/thin-provisioning.txt for details.
13SECTOR_SIZE=512
14DATA_SIZE="$(sudo blockdev --getsize64 -q ${DATA_DEV})"
15LENGTH_IN_SECTORS=$(bc <<< "${DATA_SIZE}/${SECTOR_SIZE}")
16DATA_BLOCK_SIZE=128
17LOW_WATER_MARK=32768
18
19# Create a thin-pool device
20sudo dmsetup create "${POOL_NAME}" \
21 --table "0 ${LENGTH_IN_SECTORS} thin-pool ${META_DEV} ${DATA_DEV} ${DATA_BLOCK_SIZE} ${LOW_WATER_MARK}"
We can create a systemd service to run the above script on each reboot:
1sudo nano /lib/systemd/system/devmapper_reload.service
The service file:
1[Unit]
2Description=Devmapper reload script
3
4[Service]
5ExecStart=/path/to/script/reload.sh
6
7[Install]
8WantedBy=multi-user.target
Enable the newly created service:
1sudo systemctl daemon-reload
2sudo systemctl enable devmapper_reload.service
3sudo systemctl start devmapper_reload.service
Configure Kata Containers to use Firecracker Link to heading
Next, we need to install the Kata Containers-Firecracker configuration file. We will use this file to configure Kata Containers to use the rootfs image we built in our previous post.
1sudo mkdir -p /opt/kata/configs
2sudo install -o root -g root -m 0640 /opt/kata/share/defaults/kata-containers/configuration-fc.toml /opt/kata/configs
3sudo sed -i 's/^\(initrd =.*\)/# \1/g' /opt/kata/configs/configuration-fc.toml
4# enable seccomp
5sudo sed -i '/^disable_guest_seccomp/ s/true/false/' /opt/kata/configs/configuration-fc.toml
Make sure that /opt/kata/configs/configuration-fc.toml has an image entry pointing to the rootfs we created:
117 | image = "/opt/kata/share/kata-containers/kata-containers.img"
Configure containerd Link to heading
Next, we need to configure containerd. Add a file in your path (eg. /usr/local/bin/containerd-shim-kata-fc-v2) with the following contents:
1#!/bin/bash
2KATA_CONF_FILE=/opt/kata/configs/configuration-fc.toml /usr/local/bin/containerd-shim-kata-v2 $@
Make it executable:
1sudo chmod +x /usr/local/bin/containerd-shim-kata-fc-v2
Add the relevant section in containerd’s config.toml file (/etc/containerd/config.toml):
1[plugins.cri.containerd.runtimes]
2 [plugins.cri.containerd.runtimes.kata-fc]
3 runtime_type = "io.containerd.kata-fc.v2"
Restart containerd:
1sudo systemctl restart containerd
Verify the installation Link to heading
We are now ready to launch a container using Kata with Firecracker to verify that everything worked:
1sudo ctr images pull --snapshotter devmapper docker.io/library/ubuntu:latest
2sudo ctr run --snapshotter devmapper --runtime io.containerd.run.kata-fc.v2 -t --rm docker.io/library/ubuntu:latest ubuntu-kata-fc-test uname -a