Working with Red Hat Enterprise Linux CoreOS Extensions

Red Hat Enterprise Linux CoreOS (RHCOS) is the default operating system for OpenShift. Recently OpenShift added support for RHCOS extensions which allows extending RHCOS in a controlled fashion.

In this article let’s see how to enable a new RHCOS extension. This is only required if you are an OpenShift engineer working on enhancements. As an OpenShift user you won’t need this.

The RHCOS contents are part of the OpenShift release image itself and is referred to as machine-os-content. Consequently adding a new RHCOS extension will require working with machine-os-content.

Let’s take an enhancement scenario and work through the steps.


Add a new RHCOS extension — qemu-img to be deployed via OpenShift Machine Config Operator (MCO)

At a high level adding and deploying a new RHCOS extension consists of the following steps:

Let’s go through each of these steps in detail.

Creating new machine-os-content

We need to a select a base machine-os-content image to start with. Let’s say we use the latest 4.7 nightly build image as the base.

The release image details are available from the following link —

Look for the text like this — “Pull From:”

Pull From:

Note that this will change as new builds are made available.

You’ll need OpenShift pull secret to download the content. Get it from the following link.

Assuming the pull secret is saved in the file openshift-pull-secret.txt you can extract the machine-os-content from the release payload image using the following command. This is useful if you want to view the contents and understand the layout of RHCOS.

oc image extract -a ~/openshift-pull-secret.txt — path /:/extracted-oscontent

You can now view the contents under /extracted-oscontent directory. For example you can find the current RHCOS extensions under extensions sub-directory.

The following Dockerfile extends this release image and adds the qemu-img package.

FROM dnf install -y createrepoRUN dnf install -y — downloadonly — downloaddir=/extensions/qemu-img qemu-imgRUN createrepo /extensionsENTRYPOINT [“bin/bash”]

Build the image.

podman build -t -f Dockerfile .

Note: If the host is not registered via subscription manager then image build will not be able to download the required rpm packages. An alternative mechanism is to explicitly provide yum repos during build.

podman build -v /etc/yum.repos.d/Linux-AppStream.repo:/etc/yum.repos.d/Linux-AppStream.repo -v /etc/yum.repos.d/Linux-BaseOS.repo:/etc/yum.repos.d/Linux-BaseOS.repo -t test -f Dockerfile .

Push the image to the registry.

podman push

Creating new MCO image with extension support

You’ll need to build the machine config operator (MCO) image with changes to add support for the new (qemu-img) extension.

Have a look at the following code for the MCO change to add “qemu-img” extension.

Build the MCO image.

git clone -b qemu-img
cd machine-config-operator
make image
podman tag localhost/machine-config-operator:latest

Push the image to the registry.

podman push

Creating a new OpenShift release image

The final step is to create a new OpenShift release image and using it for installation and testing the extension.

oc adm release new -n origin

Upgrade OpenShift Cluster with the new release image

For testing, we’ll upgrade an existing cluster with the new image and verify that the extension is getting installed.

oc adm upgrade --force --to-image --allow-explicit-upgrade

Update usually will take 20–30 min.

Watch Update progress by checking the PROGRESSING field.

oc get clusterversion
version 4.7.0-0.nightly-2021-01-28-060516 True True 42m Working towards 4.7.0-0.nightly-2021-01-19-083458: 84% complete

Once the upgrade is complete, check if MCO image points to the new image.

oc get pod machine-config-operator-7864794c9f-6jps8  -n openshift-machine-config-operator -o yaml[snip]containerStatuses:
- containerID: cri-o://1030351e1d11fe581757c4a43ae75bbd076e990bf7b96a2289fbf329f93a4d50image:
lastState: {}
name: machine-config-operator
ready: true

Check if osImageURL points to new machine-os-content

oc -n openshift-machine-config-operator edit configmap/machine-config-osimageurlapiVersion: v1
releaseVersion: 4.7.0–0.nightly-2021–01–19–083458
kind: ConfigMap
annotations: “true” “true” “true”
creationTimestamp: “2021–01–19T10:12:44Z”
name: machine-config-osimageurl
namespace: openshift-machine-config-operator
resourceVersion: “44890”
selfLink: /api/v1/namespaces/openshift-machine-config-operator/configmaps/machine-config-osimageurl
uid: ff1ed7b1-bf9c-414d-8384–42d66633d0e7

Deploy the new extension

Check the worker nodes to ensure the “qemu-img” extension is not installed. For example here is the o/p from one of the worker nodes in my setup

oc debug node/ip-10–0–147–144.ec2.internal
Creating debug namespace/openshift-debug-node-rvnvx …
Starting pod/ip-10–0–147–144ec2internal-debug …
To use host binaries, run `chroot /host`
Pod IP:
If you don’t see a command prompt, try pressing enter.
sh-4.4# chroot /host
sh-4.4# rpm -qa | grep qemu-img

Now apply the extension by creating a machineconfig.

Following is the sample machineconfig used to apply the extension — 80-extensions.yaml

kind: MachineConfig
labels: worker
name: 80-worker-extensions
version: 3.1.0
- qemu-img
oc create -f 80-extensions.yaml

Watch progress

oc get machineconfigpools

Wait for the worker machineconfigpool to be updated. You should see an o/p like what is shown below. The UPDATING and DEGRADED columns should be FALSE

master rendered-master-02910293114b441bd4e08b574159dbeb True False False 3 3 3 0 107m
worker rendered-worker-d17e4a8e84ef124243d8397b79348839 True False False 3 3 3 0 107m

Login to the worker nodes and check for qemu-img extension

oc debug node/ip-10–0–147–144.ec2.internal
Creating debug namespace/openshift-debug-node-wccmn …
Starting pod/ip-10–0–147–144ec2internal-debug …
To use host binaries, run `chroot /host`
Pod IP:
If you don’t see a command prompt, try pressing enter.
sh-4.4# chroot /host
sh-4.4# rpm -qa | grep qemu-img

Happy hacking !!


Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store