Cloudflare's certmgr tutorial

While running a Public Key Infrastructure (PKI), the maintenance workload due to enrolling new certificates and renewing the existing ones can quickly become overwhelming. Dealing it manually is not only cumbersome: it is frustrating too. Luckily there are ways to automate the enrollment process by providing online Registration Authority endpoints.

Cloudflare's PKI and TLS Toolkit provides both an online Registration Authority as well the client software that can be used to automatically enroll new or renew existing certificates. The aim of the "Cloudflare's certmgr tutorial - a certmgr howto" blog post is ti show how quick and easy is setting up certmgr, the certificate monitoring and automatic enrolling facility provided by Cloudflare.

The operating environment used in this post is Red Hat Enterprise Linux 9 or Rocky Linux 9 - using a different environment may lead to having to adapt or even change things.

This post is a sample from the book "A Full Featured PKI With Cloudflare's PKI and TLS Toolkit": if you are interested into a quick deep dive on this topic and be able to quickly set up a full featured nearly real-life lab with it, here is the link to the ebook and here the link to the paperback.

The Context

In this post we see how to install and configure the certmgr to track the certificate ("/etc/cfssl/tier01/ra/certs/crt.pem"), automatically renewing it along with its private key ("/etc/cfssl/tier01/ra/keys/key.pem") right before its expiration. Just to provide some context, in the lab described in the "A Full Featured PKI With Cloudflare's PKI and TLS Toolkit" that is the certificate of the online Registration Authority endpoint itself.

You can of course easily re-adapt what explained here for example to have certmgr tracking any TLS server's certificate, such as the one of a web server, of a mail server and so on. The only things you must be clear about is that certmgr must be installed and running on the host where the TLS server you want to automatically manage the certificate's lifecycle is running.

Overview

Certmgr loads its settings from JSON formatted manifests describing the needed certificates: this means that certmgr can manage the lifecycle of multiple certificates at the same time.

For each certificate described in the manifests, certmgr:

if the certificate does not exist yet, it generates first the private key, then the JSON formatted CSR and submits it to the CFSSL multi-CA Registration Authority endpoint
if the certificate exists, and the renewal threshold is hit, it automatically enrolls a new one, reloading the related TLS service (if set up to do so)

Mind that certmgr monitors the certificate existence calling the CFSSL's online Registration Authority, ... this means for example that it won't renew a non-expiring certificate only because you deleted the file from the filesystem by mistake. You must be wary also that it (correctly) does not automatically enroll a new certificate if the current one has been revoked.

Installing

Only during the installation and setup, we work as the "root" user, since almost every statement require administrative privileges and we are aware of what we are doing, it is just pointless having to prepend "sudo" at each one of them:

sudo su -

In compliance with the principle of running services with the least amount possible of privileges, we create the "cloudflare" service user and use it to run the services:

adduser -r -m -c "CloudFlare's PKI Toolkit Service User" -s /sbin/nologin cloudflare

Let's set a convenient umask so to have permission properly set when creating files and directories – this spare us from continuously having to type "chmod" statements:

umask 022

We are now ready for install certmgr.

First, decide on the certmgr’s version to install - in this post it is "3.0.3" – let’s set it into the "CERTMGR_VERSION" variable as follows:

CERTMGR_VERSION=3.0.3

The package must be owned by the "root" user, granting other users only the execution right – this way even the "cloudflare" user is not able to alter neither the binaries nor the settings.

Let's start by creating the installation directory tree beneath the "/opt/certmgr" directory:

CERTMGR_ROOT=/opt/certmgr
mkdir -m 755 ${CERTMGR_ROOT} \
${CERTMGR_ROOT}/${CERTMGR_VERSION} \
${CERTMGR_ROOT}/${CERTMGR_VERSION}/sbin \
${CERTMGR_ROOT}/${CERTMGR_VERSION}/etc

To enable keeping multiple certmgr’s versions, we also create the "current" symlink to the directory tree with the version we want to run:

ln -s ${CERTMGR_ROOT}/${CERTMGR_VERSION} ${CERTMGR_ROOT}/current

Lastly let's symlink the "etc" directory to "/etc/certmgr":

ln -s ${CERTMGR_ROOT}/${CERTMGR_VERSION}/etc /etc/certmgr

set it up to be available system-wide:

ln -s ${CERTMGR_ROOT}/current/sbin/certmgr /sbin/certmgr

Installing The Pre-built binaries

This is the easiest and quickest way of installing the Cloudflare’s certmgr.

First, set the machines’ architecture in the "ARCH" variable as follows:

ARCH=amd64

If you are working on an ARM, set ARCH=arm64.

Download the certmgr command line utility as follows:

ARCH=amd64
DOWNLOAD_URL=https://github.com/cloudflare/certmgr/releases/download
curl -L ${DOWNLOAD_URL}/v${CERTMGR_VERSION}/certmgr-linux-${ARCH}-v${CERTMGR_VERSION}.tar.gz \
| tar -C ${CERTMGR_ROOT}/${CERTMGR_VERSION}/sbin -xz
mv ${CERTMGR_ROOT}/${CERTMGR_VERSION}/sbin/certmgr-linux-${ARCH}-v${CERTMGR_VERSION} \
${CERTMGR_ROOT}/${CERTMGR_VERSION}/sbin/certmgr
chmod 755 ${CERTMGR_ROOT}/${CERTMGR_VERSION}/sbin/certmgr

Building From Sources

You can of course build the Cloudflare’s certmgr from sources: this way is useful when there is no pre-built binary for the architecture you are using, or if you are customizing it.

Since the PKI Toolkit is written in Golang, you need to download it (mind here we need to work as the “root” user) – first decide on the Golang version you want to use, for example:

GO_VERSION=20.3

then download and install Golang using the following statement:

 wget -qO- https://go.dev/dl/go1.${GO_VERSION}.linux-arm64.tar.gz | tar xvz -C /usr/local

since it is also needed git, install it as follows:

dnf install -y git

Since the best practice is not to use the root user when building and packaging software, now we must temporarily switch to the "cloudflare" user:

sudo -u cloudflare bash

Before going on, add "/usr/local/go/bin" to the PATH environment variable in the "~/.bashrc" file, then source it or disconnect the session and become the "cloudflare" user again.

We are ready to download and build certmgr - just type:

cd ~
go install github.com/cloudflare/certmgr/cmd/...@latest

guess the version we have just built:

~/go/bin/certmgr version

in this example I got version is 1.6.5 (that is a very old one by the way), ... for the sake of completeness, let's see how to install it system:

CERTMGR_VERSION=1.6.5
mkdir -p ${CERTMGR_ROOT}/${CERTMGR_VERSION}/sbin
# uncomment only if you really want to use this old version
#[ -L /opt/certmgr/current ] && rm -f /opt/certmgr/current
#ln -s ${CERTMGR_ROOT}/${CERTMGR_VERSION} ${CERTMGR_ROOT}/current
cp /home/cloudflare/go/bin/certmgr ${CERTMGR_ROOT}/${CERTMGR_VERSION}/sbin

Basic Settings

Configuring certmgr is quick and easy - create the certmgr’s configuration directory tree:

 mkdir -m 755 /etc/certmgr/conf.d

then setup the certmgr’s defaults file "/etc/certmgr/certmgr.yml":

dir: /etc/certmgr/conf.d
default_remote: https://ca01.lab.carcano.corp:9800
svcmgr: systemd
before: 72h
interval: 720m
metrics_port: 8080
metrics_address: 10.21.34.52
take_actions_only_if_running: true

this file provides reasonable defaults to certmgr so to avoid explicitly setting them in each certificate configuration file.
The above file sets the "/etc/certmgr/conf.d" as the directory where to look for certificate configuration files and assigns the following defaults.

CFSSL Multi CA Registration Authority endpoint: "https://ca01.lab.carcano.corp:9800"
Process that manages the monitored services: systemd
Certificate renewal threshold: 72 hours before expiration
Checking interval: every 720 minutes (12 hours)
Port for exporting Prometheus metrics that can be collected by: 8080
Interface to bind the metrics endpoint to: 10.21.34.52 - the IP address of the server we installed certmgr on to
Take actions (such as renewals and restart of the service) on a monitored certificate only if the
monitored service is running.

It is then necessary to set up the certmgr Systemd unit – create the "/etc/systemd/system/certmgr.service" file with the following contents:

[Unit]
Description=Cloudflares' CertMgr

[Service]
Type=simple
Restart=on-failure
RestartSec=5
User=root
Group=root
ExecStart=/opt/certmgr/current/sbin/certmgr -f /etc/certmgr/certmgr.yml

[Install]
WantedBy=multi-user.target

And reload Systemd as usual:

systemctl daemon-reload

Is certificate automation just about configuring local daemons? Successfully compiling certmgr from sources, creating isolated service users, and writing JSON manifests to monitor expiration thresholds is a fundamental operational skill.

However, deploying and managing these configuration blocks manually server-by-server becomes an engineering bottleneck in enterprise environments. True production standards require you to translate these local patterns into repeatable Ansible architectures and secure, centralized infrastructure delivery.

If you want to evaluate whether your DevOps and DevSecOps foundations are genuinely scalable or if missing automation frameworks are limiting your career growth, jump directly to the Apress Blueprint Box below to discover how to boost and evolve your career using a self-paced learning path.

Monitoring A Certificate

We are finally ready to have the first certificate monitored by certmgr. We just need to add the manifest file describing the certificate to monitor beneath the "/etc/certmgr/conf.d" directory : configure the "/etc/certmgr/conf.d/ca01.lab.carcano.corp.json" file as follows:

cat << EOF > /etc/certmgr/conf.d/ca01.lab.carcano.corp.json
{
    "service": "cfssl-mra@tier01",
    "action": "restart",
    "request": {
        "CN": "ca01.lab.carcano.corp",
        "key": {
            "algo": "ecdsa",
            "size": 256
        },
        "hosts": [
            "ca01.lab.carcano.corp",
            "10.21.34.52"
        ],
"names": [ {
                "C": "CH",
                "ST": "Tessin",
                "O": "Carcano SA"
} ]
    },
    "private_key": {
        "path": "/etc/cfssl/tier01/ra/keys/key.pem",
        "owner": "root",
        "group": "cloudflare",
        "mode": "0640"
    },
    "certificate": {
        "path": "/etc/cfssl/tier01/ra/certs/crt.pem",
        "owner": "root",
        "group": "root",
        "mode": "0644",
        "key_usages": [
            "signing",
            "key encipherment",
            "server auth"
        ]
    },
    "authority": {
        "auth_key": "${AUTH_KEY}",
        "label": "carc_r100l",
        "profile": "server",
        "root_ca": "/etc/cfssl/tier01/carc_r100l/certs/ca.crt"
} }
EOF

Are you enjoying these high quality free contents on a blog without annoying banners? I like doing this for free, but I also have costs so, if you like these contents and you want to help keeping this website free as it is now, please put your tip in the cup below:

Even a small contribution is always welcome!

The above certificate and key manifest require certmgr to perform the following tasks:

Make sure the following files exist:
- the ecdsa256 "/etc/cfssl/tier01/ra/keys/key.pem" key file, owned by the "root" user and "cloudflare" group, with Posix permission 0640
- the "/etc/cfssl/tier01/ra/certs/crt.pem" x509 certificate file, owned by the "root" user and "root" group, with Posix permission 0644
Make sure the certificate matches the following criteria:
- CN is "ca01.lab.carcano.corp"
- Subject String is "C=CH , ST=Tessin, O=Carcano SA"
- Subject Alternative Names are:
  - ca01.lab.carcano.corp
  - 10.21.34.52
- key usages are "signing", "key encipherment" and "server auth"
Make sure the certificate is not expired, using the value of the "before" attribute of the file "/etc/certmgr/certmgr.yml" as the expiration threshold.

If the above conditions are not met, certmgr generates a key and submits the Certificate Signing Request to the multi-Certificate Authority endpoint set as "default_remote" in the "/etc/certmgr/certmgr.yml" file ("https://ca01.lab.carcano.corp:9800"), selecting the "carc_r100l" CA, using the "server" profile.

The TLS certificate of the multi-Certificate Authority endpoint is validated using the "/etc/cfssl/tier01/carc_r100l/certs/ca.crt" trust-store file.

Now that we have at least one monitored certificate we can start the certmgr service and configure it to load at system boot:

systemctl enable --now certmgr

If everything is properly set, you must find entries such as the following ones in the "/var/log/messages" file:

Aug 2 07:50:05 localhost certmgr[5383]: 2023/08/02 07:50:05 [INFO] manager: processing certificate spec /CARC R100L CRL/C=CH/O=Carcano SA/ST=Tessin (attempt 1)
Aug 2 07:50:05 localhost certmgr[5383]: 2023/08/02 07:50:05 [INFO] encoded CSR
Aug 2 07:50:05 localhost certmgr[5383]: 2023/08/02 07:50:05 [INFO] manager: executing configured action due to change type key for /etc/cfssl/tier01/ra/certs/crt.pem
Aug  2 07:50:05 localhost certmgr[5383]: 2023/08/02 07:50:05 [INFO] restarting service cfssl-ra@carc_r100l
Aug  2 07:50:05 localhost certmgr[5383]: 2023/08/02 07:50:05 [INFO] manager: certificate successfully processed

And of course, the "/etc/cfssl/tier01/ra/keys/key.pem" and "/etc/cfssl/tier01/ra/certs/crt.pem" files are automatically created.

Please note how the service “cfssl-ra@carc_r100l” related to the monitored certificate was automatically restarted.

The entire set of available settings can be found on the README.md document in the certmgr git repo on GitHub - https://github.com/cloudflare/certmgr.

Beyond Local Automation: Building the DevOps and DevSecOps Foundations

Successfully orchestrating Cloudflare's certmgr service, masking paths with secure POSIX permissions, and validating endpoint TLS streams via automated hooks is an excellent technical achievement.

Yet, automating certificates on a standalone host represents just one single brick in the massive wall of modern DevOps and DevSecOps engineering on Linux. In enterprise-grade infrastructure, an expert must move past isolated runtime tools: you need to seamlessly connect certificate lifecycles with advanced kernel hardening, robust shell scripting architectures, strict version control, and automated software supply chains without vendor lock-in.

When you look closely at your daily deployment tasks, are you completely confident you master the holistic engineering layer running under the hood of the entire stack? Or are you simply connecting separate open-source tools without a comprehensive architectural strategy?

My book, "DevSecOps and DevOps for Linux: The Foundations", published by Apress, was written specifically to provoke this professional reflection. It is a comprehensive, lab-driven blueprint tailored for both students and working engineers to systematically identify and fill their knowledge gaps. Through intensive, hands-on enterprise exercises built entirely on open-source, cloud-agnostic architectures, you will develop a solid theoretical grasp backed by deep practical engineering.

Key insights covered in this volume:

The Holistic Skills Set Brick: Bridge technical engineering with team management frameworks. Master Scrum, Kanban, and Lean methodologies to design system architectures aligned with real corporate workflows.
The Shell Scripting & Unix Tools Brick: Build rigorous operational foundations. Master advanced Bash shell scripting architecture while learning how to combine core Unix tools into robust, repeatable, and enterprise-ready host automations.
The Version Control Engineering Brick: Move past basic commits. Dive deep into Git version control, mastering feature-branch workflows, repository lifecycle management, and complex conflict resolution.
The Data & Core Automation Brick: Build bulletproof data processing setups. Learn advanced RegEx, how to operate using evergreen tools such as Grep, Sed, and AWK, and how to master structured data parsing (XML, JSON, YAML) using Python and tools like xmlstarlet, jq, and yq.
The Modern Python & Automation Brick: Develop a modern Python project using pyproject.toml with pytest-based unit tests, governing the project with GNU Make for testing, building, and digitally signing RPM packages. The project is presented in an evolving fashion, showing how features are added step by step, highlighting how a properly structured Python project can be improved and evolved with minimal or no rework at all.
The Linux OS Hardening & PKI Brick: Learn the real mechanics of security. Implement X.509/PKI architectures, TLS configurations, and GPG encryption and signing, while mastering low-level kernel defenses like SELinux and Linux Capabilities.
The Compliance Check and Shift-Left Security Brick: Learn how to leverage the pre-commit framework to automate compliance checks with Pylint and Flake8, and perform security scans with Bandit and Safety, extending the security audit to the full software supply chain.
The Application Integration Brick: Master the foundational protocols used to securely interconnect enterprise microservices, including HTTP, REST, OpenAPI, SOAP, and LDAP/LDAPS.
The Infrastructure Delivery Brick: Put theory into practice with vertical, real-world labs. Move from basic scripts to engineering Ansible architectures, rootless Podman setups, image creation via Buildah, and complete Pulp3 deployments using Docker Compose.
The Enterprise GitOps Pipeline Brick: Tie everything together by automating your software supply chain. Build complete continuous deployment workflows using Gitea CI pipelines hosted natively on Kubernetes (RKE2).

Footnotes

Here it ends this post dedicated to Cloudflare's certmgr - as you see it is a really handy and easy to set up tool. The only requirement to run it is of course running a Cloudflare's PKI and TLS toolkit based PKI: if you want to quickly and easily learn how to set it up and run it, I hope you can enjoy my book "A Full Featured PKI With Cloudflare's PKI and TLS Toolkit" available both as ebook and paperback.

I hate blogs with pop-ups, ads and all the (even worse) other stuff that distracts from the topics you're reading and violates your privacy. I want to offer my readers the best experience possible for free, ... but please be wary that for me it's not really free: on top of the raw costs of running the blog, I usually spend on average 50-60 hours writing each post. I offer all this for free because I think it's nice to help people, but if you think something in this blog has helped you professionally and you want to give concrete support, your contribution is very much appreciated: you can just use the above button.

Cloudflare’s certmgr tutorial – a certmgr howto