This is the last post of the trilogy dedicated to how to set up a well structured Python project, developed with professional style, suitable to be used within the context of a Continuous Integration toolchain. This time we focus on how to package all we have done so far as RPM packages, showing how to break down everything into subpackages that also perform post installation tasks.
If you haven't read the previous two posts, you must do it right now since they are requisite to understand this post. In addition to that this post relies onto objects that are being created in the previous posts.

Read them in the following order:

  1. Python Full Featured Project
  2. Python Setup Tools
The operating environment used in this post is Red Hat Enterprise Linux 8 or CentOS 8 - using a different environment may lead to having to adapt or even change things.

Packet Management With RPM

The Redhat Package Manager (RPM) is an open packaging system developed by Red Hat Linux many, many years ago. Besides being used by Red Hat based or derived Linux distributions, it is used also by other distributions such as Suse.

RPM stores information of the installed packages within a system-wide database (the RPM db) where a lot of metadata such as software version and dependencies are collected.
The database can be managed and queried using the rpm command line utility: it can be run by any user, but only the "root" user has write access to the RPM database files.

This means that any user can query the RPM issuing commands like this:

rpm -qi openssh-server

output is as follows:

Name : openssh-server
Version : 8.0p1
Release : 6.el8_4.2
Architecture: x86_64
Install Date: Thu 24 Jun 2021 06:31:12 PM UTC
Group : System Environment/Daemons
Size : 1034496
License : BSD
Signature : RSA/SHA256, Fri 11 Jun 2021 12:29:28 AM UTC, Key ID 15af5dac6d745a60
Source RPM : openssh-8.0p1-6.el8_4.2.src.rpm
Build Date : Fri 11 Jun 2021 12:16:02 AM UTC
Build Host : ord1-prod-x86build003.svc.aws.rockylinux.org
Relocations : (not relocatable)
Packager : infrastructure@rockylinux.org
Vendor : Rocky
URL : http://www.openssh.com/portable.html
Summary : An open source SSH server daemon
Description :
OpenSSH is a free version of SSH (Secure SHell), a program for logging
into and executing commands on a remote machine. This package contains
the secure shell daemon (sshd). The sshd daemon allows SSH clients to
securely connect to your SSH server.

these are the information about an installed RPM package called "openssh-server": we requested to display this information by specifying that we want to query the RPM database ("-q" command switch) about information ("-i" command switch) on the already installed RPM package called "openssh-server".

It is straightforward that only root can update the RPM db:  the RPM db is updated only when packages get installed/updated/removed; root is the only user that is granted the right to write into system directory trees aimed at store software packages. You can of course grant the right to use installation commands to other users leveraging on sudo by configuring sudo rules.

The Redhat Package Manager installs packages of RPM format: this kind of package is a sophisticated archive that does not only pack a set of files and directories within a package file, but can also run scripts and evaluate conditionals.
It is made by putting a header structure on top of a CPIO archive.

The package itself is has four sections:

  • the file identifier (magic number) used to identify the file contents as an RPM package
  • the signature to verify the integrity of the package
  • the header or "tagged" data containing package information, version numbers and copyright
  • the payload, that is the actual CPIO archive containing the program files.

let's download the "epel-release" RPM package to give it a closer look:

yumdownloader epel-release

after a while, "epel-release-8-13.el8.noarch.rpm" RPM package gets downloaded to our system.

Since files provided by a RPM package are packaged within a CPIO archive, they can easily listed also without using the rpm command line utility itself as follows:

rpm2cpio epel-release-8-13.el8.noarch.rpm | cpio -i --list

output is as follows:

./etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-8
./etc/yum.repos.d/epel-modular.repo
./etc/yum.repos.d/epel-playground.repo
./etc/yum.repos.d/epel-testing-modular.repo
./etc/yum.repos.d/epel-testing.repo
./etc/yum.repos.d/epel.repo
./usr/lib/systemd/system-preset/90-epel.preset
./usr/share/doc/epel-release
./usr/share/doc/epel-release/README-epel-8-packaging.md
./usr/share/licenses/epel-release
./usr/share/licenses/epel-release/GPL
74 blocks

RPM provides several benefits - an interesting one is showing information about what a package provides even before we install it:

rpm -q --provides epel-release-8-13.el8.noarch.rpm

output is as follows:

config(epel-release) = 8-13.el8
epel-release = 8-13.el8

please note how this time we queried a package that is not installed yet .

RPM can be queried also to know what a package requires:

rpm -q --requires epel-release-8-13.el8.noarch.rpm

output is as follows:

config(epel-release) = 8-13.el8
redhat-release >= 8
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(PayloadIsXz) <= 5.2-1

RPM can be queried also to know if a package does execute at install or uninstall anything and the statements that are run at that time.

For example, we can query the already installed package "openssh-server" as follows:

rpm -q --scripts openssh-server

the output is as follows:

preinstall scriptlet (using /bin/sh):
getent group sshd >/dev/null || groupadd -g 74 -r sshd || :
getent passwd sshd >/dev/null || \
  useradd -c "Privilege-separated SSH" -u 74 -g sshd \
  -s /sbin/nologin -r -d /var/empty/sshd sshd 2> /dev/null || :
postinstall scriptlet (using /bin/sh):

if [ $1 -eq 1 ] ; then 
        # Initial installation 
        systemctl --no-reload preset sshd.service sshd.socket &>/dev/null || : 
fi
preuninstall scriptlet (using /bin/sh):

if [ $1 -eq 0 ] ; then 
        # Package removal, not upgrade 
        systemctl --no-reload disable --now sshd.service sshd.socket &>/dev/null || : 
fi
postuninstall scriptlet (using /bin/sh):

if [ $1 -ge 1 ] ; then 
        # Package upgrade, not uninstall 
        systemctl try-restart sshd.service &>/dev/null || : 
fi
  • preinstall scriptlet is run before installing the RPM package contents
  • postinstall scriptlet is run after installing the RPM package contents
  • preuninstall scriptlet is run before uninstalling the RPM package
  • postuninstall scriptlet is run after uninstalling the RPM package

These are of course only a few of the many benefits provided by RPM, so to let you get the gist.

Packaging as RPM provides a lot of benefits: that's why being skilled on it is valuable not only for developers but also for system engineers.

Setup The RPM Development Environment

Install The Required Tools

RPMs are built by having the rpmbuild command line utility: it is provided by the rpmbuild RPM package. We can install it along with other handy tools such as "rpmdevtools" and "rpmlint" as follows:

sudo dnf install -y rpm-build rpmdevtools rpmlint

Generate the RPM Development Tree

The building of RPM packages happens within the context of the so called RPM development tree: this directory tree is on a per user basis and can be easily generated as follows:

rpmdev-setuptree

it creates the following directory tree:

~/rpmbuild/
├── SOURCES
├── SPECS
├── BUILD
├── RPMS
└── SRPMS

where:

  • SOURCES is the directory where archives containing the source files must be put. Archives can be tar archives, either compressed or not.
  • SPECS is the directory that contains the ".spec" files: these are the files that contain all the necessary statements that describe the RPM package.
  • BUILD is the directory used during the build process, where temporary files are stored, created, moved etc.
  • RPMS is directory that holds the built RPM packages
  • SRPMS is the directory that holds the ".src.rpm" packages: these are RPM packages that provide the archive containing the source files along with the SPEC file, so providing everything is needed to rebuild the RPM package.
Never and ever develop and/or build RPM packages as the root user.
It is not really necessary to generate the RPM development tree in the use case described in this post: I'm showing you this only for the sake of completeness and so to have the chance to describe th purpose of each direcotry.

Packaging as RPM

Now that we know at least the basics of RPM, we can see how to generate an RPM package that ships the Python package we created in the previous two posts.

Generate a stub SPEC file

First and foremost let's get back to the Python3 project we developed in the previous two posts - change directory to the root of the Carcano's "foolist" project:

cd ~/fooproject

and create the "RPM" and "RPM/SPECS" directories:

mkdir RPM
mkdir RPM/SPECS
cd RPM/SPECS

RPM are built by having the rpmbuild command line utility reading the SPEC file: we can easily generate a stub SPEC file as follows:

rpmdev-newspec carcano_foolist

the outcome is the "carcano_foolist.spec" file beneath "RPM/SPECS" directories: we use this stub and complete it with the missing settings that are necessary to package "carcano_foolist" as RPM.

Configure the SPEC file

We begin by editing "RPM/SPECS/carcano_foolist.spec" file by defining a few RPM macros at the very first two lines of the file:

%{!?_version: %define _version 0.0.1 }
%global srcname carcano_foolist
  • the first line is about the "_version" macro, used to set the version of the built RPM packages: it can be explicitly set  using the "--define" command line switch of rpmbuild, avoiding having to hard-code it within the SPEC file. The first line checks if  macro is defined, and if it isn't defines it with "0.0.1" as default value.
  • the second line defines the "srcname" macro with "carcano_foolist" as a value at the global scope.
Despite the character to be used to comment-out lines used in SPEC files is '#', it does not work like so with macros - to comment-out a macro you must prepend an additional '%'. So for example, to comment on the second of the above lines, it must look like "%%global srcname carcano_foolist".

Define the Package

We can go on filling-in the stub describing the package by completing the package information tags:

Name:           python-%{srcname}
Version:        %{_version} 
Release:        1%{?dist}
Summary:        Full Features Python3 Sample Project
License:        LGPLv3+
Source0:        %{pypi_source}

we can safely delete the URL package information tag since in this example we do not make use of it. This is the meaning of each of the above tags:

  • Name: the name of the software being packaged - it must not contain spaces
  • Version: the version of the software being packaged: it should be as close as possible to the format of the original software's version, although there is the constraint of avoiding o dashes
  • Release: the minor release; note that it is up to the package builder to determine which build represents a new release and to update the release manually. Note that also here there is the constraint of avoiding o dashes
  • Summary: a one-line description of the packaged software
  • License: the license terms applicable to the software being package 
  • Source0: the Source tag has the purpose to both show where the software's developer has made the original sources available and provide the name of the original source file. This means that it can be either filename or URL to locate the archive containing the sources. We can specify as many sources as we need simply by incrementing the number at the end of the "Source" word.

"%{pypi_source}" is a RPM macro that can be used when having to deal with sources from PyPI to generate the proper URL: we can evaluate the macro so to see how it gets expanded by simply specifying the "--eval" command switch of the rpm command line utility:

rpm --eval "%pypi_source carcano_foolist 0.0.9-py3-none-any whl"

the output is as follows:

https://files.pythonhosted.org/packages/source/c/carcano_foolist/carcano_foolist-0.0.9-py3-none-any.whl"

RPM macros are snippets of "lua" language code: you can get the dump of all of them, along with their lua code, by simply specifying the "--showrc" command switch of the rpm command line utility as follows:

rpm --showrc

if you want to focus on the lua code of a certain macro, you have only to pipe it to sed - for example, to see the source code of "%{pypi_source}" macro we previously saw:

rpm --showrc | sed -n -e '/pypi_source/,/}/ p'

the output is:

-13: pypi_source	%{lua:
    local src = rpm.expand('%1')
    local ver = rpm.expand('%2')
    local ext = rpm.expand('%3')
    local url = rpm.expand('%__pypi_url')

    -- If no first argument, try %srcname, then %pypi_name, then %name
    -- Note that rpm leaves macros unchanged if they are not defined.
    if src == '%1' then
        src = rpm.expand('%srcname')
    end
    if src == '%srcname' then
        src = rpm.expand('%pypi_name')
    end
    if src == '%pypi_name' then
        src = rpm.expand('%name')
    end

    -- If no second argument, use %version
    if ver == '%2' then
        ver = rpm.expand('%version'):gsub('~', '')
    end

    -- If no third argument, use the preset default extension
    if ext == '%3' then
        ext = rpm.expand('%__pypi_default_extension')
    end

    local first = string.sub(src, 1, 1)

    print(url .. first .. '/' .. src .. '/' .. src .. '-' .. ver .. '.' .. ext)
}

From the code we can guess that if we want to use a different URL we have to set it by defining the "%__pypi_url" macro before using "%{pypi_source}" macro.

Now that we have a better understanding of RPM macros, we can go on defining / completing the following package information tags:

BuildArch:       noarch
BuildRequires:   python3-devel python3-setuptools
Requires:        python3

this is the meaning of each of the above package information tags:

  • BuildArch: the architecture the software is targeted to
  • BuildRequires: the RPM packages that are required to build this RPM
  • Requires: the packages that are required to run the software packaged within this RPM - they can be either RPM package or virtual packages provided by one or more RPM packages that use the provided tag. Note that version comparisons may also be included by following the package name with <, >, =, >=, or <=, and a version specification

then we complete the description tag as follows - note how this tag is multi-line so to let us provide a thorough description:

%description
Full Featured Python3 Sample Project

Define the SubPackages

What we have done so far has the sole purpose of defining the tags for the base RPM package: in this project we create two different RPM sub-packages.

Package information tags that are within a "%package" section are considered in the scope of the subpackage specified by the -n argument of the "%package" section.

Our first subpackage is "python3-carcano_foolist-common": it's purpose is packaging only the modules delivered by the "carcano_foolist" Python package itself :

%package -n python3-%{srcname}-common
Summary:        %{summary}
BuildRequires:  python3-devel
%{?python_provide:%python_provide python3-%{srcname}-common}

%description -n python3-%{srcname}-common
Python3 packages of the sample project

the lines to explain here are:

  • the 1st is used do declare the subpackage using the "%package" information tag  (note the "-n" option)
  • the 4th is used to automatically guess the provided Python packages
  • the 6th is used to provide the description of the subpackage using the "%description" information tag with the "-n" option followed by the name of the subpackage

The second and last subpackage is "python3-carcano_foolist" - let's define it as follows:

%package -n python3-%{srcname}
Summary:        %{summary}
BuildRequires:  python3-devel
Requires:       python3-%{srcname}-common
%{?python_provide:%python_provide python3-%{srcname}}

%description -n python3-%{srcname}
Python3 scripts and resources of the sample project

This ends the part related to the definition of the package information tags.

Prepare the Build (the %prep Stage)

We are getting closer to the more interesting parts - let's complete the %prep section as follows:

%prep
%autosetup -n %{srcname}-%{_version}

this section is executed as the %prep stage of the RPM building process. It runs the "%autosetup" macro specifying the name of the directory where to extract the contents of the package into ("-n" option).

Build the Package (the %build Stage)

We are now ready for the actual build process: it is defined in the %build section - replace the default contents of the stub  and complete it as follows:

%build
unset RPM_BUILD_ROOT
%{__python3} setup.py bdist_wheel

this section is executed as the %build stage of the RPM building process.

The official Red Hat and Fedora documentation recommend using the %py3_build family macros (py3_build, %py3_build_wheel, ...). In my lab I've not been able to comply with the recommendation, since I got an error from setuptools complaining that it cannot import the name 'find_namespace_packages'. After inspecting the failing temporary file generated by the expansion of the macro, I found that things broke after setting the RPM_BUILD_ROOT environment variable. For this reason my workaround is to unset the RPM_BUILD_ROOT environment variable and explicitly run the build command.

Install the Package into a different root directory tree (the %install Stage)

We are eventually at the install process: it is defined in the %install section - replace the default contents and complete it as follows:

The purpose of the install process is to install the files into a root directory tree different from the actual root directory - it runs during the %install stage:

%install
[ "%{buildroot}" != "/" ] && rm -rf %{buildroot}
mkdir %{buildroot}
mkdir %{buildroot}/usr
cd "%{_builddir}/%{srcname}-%{_version}/dist"
%{__python3} -m pip install --target %{buildroot}%{python3_sitelib} %{srcname}-%{_version}-py3-none-any.whl 
mkdir %{buildroot}/%{_sysconfdir}
mkdir %{buildroot}/%{_sysconfdir}/fooapp
mkdir %{buildroot}/%{_sysconfdir}/rsyslog.d
mkdir %{buildroot}/usr/bin
cp %{_builddir}/%{srcname}-%{_version}/bin/logging.conf %{buildroot}/%{_sysconfdir}/fooapp/logging.conf
cp %{_builddir}/%{srcname}-%{_version}/share/doc/fooapp/rsyslog/fooapp.conf %{buildroot}/%{_sysconfdir}/rsyslog.d/fooapp.conf
cp %{_builddir}/%{srcname}-%{_version}/bin/fooapp.py %{buildroot}/usr/bin/fooapp.py
Also here official recommendation from Red Hat and Fedora documentation to use the %py3_build family macros  (py3_install, %py3_install_wheel, ...) failed on my lab: same symptom as before, fixed by  unset the RPM_BUILD_ROOT environment variable and explicitly run the installation commands.

Run Unit Tests (the %check stage)

As he tbest practice requires, the contents of the Python package must succeed the unit tests before going on and packaging it as RPM. The section aimed at this is the %check section, so let's create it as follows:

%check
cd "%{_builddir}/%{srcname}-%{_version}"
unset RPM_BUILD_ROOT
%{__python3} setup.py nosetests >/dev/null

it runs during the %check stage.

List The Files To Be Packed Into Each Of The SubPackages

Now that the Python package has been prepared extracting its contents, built, installed into a chrootd tree and tests it has eventually come the time to pack the outcome into the RPM packages: the purpose of the %files sections is exactly to list the files that must be put into each of the packages.
This is the snippet of the %files section that specifies which files must be put inside the python3-carcano_foolist-common RPM package:

%files -n python3-%{srcname}-common
%{python3_sitelib}/carcano/foolist/__init__.py
%{python3_sitelib}/carcano/foolist/__pycache__/__init__.cpython-36.opt-1.pyc
%{python3_sitelib}/carcano/foolist/__pycache__/__init__.cpython-36.pyc
%{python3_sitelib}/carcano/foolist/__pycache__/foolist.cpython-36.opt-1.pyc
%{python3_sitelib}/carcano/foolist/__pycache__/foolist.cpython-36.pyc
%{python3_sitelib}/carcano/foolist/__pycache__/foolistitem.cpython-36.opt-1.pyc
%{python3_sitelib}/carcano/foolist/__pycache__/foolistitem.cpython-36.pyc
%{python3_sitelib}/carcano/foolist/foolist.py
%{python3_sitelib}/carcano/foolist/foolistitem.py
%{python3_sitelib}/carcano_foolist-%{_version}.dist-info/INSTALLER
%{python3_sitelib}/carcano_foolist-%{_version}.dist-info/METADATA
%{python3_sitelib}/carcano_foolist-%{_version}.dist-info/RECORD
%{python3_sitelib}/carcano_foolist-%{_version}.dist-info/WHEEL
%{python3_sitelib}/carcano_foolist-%{_version}.dist-info/top_level.txt

This is the snippet of the %files section that specifies which files must be put inside the python3-carcano_foolist RPM package:

%files -n python3-%{srcname}
%config  %{_sysconfdir}/fooapp/logging.conf
%config  %{_sysconfdir}/rsyslog.d/fooapp.conf
/usr/bin/fooapp.py

Add Post-Installation statements

The RPM package python3-carcano_foolist requires also to execute a few post installation statements: since it provides an additional configuration file to rsyslog, it also requires reloading rsyslog to apply the new configuration.

Post-installation tasks are defined within the %post section, where as tasks that must run after an uninstall should be put into %postun section:

%post -n python3-%{srcname}
systemctl restart rsyslog

%postun -n python3-%{srcname}
systemctl restart rsyslog

of course we must scope the %post and %postun sections so as to refer them only to the "python3-carcano_foolist" RPM package.

Add the Changelog

The %changelog section is used to provide details about the evolution of the packaged software across the releases.

%changelog
* Mon Jun 14 2021 Marco Antonio Carcano <me@mydomain.tld>
First release

Of course it is mandatory to fill-in it, but do not underestimate it: you must be tidy, since it can provide valuable information to who is installing the RPM package.

The whole SPEC file

For your convenience, this is the full listing of "RPM/SPECS/carcano_foolist.spec" file:

%{!?_version: %define _version 0.0.1 }
%global srcname carcano_foolist

Name:           python-%{srcname}
Version:        %{_version} 
Release:        1%{?dist}
Summary:        Full Features Python3 Sample Project
License:        LGPLv3+
Source0:        %{pypi_source}

BuildArch:       noarch
BuildRequires:   python3-devel python3-setuptools
Requires:        python3

%description
Full Featured Python3 Sample Project

%package -n python3-%{srcname}-common
Summary:        %{summary}
BuildRequires:  python3-devel
%{?python_provide:%python_provide python3-%{srcname}-common}

%description -n python3-%{srcname}-common
Python3 packages of the sample project

%package -n python3-%{srcname}
Summary:        %{summary}
BuildRequires:  python3-devel
Requires:       python3-%{srcname}-common
%{?python_provide:%python_provide python3-%{srcname}}

%description -n python3-%{srcname}
Python3 scripts and resources of the sample project

%prep
%autosetup -n %{srcname}-%{_version}

%build
unset RPM_BUILD_ROOT
%{__python3} setup.py bdist_wheel

%install
[ "%{buildroot}" != "/" ] && rm -rf %{buildroot}
mkdir %{buildroot}
mkdir %{buildroot}/usr
cd "%{_builddir}/%{srcname}-%{_version}/dist"
%{__python3} -m pip install --target %{buildroot}%{python3_sitelib} %{srcname}-%{_version}-py3-none-any.whl 
mkdir %{buildroot}/%{_sysconfdir}
mkdir %{buildroot}/%{_sysconfdir}/fooapp
mkdir %{buildroot}/%{_sysconfdir}/rsyslog.d
mkdir %{buildroot}/usr/bin
cp %{_builddir}/%{srcname}-%{_version}/bin/logging.conf %{buildroot}/%{_sysconfdir}/fooapp/logging.conf
cp %{_builddir}/%{srcname}-%{_version}/share/doc/fooapp/rsyslog/fooapp.conf %{buildroot}/%{_sysconfdir}/rsyslog.d/fooapp.conf
cp %{_builddir}/%{srcname}-%{_version}/bin/fooapp.py %{buildroot}/usr/bin/fooapp.py

%check
cd "%{_builddir}/%{srcname}-%{_version}"
unset RPM_BUILD_ROOT
%{__python3} setup.py nosetests >/dev/null

%files -n python3-%{srcname}-common
%{python3_sitelib}/carcano/foolist/__init__.py
%{python3_sitelib}/carcano/foolist/__pycache__/__init__.cpython-36.opt-1.pyc
%{python3_sitelib}/carcano/foolist/__pycache__/__init__.cpython-36.pyc
%{python3_sitelib}/carcano/foolist/__pycache__/foolist.cpython-36.opt-1.pyc
%{python3_sitelib}/carcano/foolist/__pycache__/foolist.cpython-36.pyc
%{python3_sitelib}/carcano/foolist/__pycache__/foolistitem.cpython-36.opt-1.pyc
%{python3_sitelib}/carcano/foolist/__pycache__/foolistitem.cpython-36.pyc
%{python3_sitelib}/carcano/foolist/foolist.py
%{python3_sitelib}/carcano/foolist/foolistitem.py
%{python3_sitelib}/carcano_foolist-%{_version}.dist-info/INSTALLER
%{python3_sitelib}/carcano_foolist-%{_version}.dist-info/METADATA
%{python3_sitelib}/carcano_foolist-%{_version}.dist-info/RECORD
%{python3_sitelib}/carcano_foolist-%{_version}.dist-info/WHEEL
%{python3_sitelib}/carcano_foolist-%{_version}.dist-info/top_level.txt

%files -n python3-%{srcname}
%config  %{_sysconfdir}/fooapp/logging.conf
%config  %{_sysconfdir}/rsyslog.d/fooapp.conf
/usr/bin/fooapp.py

%post -n python3-%{srcname}
systemctl restart rsyslog

%postun -n python3-%{srcname}
systemctl restart rsyslog

%changelog
* Mon Jun 14 2021 Marco Antonio Carcano <me@mydomain.tld>
First release

since we are using nose to perform unit-tests, we must ensure that it is installed:

sudo dnf install -y python3-nose

in addition to that, let's ensure we have an as current as possible version of wheel and setuptools:

sudo python3 -m pip install -U wheel setuptools

Add the "rpm" target to the Makefile

As we did in the previous post, since we love to work within the context of a Continuous Integration toolchain, let's modify the Makefile we created so far by adding the "rpm" target as follows:

rpm: sdist
	$(info -> Makefile: packaging as RPM ...)
	[ -d ~/rpmbuild ] || mkdir ~/rpmbuild
	[ -d ~/rpmbuild/SOURCES ] || mkdir ~/rpmbuild/SOURCES
	[ -d ~/rpmbuild/SPECS ] || mkdir ~/rpmbuild/SPECS
	mv src/dist/carcano_foolist-${RELEASE}.tar.gz ~/rpmbuild/SOURCES
	cp RPM/SPECS/carcano_foolist.spec ~/rpmbuild/SPECS
	cd ~/rpmbuild/SPECS; rpmbuild --define "_version ${RELEASE}" -ba carcano_foolist.spec

we can now build everything as a whole as follows:

export RELEASE="0.0.2"
make rpm
Take in account that in the SPEC file we specified "BuildRequires python3-devel", ... this means that you have to install it before being able to build these RPMs.

if everything has properly been setup, the output is as follows:

-> Makefile: validating RELEASE=0.0.2 format
-> Makefile: cleanup previous builds ... 
-> Makefile: building the sdist distribution package ...
running sdist
running egg_info
creating carcano_foolist.egg-info
writing carcano_foolist.egg-info/PKG-INFO
...
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/grimoire/rpmbuild/BUILDROOT/python-carcano_foolist-0.0.2-1.el8.x86_64
Wrote: /home/grimoire/rpmbuild/SRPMS/python-carcano_foolist-0.0.2-1.el8.src.rpm
Wrote: /home/grimoire/rpmbuild/RPMS/noarch/python3-carcano_foolist-common-0.0.2-1.el8.noarch.rpm
Wrote: /home/grimoire/rpmbuild/RPMS/noarch/python3-carcano_foolist-0.0.2-1.el8.noarch.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.0G9mQa
+ umask 022
+ cd /home/grimoire/rpmbuild/BUILD
+ cd carcano_foolist-0.0.2
+ /usr/bin/rm -rf /home/grimoire/rpmbuild/BUILDROOT/python-carcano_foolist-0.0.2-1.el8.x86_64
+ exit 0

We now have a complete project we can use into a continuous improvement loop: this means that we can go-on with the regular development cycle, fixing errors and adding features when needed and rebuilding things.

Digitally sign the RPM package

Before publishing an RPM package into a repository from where it can be downloaded, we must digitally sign it: by doing so, tools such as yum, dnf, zipper and such can verify the signature by using the public key that we publish.

The default behaviour of yum and dnf is cancel the installation process of unsigned packages: despite you can reconfigure them to skip this check, or use an option to make them temporarily skip it, it is not wise to install unsigned packages, ... for the same reason it is not wise to take candy from strangers.

Generate The GPG Key

Signing is made using the GNU Privacy Guard (GPG), so the very first thing is generating the GNU Privacy Guard (GPG) key pair used to sign the RPM packages.

These keys are used to sign every package built by the corporation: this means that the value of the "real name" must be something that identifies the whole corporation's development facilities. For this blog I'm using "The Grimoire Of A Modern Linux Professional" as my real name and "ci@grimoire.carcano.ch" as email, pretending this is the email address of the corporate's continuous integration toolchain.

Let's create the GPG key pair by entering the following command:

gpg --expert --pinentry-mode loopback --full-gen-key
Note the use of "--pinentry-mode loopback" command option: it is needed when working as user we have been been switched to by using sudo. I wanted to show it since the common practice is to first login via SSH using the corporate user account, and then switch to a service account (for example the one of continuous integration). In such a scenario, if you omit this option, several failures because of permission violations happens.

This is the transcript of my answer to the questions:

Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
   (7) DSA (set your own capabilities)
   (8) RSA (set your own capabilities)
   (9) ECC and ECC
  (10) ECC (sign only)
  (11) ECC (set your own capabilities)
  (13) Existing key
  (14) Existing key from card
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want for the subkey? (2048) 
Requested keysize is 2048 bits
Please specify how long the key should be valid.
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 10y
Key expires at Sat 25 Oct 2031 05:00:56 PM UTC
Is this correct? (y/N) y

GnuPG needs to construct a user ID to identify your key.

Real name: The Grimoire Of A Modern Linux Professional
Email address: ci@grimoire.carcano.ch
Comment: 
You selected this USER-ID:
    "The Grimoire Of A Modern Linux Professional <ci@grimoire.carcano.ch>"

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o

Let's confirm typing "o", and wait for everything to complete.

Let's now have a look at the generated keys:

gpg --list-keys

the output is as follows:

gpg: checking the trustdb
gpg: marginals needed: 3 completes needed: 1 trust model: pgp
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: next trustdb check due at 2031-10-25
/home/grimoire/.gnupg/pubring.kbx
---------------------------------
pub rsa2048 2021-10-27 [SC] [expires: 2031-10-25]
2A6018DE3B4282CE229A897E5BA662C0EBD6F747
uid [ultimate] The Grimoire Of A Modern Linux Professional <ci@grimoire.carcano.ch>
sub rsa2048 2021-10-27 [E] [expires: 2031-10-25]

The public key related to the private one used to sign must be published on a repository that can be accessed by everybody who needs to install RPM packages signed by us: otherwise they have no way to check the signature.

This mean that we must export the public key as follows:

gpg --export -a ci@grimoire.carcano.ch > ~/RPM-GPG-KEY-grimoire

and publish it on the repository.

If you want to know more about digital signature or simply want to quickly review cryptography related stuff, don't miss the post "Cryptography quick guide - understand symmetric and asymmetric cryptography and how to use it with OpenSSL".

Sign the RPM Package

The signing feature for the rpm command line utility is provided by the rpm-sign RPM package - let's install it as follows:

sudo dnf install -y rpm-sign

then we have to configure the RPM macros of the current user:

  • since you can have several keys within your GPG keyring, we must specify which GPG key must be used to sign RPMs: here we use "ci@grimoire.carcano.ch", the GPG key we just created
  • although this is optional, the GPG command we use to sign - here I'm doing this since I wanted to add the "--pinentry-mode loopback"
cat << \EOF >> ~/.rpmmacros
%_gpg_name <ci@grimoire.carcano.ch>
%__gpg_sign_cmd %{__gpg} gpg --force-v3-sigs --no-armor --pinentry-mode loopback --no-secmem-warning -u "%{_gpg_name}" -sbo %{__signature_filename} --digest-algo sha256 %{__plaintext_filename}'
EOF

now we have setup everything and we are eventually ready to sign the packages - the command is as follows:

rpm --addsign ~/rpmbuild/RPMS/noarch/python3-carcano_foolist*.rpm

after typing the password to unlock the secret key for the signature, the RPM packages get signed.

We can of course add a "sign" target to the Makefile:

sign:
	$(info -> Makefile: digitally signing the RPM packages...)
	rpm --addsign ~/rpmbuild/RPMS/noarch/python3-carcano_foolist*.rpm

if you are using a RPM repository such as Red Hat Network Satellite Server and you have a channel dedicated to the delivery of the software you develop, this target is also suitable for the purpose of pushing the signed RPM package to them too. That is: you sign your software and immediately publish it.

Just as an example, this is the command we'd use to push to Red Hat Network Satellite 6.x (or Katello):

hammer repository upload-content --organization Grimoire --product Carcano-Stuff --path ~/rpmbuild/RPMS/noarch/python3-carcano_foolist*.rpm --name my-amazing-software-repo

search the web for "configure passwordless login for hammer" if you don't want to interactively type the credentials to login to the Red Hat Network Satellite Server.

This is the command we'd use to push to Red Hat Network Satellite 5.x (or Spacewalk):

rhnpush --username=ciuser --channel "my-amazing-software-channel" --nosign --server=<FQDN> ~/rpmbuild/RPMS/noarch/python3-carcano_foolist*.rpm

to run the sign target of the Makefile, simply type:

export RELEASE="0.0.2"
make sign

again, right after typing the password to unlock the secret key for the signature, the RPM packages get signed.

Verify the Signature of the Package

Let's now pretend to be the ones that need to install the RPM package: since the delivered RPM package is signed, we must import the public key used to sign it into an RPM database:

sudo rpm --import ~/RPM-GPG-KEY-grimoire

we are now ready to install the RPM package using "yum", "dnf" or even the simple "rpm" command tool.

If we are not interested into installing it, but we anyway wot to check the signature, simply type:

rpm --checksig ~/rpmbuild/RPMS/noarch/python3-carcano_foolist*.rpm

the output is as follows:

/home/grimoire/rpmbuild/RPMS/noarch/python3-carcano_foolist-0.0.2-1.el8.noarch.rpm: digests signatures OK
/home/grimoire/rpmbuild/RPMS/noarch/python3-carcano_foolist-common-0.0.2-1.el8.noarch.rpm: digests signatures OK

the RPM packet is signed and the signature is valid if the word "signatures OK" appears at the end of the  line.

Footnotes

Here it ends the third and last part of this trilogy of posts dedicated to Python: I hope you enjoyed it. You now also know how to create an RPM package. I hope you enjoyed the whole trilogy, and that it provided you with good hints about how to set up and manage your Python projects.

Many years ago, as a child I saw an interview with Freddie Mercury in which he said "whatever you do, do it with style". I was really impressed by that sentence: he was right - doing things and doing things with style is a completely different matter.

In this trilogy I wanted to share my style, ... probably there are things that can be improved a bit more, but at least setting things like so has proved to be a clean and easy way of doing, that promotes maintainability. I hope you appreciate the strive.

Writing a post like this takes a lot of hours. I'm doing it for the only pleasure of sharing knowledge and thoughts, but all of this does not come for free: it is a time consuming volunteering task. This blog is not affiliated to anybody, does not show advertisements nor sells data of visitors. The only goal of this blog is to make ideas flow. So please, if you liked this post, spend a little of your time to share it on Linkedin or Twitter using the buttons below: seeing that posts are actually read is the only way I have to understand if I'm really sharing thoughts or if I'm just wasting time and I'd better give up.

 

 

 

1 thought on “Packaging a Python Wheel as RPM

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>