Pulp is the repository manager used by Katello (the upstream project of Red Hat Network Satellite server 6.x). This Pulp version is really exciting, since it comes with several plugins that let you host several kind of repositories:
- generic files
- software packages: RPM packages, Deb Packages and even Container Images
- software modules: Python modules, Maven contents, Ruby Gem
- contents configuration management software: Ansible roles and collections and Chef cookbooks
In this post I'll show you how easy it is to install Pulp3 as container using the official container image.
Install the container management runtimes
Since we are dealing with a container image, we need something to manage containers: in this post we use Podman, but everything works pretty same way even using Docker.
Let's install Podman:
sudo dnf install -y podman
if you don't have a Red Hat subscriptions to download container images from Red Hat's CDN, you can configure "/etc/containers/registries.conf" to use only "docker.io" registry:
# To ensure compatibility with docker we've included docker.io in the default search list. However
# Red Hat does not curate, patch or maintain container images from the docker.io registry.
[registries.search]
registries = ['docker.io']
Create filesystems to store settings and data
Let's begin by properly partitioning the system: in this example we configure "repos" LVM Volume Group to use "/dev/sdb" disk as Physical Volume:
DISK="/dev/sdb"
sudo parted -a optimal ${DISK} mklabel gpt
SDB_DISK_SIZE=$(sudo parted ${DISK} unit s print free |grep -v "^$" |tail -n 1|awk '{ print $2 }')
sudo parted -a optimal ${DISK} mkpart primary 2048s ${SDB_DISK_SIZE=}
sudo parted ${DISK} set 1 lvm on
sudo pvcreate ${DISK}1
sudo vgcreate repos ${DISK}1
now let's create the Logical Volumes necessary to keep things separated so to avoid to run out of disk space simply because somebody by blindly uploading software fill-up the whole storage and cause Pulp3 service to fail.
sudo lvcreate -L 3GiB -n pulp_storage repos
sudo lvcreate -L 3GiB -n pulp_pgsql repos
sudo lvcreate -L 3GiB -n pulp_containers repos
let's create the XFS filesystems into them:
sudo mkfs.xfs /dev/mapper/repos-pulp_storage
sudo mkfs.xfs /dev/mapper/repos-pulp_pgsql
sudo mkfs.xfs /dev/mapper/repos-pulp_containers
eventually let's create the necessary mount points and configure the automatic mount at boot:
sudo mkdir -m 755 -p /opt/pulp/storage
sudo mkdir -m 755 /opt/pulp/pgsql
sudo mkdir -m 755 /opt/pulp/containers
sudo tee -a /etc/fstab >/dev/null <<-EOF
/dev/mapper/repos-pulp_storage /opt/pulp/storage xfs defaults 0 0
/dev/mapper/repos-pulp_pgsql /opt/pulp/pgsql xfs defaults 0 0
/dev/mapper/repos-pulp_containers /opt/pulp/containers xfs defaults 0 0
EOF
and mount them all:
sudo mount -a
let's verify that everything has been properly setup:
df -Ph /opt/pulp/storage/ /opt/pulp/pgsql/ /opt/pulp/containers/
the outcome should be as follows
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/pulp-storage 3.0G 54M 3.0G 2% /opt/pulp/storage
/dev/mapper/pulp-pgsql 3.0G 54M 3.0G 2% /opt/pulp/pgsql
/dev/mapper/pulp-containers 3.0G 54M 3.0G 2% /opt/pulp/containers
Create the directory where to store Pulp3 configuration files:
sudo mkdir -m 755 /opt/pulp/settings
sudo ln -s /opt/pulp/settings/ /etc/pulp
Configure the Pulp3 service
Let's create the main configuration file:
CONTENT_ORIGIN='http://repo-ci-up2a001.core.carcano.local:8080'
ANSIBLE_API_HOSTNAME='http://repo-ci-up2a001.core.carcano.local:8080'
ANSIBLE_CONTENT_HOSTNAME='http://repo-ci-up2a001.core.carcano.local:8080/pulp/content'
TOKEN_AUTH_DISABLED=True
as you can see, only a very few settings are necessary to have an up and running Pulp3.
Run the Pulp3 container for the first time
We are ready to create the Pulp3 Container - let's start by pulling the image:
sudo podman pull pulp/pulp-fedora31
it takes some time to download the image: it's quite a huge one.
Trying to pull docker.io/pulp/pulp-fedora31...
Getting image source signatures
Copying blob d148f84634cf done
Copying blob d26278d60d49 [==============>-----------------------] 107.2MiB / 278.9MiB
Copying blob 854946d575a4 done
When finished, create the container as follows:
podman run --detach \
--publish 8080:80 \
--name pulp \
--volume /opt/pulp/settings:/etc/pulp:Z \
--volume /opt/pulp/storage:/var/lib/pulp:Z \
--volume /opt/pulp/pgsql:/var/lib/pgsql:Z \
--volume /opt/pulp/containers:/var/lib/containers:Z \
--device /dev/fuse \
pulp/pulp-fedora31
Wow the container should be up and running.
Set the administrative password
We should set the administrative password as follows:
sudo podman exec -it pulp bash -c 'pulpcore-manager reset-admin-password'
just type the password you want to use.
Create the systemd unit
There's one thing still missing: the systemd unit file to run it as a service. We can easily create is as follows:
sudo tee -a /etc/systemd/system/pulp.service >/dev/null <<-EOF
[Unit]
Description=Pulp
Wants=syslog.service
[Service]
Restart=always
ExecStart=/usr/bin/podman start -a pulp
ExecStop=/usr/bin/podman stop -t 2 pulp
[Install]
WantedBy=multi-user.target
EOF
let's reload systemd to make it aware of this new unit file:
sudo systemctl daemon-reload
We should of course configure selinux to grant containers to manage cgroup, or the sytemd unit won't work:
setsebool -P container_manage_cgroup on
we can enable the unit to start at boot:
systemctl enable pulp
restart the server
sudo shutdown -r now
At the next boot your Pulp3 repository should be up and running: open the following URL with your borwser:
Have a go on Pulp3
http://127.0.0.1:8080/pulp/api/v3/docs/
You should get the documentation of the API:
Footnotes
Here it ends first post about Pulp3: I hope you enjoyed it. Pulp3 is certainly a very good piece of software and looks very promising. You can use it as a local repository to mirror online repositories as well as the Definitive Media Library where to store your software, artifacts and configuration management stuff.
The only thing it still lacks is a web UI, but it's API is really well designed and easy to use, and this makes it very suitable to work with automation tools and scripts: we'll see how to use it in the next post.
Well, what to do next? A very hot topic is to use it as a local Ansible-Galaxy repository to store roles and collections. This will be the topic of the next post.
I hate blogs with pop-ups, ads and all the (even worse) other stuff that distracts from the topics you're reading and violates your privacy. I want to offer my readers the best experience possible for free, ... but please be wary that for me it's not really free: on top of the raw costs of running the blog, I usually spend on average 50-60 hours writing each post. I offer all this for free because I think it's nice to help people, but if you think something in this blog has helped you professionally and you want to give concrete support, your contribution is very much appreciated: you can just use the above button.
richg says:
Hi, First let me say you have a great Blog. I only found you today trying to figure out pulp3. Background, I work for a large but small company, we have satellite in place at the moment, and are up for renewal. We really only use it as a repo for our patching. I have tried to push for it to be used for other things, insight, provisioning VM, AMI updates, but I contantly get push back from the other team members. So my thought was to go to management and offer pulp3 as a way of saving the company 100K. I have to admit while I unstand the applications, I struggle with Open Source at times as I have to rely on folks like yourself with docs like this for getting me started. I have managed to save them money on Ansible I have that running on Kubernetes using AWX great product took a while but I manged to get it up on three different nodes. Ok enough of my dribble as I am sure you are busy. At the moment I have been trying to get your instructions to work for me, I do realize it is over a year old and things change very quickly in this industry. I am trying to get going on my home system before I try at work so I can get all my ducks in row before offering to the company. Ok, so when I try and do the podman run with you instructions if fails with an Error: setxattr /opt/pulp/settings: Operation not premitted, not sure if you can provide any direction but it sure would help. I can provide some more detaisl on how I am doing this if you like, this is getting very long winded here. I would love to talk to your further as I am very impressed with you accomplishments, I have worked in this industry for the past 40 years now I am not a youngester any more 67 in sept. I hope you will reply here.. I will keep checking back.
Marco Antonio Carcano says:
Hello Richard, I’m pleased you like the blog – unfortunately as you pointed out the post is quite outdated – pulp3 has been improved in the meantime (think that when I played with it was at a so early stage that I had to write a CLI on my own since it hadn’t anything to interact with except the API). Anyway the error message you got looks very similar to this: https://github.com/containers/podman/issues/14054 – they suggest updating at least to podman 4.1.
richg says:
Thank you for taking time out of your day to reply. I am just now trying to go through all the information you have written. I would to say thank you for taking the time to share your knowledge with the IT comm. I for one do appreciate it.
I thought that when I was installing your version of pulp, but I wanted to give it a try anyway. While I love open source I also hate it as I have dig for everything sometimes, but I guess that is what keep my mind sharp having to think. All the best Sir, I will let you know my opinion after reading some more. Have a great week.