Deploying your Python Code to the uyuni-server

Overview

Uyuni is a very powerful open source systems management solution that can be used to manage various Linux distributions.

The Uyuni community is very open for contributions from anyone who to help make it better. However, it can be a little bit tricky to set up your development environment, write your modifications and deploy your version locally.

In this article, we'll try to briefly illustrate how you can set up your local environment, deploy your version to the local uyuni-server and test it.

Note: In this article we'll focus only on the Python part of Uyuni.

Setting Up the Environment: sumaform

The Uyuni community uses a tool called sumaform to set up and configure the test environment for Uyuni development.

Sumaform is a way to quickly configure test Uyuni and SUSE Manager infrastructures with clients and servers. [1] You can follow the steps in the sumaform GitHub page to effectively set up the environment.
Note: It's recommended to use one of openSUSE's Linux distributions to develop Uyuni. For example: openSUSE Tumbleweed, openSUSE Leap 15.5, etc..

After completing all the steps of the sumaform setup, you'll normally have two virtual machines installed on your host OS, which are server and minion.
The server VM is normally a openSUSE Leap 15.5 machine with some useful development tools already installed.
Inside this server VM, there's a running podman container called uyuni-server which contains all the code and programs for Uyuni:

After completing setting up the development environment using sumaform, you can start the configured VMs:

$ virsh start server
$ virsh start minion

After that, you can wait a few seconds until the VMs completely start, and connect to the server VM using ssh:

$ ssh root@server.tf.local (password: linux)

Once logged in to the server VM, you can start the uyuni-server container using mgrctl tool, which will be already installed on the VM:

$ mgrctl term

Deploying Changes to uyuni-server

Once completed the sumaform configuration, you can clone the uyuni GitHub repository, follow the contribution guides and make your contribution.

Once you have your changes ready, you can deploy these changes by copying the affected files to the uyuni-server container. The best way to do this is by using a podman connection.
To do this your should first install podman on your host machine (installation steps will depend on your Linux distribution).
Then, create a new podman connection :

$ podman --log-level=debug system connection add server ssh://root@server.tf.local
$ podman system connection ls # [optional: to visualize the connection]
$ export CONTAINER_CONNECTION=server # NOTE: this will only apply to the current shell instance  
$ echo $CONTAINER_CONNECTION # [optional]

Now, we can copy our files easily using the mgrctl cp tool:

$ mgrctl cp path/to/your/file server:/usr/lib/...

Note: Sometimes when copying python scripts that replaces old ones, you might encounter an unexpected ModuleNotFoundError. I still haven't identify the cause of this error, but if the changes are not huge, you can write them directly to the scripts on the uyuni-server using vim. For example adding some changes to reposync.py:

uyuni-server:$ vim /usr/lib/python3.6/site-packages/spacewalk/satellite_tools/reposync.py
# Add your changes..
esc + :wq # save and exit

But you should be aware of the vim's python formatting issues, that's why you should add some vim configuration to be able to write well formatted python code in vim.
To do this you can add the following information to the /root/.vimrc config file (in the server VM):

 uyuni-server:$ echo "set tabstop=4 softtabstop=4 shiftwidth=4 expandtab smarttab autoindent" > /root/.vimrc

Deploying Python Code: lzreposync

In this section we'll illustrate how to deploy the lzreposync service (currently still in development on the date of writing this article), which might contain scripts and files that doesn't have their corresponding ones in uyuni-server.

First, you should create a new directory, python, on the uyuni-server to put your project files in:

uyuni-server:$ cd /usr/lib
uyuni-server:$ mkdir python

Then you can copy all the project files from your host to that new location using mgrctl cp:

localhost:$ cd path_to/uyuni/python
localhost:$ mgrctl cp ./lzreposync server:/usr/lib/python
localhost:$ mgrctl cp ./billingdataservice server:/usr/lib/python
localhost:$ mgrctl cp ./linting server:/usr/lib/python
localhost:$ mgrctl cp ./spacewalk server:/usr/lib/python
localhost:$ mgrctl cp ./uyuni server:/usr/lib/python
localhost:$ mgrctl cp ./rhn server:/usr/lib/python
localhost:$ mgrctl cp ./test server:/usr/lib/python # [optional]

Then, you should create a python virtual environment and install the required libraries, and that's after installing some required tools:

# Installing the required programs
uyuni-server:$ zypper install python311
uyuni-server:$ zypper install python311-pip
uyuni-server:$ zypper install python311-rpm # for rpm binding

# Setting up the environment
uyuni-server:$ cd /usr/lib/python
uyuni-server:$ python3.11 -m venv venv --system-site-packages
uyuni-server:$ . venv/bin/activate
(venv)uyuni-server:$ cd lzreposync
(venv)uyuni-server:$ pip install -e .
(venv)uyuni-server:$ pip install pyopenssl
(venv)uyuni-server:$ pip install rpm
(venv)uyuni-server:$ pip install psycopg2-binary
(venv)uyuni-server:$ pip install ... # potentially other libraries

Note that, at this stage, some modules might not be correctly imported when running lzreposync, so we you should first add the project's root path to a _.pth file under site-packages folder:

uyuni-server:$ cd /usr/lib/python
uyuni-server:$ echo "/usr/lib/python" > venv/lib64/python3.11/site-packages/uyuni_python_paths.pth

Now to run the lzreposync service, you can simply execute:

uyuni-server:$ lzreposync # provide the required arguments

In One Script

The following shell script groups all the previous steps together and make it easier to deploy your version in one command:

#!/bin/sh

podman --log-level=debug system connection add server ssh://root@server.tf.local
export CONTAINER_CONNECTION=server

virsh start minion
virsh start server

mgrctl exec 'zypper --non-interactive install python311'
mgrctl exec  'zypper --non-interactive install python311-pip'
mgrctl exec 'zypper --non-interactive install python311-rpm'
mgrctl exec  'zypper --non-interactive install python311-devel'
mgrctl exec 'mkdir /usr/lib/python'
mgrctl exec 'python3.11 -m venv /usr/lib/python/venv --system-site-packages'
mgrctl exec  '. /usr/lib/python/venv/bin/activate'

mgrctl cp /path/to/uyuni/python/lzreposync/ server:/usr/lib/python
mgrctl cp /path/to/uyuni/python/linting/ server:/usr/lib/python
mgrctl cp /path/to/uyuni/python/spacewalk/ server:/usr/lib/python
mgrctl cp /path/to/uyuni/python/uyuni/ server:/usr/lib/python
mgrctl cp /path/to/uyuni/python/rhn/ server:/usr/lib/python
mgrctl cp /path/to/uyuni/python/billingdataservice/ server:/usr/lib/python

mgrctl exec '/usr/lib/python/venv/bin/pip install pytest'
mgrctl exec '/usr/lib/python/venv/bin/pip install pycurl'
mgrctl exec '/usr/lib/python/venv/bin/pip install -e /usr/lib/python/lzreposync/'
mgrctl exec '/usr/lib/python/venv/bin/pip install pyopenssl'
mgrctl exec '/usr/lib/python/venv/bin/pip install rpm'
mgrctl exec '/usr/lib/python/venv/bin/pip install salt' 

mgrctl exec 'echo "/usr/lib/python" > /usr/lib/python/venv/lib64/python3.11/site-packages/uyuni_python_paths.pth'
mgrctl exec 'echo "ENABLE_NVREA = 1" >> /etc/rhn/rhn.conf'

You can copy it into a file named deploy.sh and execute it like this: sh deploy.sh . Normally, after executing this shell script, your lzreposync service version will be successfully deployed on the uyuni-server under /usr/lib/python. So, you can connect to the uyuni-server, cd to /usr/lib/python, activate the environment (. venv/bin/activate), and execute the lzreposync command.

Conclusion

In this article, we tried to briefly illustrate how to setup the uyuni development environment and how to deploy your changes (python part) to the uyuni-server container. I hope it was useful.