Migrating Uyuni from Version 3.2 to Version 4

The migration from Uyuni 3.2 to Uyuni 4 works in the same way as the migration from Uyuni 2.1 to Uyuni 3. The migration happens from the original machine to a new one. There is no in-place migration. While this has the drawback that you temporarily need two machines, it also has the advantage that the original machine will remain fully functional in case something goes wrong.

Migration Process

The whole process may be tricky, so it is strongly advised that the migration is done by an experienced consultant.

Given the complexity of the product, the migration is an "all or nothing" procedure, so if something goes wrong you will need to start over. Error handling is very limited. Nevertheless, it should work correctly if you follow these instructions carefully.

Time-Consuming Operation

The migration involves dumping the whole database on the source machine and restoring it on the target machine. All of the channels and packages then need to be copied to the new machine. You should expect the entire process to take several hours.

Prerequisites

Latest Updates

The source machine needs to run Uyuni 3.2 with all the latest updates applied. Before starting the migration process, make sure that the machine is up to date and all updates have been installed sucessfully.

During migration the database from the source machine needs to get dumped and this dump needs to be temporarily stored on the target system. The dump gets compressed with gzip using the default compression options (maximum compression only yields about 10% of space savings but costs a lot of runtime). Check the disk usage of the database; for example, a small test installation:

du -sch /var/lib/pgsql/data
==>
1,8G    data
1,8G    total

Ensure that you have at least 30% of this value available in /var/spacewalk/tmp.

For larger installations the ratio might be better (space required for the database dump might be less than 30%).

  1. Temporary Storage Space

The dump will be written to the directory /var/spacewalk/tmp. The directory will be created if it does not exist. If you want the dump to be stored somewhere else, change the definition of the variable $TMPDIR at the beginning of the script to suit your needs.

Setting Up the Target Machine

To prepare the target machine (with the example host name suma40) proceed as follows:

Procedure: Setup Target Machine
  1. On the target machine, install “Uyuni” Server 4.0 with the Unified Installer (SUSE Linux Enterprise 15 SP1). For more information about installing Uyuni, see [install-server-unified].

  2. Run yast2 susemanager_setup to set up Uyuni.

  3. On the first Uyuni setup screen, ensure that Migrate a SUSE Manager compatible server is marked instead of Set up SUSE Manager from scratch.

  4. On the second screen, enter the name of the source system as Hostname of source SUSE Manager Server as well as the domain name. Enter the database credentials of the source system.

  5. On the next screen, you need to specify the IP address of the Uyuni 4 target system. Generally, this parameter should default to the correct value and you only should need to press Enter. Only in the case of multiple IP addresses you might need to specify the one that should be used during migration.

    Faking the Host Name
    During the migration process, the target system will fake its host name to be the same as the source system, this is necessary as the host name of a Uyuni installation is vital and should not be changed once set. Be careful when you log in to your systems during migration; they both will present you with the same host name.
  6. Continue by following the normal installation steps.

    Database Parameters
    Specify the database parameters using the same database parameters as the source system. Even if you intend to change it later on, during migration you must use the the same database credentials as the original Uyuni database.
  7. After all the data has been gathered, YaST will terminate.

Performing the Migration

The migration will not start automatically. You will need to trigger it manually, with this command:

/usr/lib/susemanager/bin/mgr-setup -m

This command reads the data that was gathered during the setup procedure, sets up Uyuni onto the new target machine, and transfers all of the data from the source machine. Several operations need to be performed on the source machine using SSH, so you will be prompted once for the root password of the source machine. A temporary SSH key named migration-key is created and installed on the source machine, so you need to give the root password only once. The temporary SSH key will be deleted after successful migration.

Depending on the size of the installation, the migration can take several hours. When the migration has finished successfully, you will be notified ("Migration complete.") and prompted to shut down the source machine. Re-configure the network of the target machine to use the same IP address and host name as the original machine and restart it. The target machine is now a fully functional replacement for your previous Uyuni 3.2 installation.

These numbers from a test installation illustrate the approximate time it takes to dump and import the database:

14:53:37   Dumping remote database to /var/spacewalk/tmp/susemanager.dmp.gz on target system. Please wait...
14:58:14   Database successfully dumped. Size is: 506M
14:58:29   Importing database dump. Please wait...
15:05:11   Database dump successfully imported.

In a test installation with a 1,8GB database, we recorded these times:

  • Dumping the database took around five minutes

  • Importing the dump onto the target system took an additional seven minutes

For big installations this can take up to several hours. You should also account for the time it takes to copy all the package data to the new machine. Depending on your network infrastructure and hardware, this can also take a significant amount of time.

A complete migration can consume a lot of time. This is caused by the amount of data that must be copied. Total migration time can be greatly decreased by eliminating the need to copy data prior to performing the migration (for example, channels, packages, auto-install images, and any additional data). You can gather all data via YaST by running:

mgr-setup -r

The mgr-setup -r command copies the data from the old server to the new one. This command can be run at any time and your current server will remain fully functional. When the migration has been initiated only data changed since running mgr-setup -r will need to be transferred. This significantly reduces downtime.

On large installations, transferring the database (which involves dumping the database onto the source machine and then importing the dump onto the target system) will still take some time. During the database transfer no write operations should occur. To avoid this, the migration script shuts down any Uyuni database services running on the source machine.

Packages on External Storage

If you have package data on external storage (for example, if you have an NFS mount at /var/spacewalk/packages), you do not need to copy this data to the new machine. After migration has completed, edit the script located in /usr/lib/susemanager/bin/mgr-setup and remove the respective rsync command (it will be around line 442).

Mounting External Storage

Make sure your external storage is mounted on the new machine before starting the system for the first time. Also make sure /srv/www/htdocs/pub is mounted if it exists on an external storage device.

All other required files and directories that have not been copied by the migration tool, should be manually copied to the new server.

Troubleshooting

This section describes some common problems found after migration.

Web UI Fails to Load

It is possible that the Web UI may break during migration. This behavior is not a bug, but a browser caching issue. The new machine has the same host name and IP address as the old machine. This duplication can confuse some Web browsers. If you experience this issue reload the page. For example, in Firefox pressing the key combination Ctrl+F5 should resume normal functionality.

Not Enough Disk Space

In case of trouble check available disk space. It is recommended to have /var/spacewalk and /var/lib/pgsql on separate (XFS) file systems. Make sure to remove the subvolume entry in /etc/fstab for the subvolume of /var/lib/pqsql when using a separate file system and reboot the server first before continuing.

Corrupted Database Dump

Check the output of the following command (replace <SUMA_3.2_MACHINE> with the actual host name of your 3.2 source machine):

ssh root@<SUMA_3.2_MACHINE> "su -s /bin/bash - postgres -c exit"

This command must not produce any output. Output can lead to a corrupted transfer of the archive with the database dump. Re-visit your bash environment on the 3.2 source machine (for example, the .bashrc file) and make sure no extra text is printed on the shell start.

Retrying to Set Up the New Server

To retry setting up the new server, perform the following steps on the new server machine:

  1. remove /root/.MANAGER_SETUP_COMPLETE

  2. stop postgresql and remove /var/lib/pgsql/data

  3. set the hostname correctly (it now has the host name from the old Uyuni server)

  4. correct the /etc/hosts file

  5. on the new server check /etc/setup_env.sh and see if the correct database name is set:

    MANAGER_DB_NAME='susemanager'
  6. reboot the server before running mgr-setup again.