Monthly Archives: January 2024

FreeIPA server upgrade CentOS 7 to *EL 9

Ukategorisert

This is not really a personal project, but I post it here for lack of other options, and I hope it helps someone.

Migrating FreeIPA servers from CentOS 7 to RHEL 9 is working, but not officially supported. There are two failures that reliably occurred and the solutions are described below.

Plan: Remove the original IPA servers and install new VMs with the same IP addresses and host names. Conditions: both OSes are updated to the newest patches as of 2024-01-17.

Procedure outline; without problems:

  1. (Generate SID for all users; this may prevent Problem 2)
  2. Remove ipa2 from topology using WebUI, then shutdown, install RHEL9.
  3. Join newly installed ipa2 to the IPA domain.
  4. Run migration from ipa1 to the new ipa2.
  5. Remove ipa1 from the topology and reinstall OS.
  6. Join ipa1 to domain.
  7. Replicate to ipa1.

Problem 1: Crash during the first replication from Cent7 master to new RHEL9 master (step 3)

The error looks something like this (in the output of ipa-replica-install --setup-ca):

CalledProcessError(Command [\'/usr/bin/openssl\', \'pkcs12\',
\'-in\',
\'/tmp/tmp7jrs5dqp/import.p12\', \'-clcerts\', \'-nokeys\',
\'-out\',
\'/var/lib/ipa/ra-agent.pem\', \'-password\',
\'file:/tmp/tmp7jrs5dqp/passwd\'] returned non-zero exit status 1:
\'Error outputting keys and
certificates\n80EB2D6B5D7F0000:error:0308010C:digital envelope
routines:inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:346:Global
default library context, Algorithm (RC2-40-CBC : 0),
Properties ()\n\')\n')
[error] FileNotFoundError: [Errno 2] No such file or directory:
'/var/lib/ipa/ra-agent.key'
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

Reference: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org/thread/KNTSO4CTCSWOMJEEXHEDPN2CYZVJUIIO/ and linked bugs.

The solution is presented as updating the source IPA server – but this is not possible on Cent7/EPEL7. Instead the following modification can be made on the target server: Edit /etc/ssl/openssl.cnf and uncomment the following in [providers_sect] to allow the legacy cipher:

##legacy = legacy_sect
...
##[legacy_sect]
##activate = 1

You could re-comment them after the successful migration.

Problem 2: Unable to authenticate with Kerberos on the new IPA master

The problem may be prevented by generating SIDs for all users BEFORE the upgrade – see the ipa config-mod command below, but this was not tested. The following assumes that you didn’t do this, and explains how to fix the issue.

kinit admin works, but kinit doesn’t work for any other user. Actually it could work for some users, and will work for new users created after the migration. The error from kinit is:

kinit: Generic error (see e-text) while getting initial credentials

The error in /var/log/krb5kdc.log is something like this:

May 12 08:41:42 ipa6.example.com krb5kdc[1573](info): AS_REQ (4 etypes
{aes256-cts-hmac-sha1-96(18), aes256-cts-hmac-sha384-192(20), aes128-cts-hmac-sha256-128(19), aes128-cts-hmac-sha1-96(17)}) 192.168.0.6: HANDLE_AUTHDATA: user(a)EXAMPLE.COM for krbtgt/EXAMPLE.COM(a)EXAMPLE.COM, No such file or directory

From this posting: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org/thread/RSAXURG6AR3MWONR3CZSOOI5ULDB2UVC/

The cause of these is that the SID is missing for these users. That’s the cause of the “No such file or directory” error. The SID is needed by FreeIPA’s kerberos after an upgrade, because Kerberos wants to genereate a PAC (Privileged Attribute Certificate) by default now. You can check for a SID with 

ipa user-show USERNAME --all

There should be an “ipantsecurityidentifier” line. To fix this, sources will typically tell you to run:

ipa config-mod --enable-sid --add-sids

The problem is that this won’t work if there’s not a valid ID range for the user’s POSIX UID. It’s quite likely that the users won’t be in valid ID ranges after a migration – because the new replicas will have new ID ranges. There will be error messages in the relevant log file under /var/log/dirserv to this effect.

To fix this go into the IPA WebUI and create ID ranges for any UID in the domain. Then the --add-sids function will work, and then Kerberos will work. All the user’s IDs need to be within the range defined by Base ID and Range size for one of the listed ranges. The Primary RID base and Secondary RID base needs to be set to some values so the corresponding RID ranges don’t overlap with existing RID ranges. RID defines the last component of the SIDs that will be generated, and is the unique part per user or group within a domain.

Problem 3: Unable to complete the first replication

When performing the change in the production environment I had another issue. I had the following error: https://lists.fedoraproject.org/archives/list/freeipa-users@lists.fedorahosted.org/thread/5PHFG7FLA3JZ3Z527BPUDPMMO67XIBUK/#IHIPPVMMIWV2TL7BNLW55XII3OIQ62HK

There’s a work-around by Oleg Baranov, that probably works. The solution is to change the password storage format. For setting up this if you don’t have the directory manager password available, the following can be used. Create /root/fixing.ldif:

dn: cn=config
changetype: modify
replace passwordStorageScheme
passwordStorageScheme: PBKDF2_SHA256

Enable this config file when installing the server:

# ipa-replica-install --setup-ca --dirsrv-config-file=/root/fixing.ldif