Downgrading below 5.12.0 with external authentication¶

Affected scenario¶

RhodeCode Enterprise stores the authentication plugin a user is bound to in the users.extern_type column, and an authentication plugin only authenticates users whose extern_type it recognises.

Up to 5.11.x a plugin persisted its name into that column. Since 5.12.0 the plugin uid is persisted instead. For most plugins the two are identical and nothing changes, but for a few of them they differ:

Plugin	5.12.0+ (`uid`)	up to 5.11.x (`name`)
LDAP + User Groups (RhodeCode Enterprise)	`ldap_group`	`ldap`
Jasig CAS	`jasig_cas`	`jasig-cas`
Authentication token	`token`	`authtoken`
Azure Entra SAML (RhodeCode Enterprise)	`azureentra_saml`	`azureentra`
OneLogin SAML (RhodeCode Enterprise)	`onelogin_saml`	`onelogin`
Duo Security SAML (RhodeCode Enterprise)	`duosecurity_saml`	`duosecurity`

Every user who logs in while the instance runs 5.12.0 or newer gets the new value written to their row. Releases up to 5.11.x do not know about those values, so after a downgrade to 5.11.x or older those users are refused by every authentication plugin.

The most common case is the LDAP one: on 5.12.0 users authenticated through the “LDAP + User Groups” plugin get extern_type='ldap_group', and after a downgrade to for example 5.9.1 neither the CE ldap plugin nor the EE ldap_group plugin accepts them, because in that release both plugins only accept extern_type='ldap'.

Note

The opposite direction depends on the plugin and on the build you roll forward to.

The two LDAP plugins and the three SAML plugins (Azure Entra, OneLogin, Duo Security) accept both the uid and the legacy name in every 5.12.x and 5.13.x release. Users of those plugins that were normalized before a downgrade can be rolled forward without a second migration.
Jasig CAS and the authentication token plugin accept the legacy name only from the build that generalized this behaviour; the original 5.12.0 release accepts their uid spelling only. Check the release notes of your target version, and if it does not contain that fix: Jasig CAS is an external plugin, so letting the affected users log in once after the roll forward rewrites their extern_type on its own. The token plugin is not, so token users have to be corrected from the saved dry-run list or restored from the backup.

Symptom¶

Affected users cannot log in over the web UI, and Git / Mercurial / Subversion operations over HTTP and SSH are rejected for them.
Newly created users and users bound to unaffected plugins are not impacted.

With debug logging enabled the authentication chain logs one line per plugin:

DEBUG [rhodecode.authentication.base] User `jdoe` is bound to `ldap_group`
auth type. Plugin allows only ['ldap'], skipping

Verification¶

Run the following query against the RhodeCode Enterprise database to list the users that would be locked out by a downgrade:

SELECT extern_type, count(*)
FROM users
WHERE extern_type IN (
    'ldap_group', 'jasig_cas', 'token',
    'azureentra_saml', 'onelogin_saml', 'duosecurity_saml'
)
GROUP BY extern_type
ORDER BY extern_type;

An empty result means the instance is safe to downgrade as far as extern_type is concerned.

Remediation¶

Run the rc-normalize-extern-type command before stopping the running instance for the downgrade. It rewrites the affected rows back to the values the older release understands.

The command is a dry-run by default and prints exactly which users it would change:

$ rc-normalize-extern-type /etc/rhodecode/conf/rhodecode.ini
Would update 2 user(s):
  jdoe: ldap_group -> ldap
  asmith: ldap_group -> ldap

Dry-run, nothing was written. Re-run with --apply to persist the changes.

Once the reported list looks right, apply it:

$ rc-normalize-extern-type /etc/rhodecode/conf/rhodecode.ini --apply

Use --extern-type to restrict the run to a single plugin, for example when only the LDAP group users need to be normalized:

$ rc-normalize-extern-type /etc/rhodecode/conf/rhodecode.ini \
    --extern-type ldap_group --apply

The command only rewrites rows whose extern_type is one of the values in the table above. Users bound to any other plugin, users with an empty extern_type and every other user attribute are left untouched.

Safety notes and rollback¶

Take a database backup before running the command with --apply. See Backup and Restore.
Always inspect the dry-run output first. It lists every row that will change.
Only users.extern_type is modified. Passwords, tokens, permissions, group memberships and extern_name are not touched, so no user has to re-authenticate or be re-provisioned.
Keep the dry-run output. It is the only exact record of which rows were changed: after the run those users are indistinguishable from users that were bound to the legacy plugin all along, so there is no query that can reconstruct the list.
To roll back, restore the database backup taken before the run, or re-apply the new spelling to the exact usernames from the dry-run output. Do not try to reverse the change with a broad UPDATE ... WHERE extern_type = 'ldap': that also rewrites users who were never bound to the LDAP group plugin.
A rollback is usually unnecessary. Users bound to the legacy value keep authenticating on any build that accepts both spellings (see the note above). External plugins (LDAP, Jasig CAS, SAML) additionally rewrite extern_type back to the current value on the next successful login. The token plugin is not an external plugin and never writes the column, so token users stay on the legacy value until they are corrected from the saved dry-run list or restored from the backup.
Restart the RhodeCode Enterprise instance after applying the change so that cached authentication results are dropped.