MDBF-1104 Auto-determine version name of distro upgrade tests by grooverdan · Pull Request #786 · MariaDB/buildbot

grooverdan · 2025-07-03T04:48:29Z

to correct the minor/major upgrade tests from failing with corrected handing of SLES/RPM repos.

RazvanLiviuVarzaru · 2025-07-03T06:49:13Z

@grooverdan I'd prefer this format, below is just an example how I imagine it (all the other builders are honoring it):
At the root e.g. https://mirror.mariadb.org/yum/10.11/
to have: (those are just symlinks)

sles1506-amd64/
sles1507-amd64/
sles1506-s390x/
sles1507-s390x/

A very good example already honoring this format is rhel as in:

This way we don't need many hardcoded code paths in buildbot.
I see you are already in contact with dbart, can you discuss this approach with him if you find my proposal acceptable?

grooverdan · 2025-07-03T08:03:40Z

The symlinks always looked like a hack to me. The use of non-RPM arch names (amd64) and inconsistent version naming makes the repos that end users use requiring manual editing on distro upgrade. The interfaces that I'd like used by users are ones that can be installed ones across distro versions (including upgrades) and arches like:

https://mariadb.org/download/?t=repo-config&d=SUSE+Enterprise+Linux&v=11.4
MDEV-36945
or https://mariadb.org/download/?t=repo-config&d=Red+Hat+Enterprise+Linux+9&v=11.4 as a file.

As its a mature form that I'd like this to continues for end users, I'd like to see our CI environment testing the upgrades like an end users would in their repofiles.

This way we don't need many hardcoded code paths in buildbot.

hmm, if Rocky Linux/AlmaLinux where fixed we could just use the $releasever/$basearch in bash_lib.sh - yay, another consistenty and less hardcoded paths.

RazvanLiviuVarzaru · 2025-07-03T08:12:12Z

The symlinks always looked like a hack to me. The use of non-RPM arch names (amd64) and inconsistent version naming makes the repos that end users use requiring manual editing on distro upgrade. The interfaces that I'd like used by users are ones that can be installed ones across distro versions (including upgrades) and arches like:

https://mariadb.org/download/?t=repo-config&d=SUSE+Enterprise+Linux&v=11.4

MDEV-36945

or https://mariadb.org/download/?t=repo-config&d=Red+Hat+Enterprise+Linux+9&v=11.4 as a file.

As its a mature form that I'd like this to continues for end users, I'd like to see our CI environment testing the upgrades like an end users would in their repofiles.

I understand your concern and I agree with you that we should install / upgrade in the same way a user would do it.
I don't like big changes done fast, 2-3 weeks before the release, especially in this regard.

This is why I advocate for a two step approach:

SHORT-TERM, BEFORE THE RELEASE: fix SLES!. It's not a big deal to create the symlinks that I've mentioned above
LONG-TERM, AFTER THE RELEASE: structural change , adapt our builders to mimic end user behavior. And this should be a separate MDBF with proper testing.

grooverdan · 2025-07-03T08:13:38Z

sure, can split tomorrow.

RazvanLiviuVarzaru · 2025-07-03T08:19:24Z

sure, can split tomorrow.

If they are done like that, the only change needed in this patch is that version_name attributes will get 1506 & 1507

RazvanLiviuVarzaru · 2025-08-14T06:26:18Z

Lots of bash lib changes so adding @fauust also as co-reviewer.

RazvanLiviuVarzaru · 2025-08-18T10:54:57Z

+  url_path="$ID/$VERSION_ID/$(rpm --eval '%_arch')"
+  mirror_url="https://rpm.mariadb.org/$branch/$url_path"
+  archive_url="https://archive.mariadb.org/mariadb-$branch/yum/$url_path"


Kindly request to evaluate the resulted url_path for all supported distros and compare with our mirrors.

VERSION_ID evaluates to major.minor, at least on rhel.
at mirror/archive only major is present. Example https://mirror.mariadb.org/yum/11.8/rhel/9/x86_64/

e.g.

:~$ docker run -it --platform amd64 --rm quay.io/mariadb-foundation/bb-worker:rhel9-srpm bash -c 'source /etc/os-release && echo "$ID/$VERSION_ID/$(rpm --eval '%_arch')"' rhel/9.6/x86_64 ~$ docker run -it --platform amd64 --rm quay.io/mariadb-foundation/bb-worker:rhel10-srpm bash -c 'source /etc/os-release && echo "$ID/$VERSION_ID/$(rpm --eval '%_arch')"' rhel/10.0/x86_64

OpenSUSE is a whole different story.

$ docker run -it --platform amd64 --rm quay.io/mariadb-foundation/bb-worker:opensuse1506-srpm bash -c 'source /etc/os-release && echo "$ID/$VERSION_ID/$(rpm --eval '%_arch')"' opensuse-leap/15.6/x86_64

and on mirror https://mirror.mariadb.org/yum/11.8/opensuse/15.6/x86_64/

RazvanLiviuVarzaru · 2025-08-18T11:06:26Z

      bb_log_warn "Columnstore was not found in packages, the test will not be run"
      exit
-    elif [[ $version_name == "sid" ]]; then
+    elif [[ $VERSION_CODENAME == "sid" ]]; then


I have no idea why

$ docker run -it --platform amd64 --rm debian:sid bash -c 'source /etc/os-release && echo "/$ID/dists/$VERSION_CODENAME"' /debian/dists/trixie

prints trixie so the elif won't work.

dist_name, version_name and arch can now be determined by install/upgrade scripts. By using the /etc/os-version defines consistent across RPM based distros we can obtain the path of install/upgrade tests based on $ID, being the distro and $VERSION_ID, as the release version. For RHEL and derivities/clones the VERSION_ID is of the form X.Y that needs be reduced to just X because of ABI consistency. A single number X is the way its named on our mirrors. For SLES/OpenSUSE without an ABI consistentcy the VERSION_ID of X.Y version is used. Centos and Fedora use a single number version. Debian/Ubuntu no-longer need version_name as /etc/os-release provides VERSION_CODENAME and $(arch), the command rather than env variable, is the architecture.

Without disto/distro_name being passed we'll use the pkg_cmd as the check to see if this is a sles/suse distro.

RazvanLiviuVarzaru · 2025-08-19T06:40:41Z


+source /etc/os-release
+
+if [ "${PRETTY_NAME##*/}" == "sid" ]; then


Are you sure?

$ docker run --rm debian:sid bash -c 'source /etc/os-release && echo $PRETTY_NAME' Debian GNU/Linux 13 (trixie)

Ah ok, updated image and sid was unlocked. Right.

$ docker run --rm debian:sid bash -c 'cat /etc/os-release' PRETTY_NAME="Debian GNU/Linux forky/sid" NAME="Debian GNU/Linux" VERSION_CODENAME=forky ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"

We should ask @fauust to update the SID VM.

corrected with explicit sid handling as the VERSION_CODENAME rapidly changes after the release of a stable version.

if [ "${PRETTY_NAME##*/}" == "sid" ]; then # Sid has the VERSION_CODENAME of the next # release which isn't useful to us. VERSION_CODENAME=sid fi

We should ask @fauust to update the SID VM.

regularly - #827

That's only the autobake container.
Should discuss with @fauust for a VM update strategy. It involves Ansible but I am not sure it is possible since most of the VM updates require a master-libvirt restart.

The libvirt restart occurs only in the case of a new VM deployment, see: https://gitlab.com/mariadb/sysadmin/-/blob/main/ansible/roles/bb_worker_vm/tasks/libvirt.yml?ref_type=heads#L29, so updating a VM is an easy task that can be done anytime (unless the queue on BB is going to trigger a VM start).

fauust

Very nice job @grooverdan! Some small comments, see below.

Otherwise previous test would have detected it as missing.

…mirror And this removes a double processing of /etc/os-release

RazvanLiviuVarzaru

Approved from me, thank's a lot @grooverdan for simplifying this.
I can merge it after @fauust approves.

grooverdan requested a review from RazvanLiviuVarzaru July 3, 2025 04:48

grooverdan force-pushed the MDBF-1067 branch from d776c21 to 407b9bc Compare July 3, 2025 05:03

grooverdan force-pushed the MDBF-1067 branch 2 times, most recently from 6a2c7f5 to e23539d Compare July 4, 2025 04:38

grooverdan mentioned this pull request Jul 4, 2025

MDBF-1067: sles - change 15.7 name #787

Merged

grooverdan mentioned this pull request Jul 18, 2025

MDEV-36945: disable sles minor upgrade tests until after Q3 2025 #810

Closed

grooverdan force-pushed the MDBF-1067 branch 2 times, most recently from fbcec67 to ca9253a Compare August 14, 2025 04:21

grooverdan changed the title ~~MDBF-1067 sles upgrades - 15.6 and 15.7 are distinct use rpm repos consistently~~ MDBF-1104 Auto-determine version name of distro upgrade tests Aug 14, 2025

grooverdan force-pushed the MDBF-1067 branch 3 times, most recently from 95dbe59 to dfaf33c Compare August 14, 2025 04:41

RazvanLiviuVarzaru requested a review from fauust August 14, 2025 06:23

RazvanLiviuVarzaru reviewed Aug 18, 2025

View reviewed changes

Comment thread scripts/rpm-upgrade.sh

RazvanLiviuVarzaru reviewed Aug 18, 2025

View reviewed changes

Comment thread scripts/bash_lib.sh

grooverdan added 2 commits August 19, 2025 14:21

rpm-upgrade.sh - sles/suse detection

13aeccf

Without disto/distro_name being passed we'll use the pkg_cmd as the check to see if this is a sles/suse distro.

grooverdan force-pushed the MDBF-1067 branch from dfaf33c to 13aeccf Compare August 19, 2025 04:24

RazvanLiviuVarzaru reviewed Aug 19, 2025

View reviewed changes

Comment thread scripts/bash_lib.sh Outdated

RazvanLiviuVarzaru reviewed Aug 19, 2025

View reviewed changes

grooverdan added 2 commits August 25, 2025 14:25

MDBF-1067: rpm opensuse-leap is opensuse on MariaDB repos

e140909

fix: compatibility spelling in bash_lib.sh script

4d32caa

fauust reviewed Aug 27, 2025

View reviewed changes

Comment thread scripts/bash_lib.sh

Comment thread scripts/bash_lib.sh Outdated

grooverdan commented Aug 27, 2025

View reviewed changes

Comment thread scripts/deb-install.sh

grooverdan added 4 commits August 28, 2025 09:05

rpm_setup_mariadb_mirror: style - if -> case statement

33588c6

deb-upgrade: columnstore is on sid

a055e88

Otherwise previous test would have detected it as missing.

deb_setup_mariadb_mirror: source /etc/os-release here

6184520

deb-upgrade: get_packages_file_mirror - depends on deb_setup_mariadb_…

3d95c42

…mirror And this removes a double processing of /etc/os-release

grooverdan mentioned this pull request Aug 28, 2025

MDBF 1102 - Implement deb upgrade from distro default #826

Merged

RazvanLiviuVarzaru approved these changes Aug 28, 2025

View reviewed changes

fauust merged commit ae4eadb into MariaDB:dev Aug 28, 2025
3 checks passed

grooverdan deleted the MDBF-1067 branch April 15, 2026 06:10


		source /etc/os-release

		if [ "${PRETTY_NAME##*/}" == "sid" ]; then

Uh oh!

Conversation

grooverdan commented Jul 3, 2025

Uh oh!

RazvanLiviuVarzaru commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

grooverdan commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RazvanLiviuVarzaru commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

grooverdan commented Jul 3, 2025

Uh oh!

RazvanLiviuVarzaru commented Jul 3, 2025

Uh oh!

RazvanLiviuVarzaru commented Aug 14, 2025

Uh oh!

RazvanLiviuVarzaru Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RazvanLiviuVarzaru Aug 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RazvanLiviuVarzaru Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

RazvanLiviuVarzaru Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

RazvanLiviuVarzaru Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

grooverdan Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

grooverdan Aug 25, 2025

Choose a reason for hiding this comment

Uh oh!

RazvanLiviuVarzaru Aug 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fauust Aug 27, 2025

Choose a reason for hiding this comment

Uh oh!

fauust left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RazvanLiviuVarzaru left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

RazvanLiviuVarzaru commented Jul 3, 2025 •

edited

Loading

grooverdan commented Jul 3, 2025 •

edited

Loading

RazvanLiviuVarzaru commented Jul 3, 2025 •

edited

Loading

RazvanLiviuVarzaru Aug 18, 2025 •

edited

Loading

RazvanLiviuVarzaru Aug 18, 2025 •

edited

Loading

RazvanLiviuVarzaru Aug 25, 2025 •

edited

Loading