Skip to content

MDBF-741 Sequence version of ubasan - #815

Merged
RazvanLiviuVarzaru merged 18 commits into
MariaDB:devfrom
grooverdan:MDBF-741-ubsan-seq
Aug 26, 2025
Merged

MDBF-741 Sequence version of ubasan#815
RazvanLiviuVarzaru merged 18 commits into
MariaDB:devfrom
grooverdan:MDBF-741-ubsan-seq

Conversation

@grooverdan

@grooverdan grooverdan commented Jul 30, 2025

Copy link
Copy Markdown
Member

Add New Build template

Checklist

  • Make changes os_info.yaml
  • Schedule the builder for the appropriate branch in constants.py
  • Add builder configuration

@grooverdan grooverdan mentioned this pull request Jul 30, 2025
Closed
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Jul 31, 2025
View reviewed changes
Comment thread configuration/builders/sequences/sanitizers.py Outdated
@grooverdan
grooverdan force-pushed the MDBF-741-ubsan-seq branch from 6cd6bc0 to 59a0b3b Compare July 31, 2025 08:32
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Jul 31, 2025
View reviewed changes
Comment thread configuration/steps/commands/download.py Outdated
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Jul 31, 2025
View reviewed changes
Comment thread master-migration/master.cfg Outdated
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Jul 31, 2025
View reviewed changes
Comment thread master-migration/master.cfg Outdated
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Jul 31, 2025
View reviewed changes
Comment thread configuration/steps/commands/base.py Outdated
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Jul 31, 2025
View reviewed changes
Comment thread configuration/steps/commands/base.py Outdated
@grooverdan
grooverdan force-pushed the MDBF-741-ubsan-seq branch 7 times, most recently from 93501a0 to 581cfd8 Compare August 1, 2025 04:15
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Aug 1, 2025
View reviewed changes
Comment thread configuration/steps/commands/download.py Outdated
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Aug 1, 2025
View reviewed changes
Comment thread configuration/builders/sequences/helpers.py Outdated
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Aug 1, 2025
View reviewed changes
Comment thread master-migration/master.cfg Outdated
@RazvanLiviuVarzaru

RazvanLiviuVarzaru commented Aug 1, 2025

Copy link
Copy Markdown
Collaborator

Thanks @grooverdan for your patience and re-work,
A few more comments to address then I can start testing this patch on dev and see if it needs more fixing.

RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Aug 1, 2025
View reviewed changes
Comment thread configuration/builders/sequences/sanitizers.py Outdated
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Aug 1, 2025
View reviewed changes
Comment thread configuration/builders/sequences/sanitizers.py Outdated
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Aug 1, 2025
View reviewed changes
Comment thread configuration/builders/sequences/sanitizers.py
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Aug 1, 2025
View reviewed changes
Comment thread configuration/builders/sequences/sanitizers.py
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Aug 1, 2025
View reviewed changes
Comment thread configuration/steps/commands/download.py Outdated
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Aug 1, 2025
View reviewed changes
Comment thread configuration/builders/sequences/sanitizers.py Outdated
@RazvanLiviuVarzaru

RazvanLiviuVarzaru commented Aug 1, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

@grooverdan I wrote a lot, I better provide a summary of what needs to be fixed:

  • remove mkdir bld step
  • remove container commit for compile step
  • fix FetchGitHub -> wrong value for branch
  • wrap get_mtr_normal_steps with InContainer
  • CompileCMakeCommand fix case when verbose is False
  • CMakeGenerator fix source path . Proposed "../src"
  • halt on failure = False for MTR steps . (so all MTR steps run)
  • you need save_mtr_logs after MTR steps

A latest 10.11 run:

RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Aug 1, 2025
View reviewed changes
Comment thread configuration/builders/sequences/sanitizers.py
RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Aug 21, 2025
View reviewed changes
Comment thread master-migration/master.cfg Outdated
@RazvanLiviuVarzaru

RazvanLiviuVarzaru commented Aug 25, 2025

Copy link
Copy Markdown
Collaborator

@grooverdan I've made 3 new fixes on this patch, the second one being my fault.
I am ready to merge this patch but first I want to ask. Are you sure you don't want to address first the failing MTR tests (or at least report them)?
as in https://buildbot.dev.mariadb.org/#/builders/701/builds/10

@grooverdan

grooverdan commented Aug 26, 2025

Copy link
Copy Markdown
Member Author

@grooverdan I've made 3 new fixes on this patch, the second one being my fault. I am ready to merge this patch but first I want to ask. Are you sure you don't want to address first the failing MTR tests (or at least report them)? as in https://buildbot.dev.mariadb.org/#/builders/701/builds/10

All are reported. 1 progress, 1 just provided solution for. Other two slightly stalled.

main.long_unique_bugs, innodb_fts.innodb-fts-ddl ctype-uca-scanner_next.inl:84 -> MDEV-36467
rpl.rpl_mixed_mixing_engines (and other annotate rpl bugs) -> MDEV-37356
main.derived_split_innodb -> MDEV-37438
roles.acl_load_mutex-5170 -> MDEV-36345

@RazvanLiviuVarzaru

RazvanLiviuVarzaru commented Aug 26, 2025

Copy link
Copy Markdown
Collaborator

Quote reply

Perfect, thank you!
I will merge it then, it's good to have it running for any regressions or new bugs.

grooverdan and others added 18 commits August 26, 2025 10:23
@grooverdan @RazvanLiviuVarzaru
Add to helper functions args additional_mtr_options and env_vars
56acc70 
To provide a mechanism to add a simple number of MTR options
the helper sequence functions gain an additional argument.

Additionally env_vars added as argument too, defaulting to
an empty list for simplicity of concatination.
@grooverdan @RazvanLiviuVarzaru
Add a BashCommand
02a05d6 
This takes as a single command string and runs its
BB interpolated output as a command.
@grooverdan @RazvanLiviuVarzaru
Add ASAN_SCOPED as Cmake WITH option
7ea3f53
@grooverdan @RazvanLiviuVarzaru
Add FetchGitHub as a downloading function
c0a8b50 
This fetches from a github repository withing a branch and saves a
file to the workdir..
@grooverdan @RazvanLiviuVarzaru
MDBF-741: Add sequence file for sanitizers(asan_ubsan)
ce0cdcb 
asan_ubsan returns a sequence of steps for building and running
a ASAN/UBSAN build.
@grooverdan @RazvanLiviuVarzaru
MDBF-741 Add UBSAN/ASAN builder with/without debug
6bf187e 
from MariaDB-10.11
@grooverdan @RazvanLiviuVarzaru
MDBF-741 10.6 asan/ubsan builders replaced with clang20 asan/ubsan bu…
e2950a0 
…ilders

In versions 10.11+.

Because of unported changes to 10.6, clang-20 needs to start at 10.11.

There's no value in testin on an old version and the clang version is
superiour in coverage compared to the old gcc version.
@grooverdan @RazvanLiviuVarzaru
asan_ubsan: explict ClangCompiler
b7e939e
@grooverdan @RazvanLiviuVarzaru
Add get_mtr_spider_steps helper
9b3a5e4
@grooverdan @RazvanLiviuVarzaru
CompileCMakeCommand: correct verbose
2f08dfc
@grooverdan @RazvanLiviuVarzaru
remove BashCommand import
b7b9907
@grooverdan @RazvanLiviuVarzaru
asan/ubsan no debug builder for now
6e519f9 
codebase need correcting before enabling:
* mroonga mecab - type mismatch
  https://github.com/MariaDB/server/pull/4145/files#diff-8ddee6ac5028513dbaa09ee8fb74308f4fab53b833cf31485fff019a6e971304
* client/mysql.cc
source/client/mysql.cc:3219:16: error: passing an object that undergoes default argument promotion to 'va_start' has undefined behavior [-Werror,-Wvarargs]
 3219 |   va_start(ap, type);
* strings/uca-dump --name-prefix=uca1400 hit leak sanitizer
@grooverdan @RazvanLiviuVarzaru
asan_ubsan; remove disable of embedded_server
3884e0a 
This was the default anyway so no need to do this explicitly.
@grooverdan @RazvanLiviuVarzaru
asan_ubsan: add mtr_junit_reporter
b19adea
@grooverdan @RazvanLiviuVarzaru
asan_ubsan builder: identation
957f7ed
@RazvanLiviuVarzaru
Remove debug variant from constants
f7145cb 
Removed 'amd64-ubasan-clang-20-debug' from the constants list.
@RazvanLiviuVarzaru
Remove empty builders initialization
64bc640 
Revert part of 44c5145 change
@RazvanLiviuVarzaru
Add bind mount for MASTER_PACKAGES_DIR
cc3166b 
Packages mount is required for saving MTR logs
@RazvanLiviuVarzaru
RazvanLiviuVarzaru merged commit 23394b4 into MariaDB:dev Aug 26, 2025
4 checks passed
@grooverdan
grooverdan deleted the MDBF-741-ubsan-seq branch August 26, 2025 21:50
@grooverdan

grooverdan commented Aug 28, 2025

Copy link
Copy Markdown
Member Author

#4265 solving the rpl.rpl_*mixing_engines

@grooverdan grooverdan mentioned this pull request Aug 28, 2025
Closed
5 tasks
@grooverdan

grooverdan commented Aug 28, 2025

Copy link
Copy Markdown
Member Author

MariaDB/server#4266 for roles.acl_load_mutex-5170 -> MDEV-36345

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@RazvanLiviuVarzaru RazvanLiviuVarzaru RazvanLiviuVarzaru approved these changes

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@grooverdan @RazvanLiviuVarzaru