[config] Checker label fixes and 6.29 profile update - #5006
Conversation
Three misspellings left the affected checkers without a severity and outside the CWE Top 25 guideline, with nothing to signal it: a misspelled key is silently ignored, and so is a guideline name that descriptions.json does not define. alpha.cplusplus.MisusedMovedObject carried only a doc url. It is the pre-rename name of cplusplus.Move, and we still support the clang versions that only have the old name, so it takes that entry's mapping. cplusplus.Move itself had no doc url at all, so it gets one. alpha.core.StdVariant also carried only a doc url. Retrieving an inactive alternative with std::get throws std::bad_variant_access, so a true positive means the code will fail at run time, which is HIGH by the descriptions.json definition. No profile, since it is alpha. cppcheck-redundantContinue was in default but in neither sensitive nor extreme. The profiles are nested by definition.
Removed checkers from the default profile that are too noisy according to our labeling criteria in scripts/llm-scripts/checker_labeling_instructions.md, and the noisiest of those from sensitive as well. security.insecureAPI.strcpy goes the other way and gains sensitive, which is the band its report count puts it in. clang-diagnostic-format-nonliteral is held in default despite crossing the 30x median threshold. Nearly all of its new reports are the vim gettext idiom, and the high number of reports seems to be a result of a clang regression around __attribute__((format_arg)), so the count is not representative. clang renamed alpha.unix.cstring.UninitializedRead to unix.cstring.UninitializedRead since the last release. Add an entry for the new name, and give the alpha one the same EXP33-C mapping, since we still support the clang versions that only have it.
The config coverage job reports 18 checkers the analyzers know about but our label files do not. Without an entry they show up as UNSPECIFIED severity and are invisible to every profile and guideline query. Severities follow the CodeChecker severity definitions. Where a new checker is a move or an alias of one we already label, it inherits that entry's labels, so the same check cannot end up carrying two different severities. Profiles are deliberately left empty: none of these checkers existed in the analyzers used for the release measurement, so there is no report count to weigh against the median that our profile policy is based on. They can be placed once a run covers them.
7784ccd to
d243341
Compare
These tests analyse with the default profile and hard-code the resulting report counts and hashes, so dropping deadcode.DeadStores from default broke 41 of them across the analyzer and web suites. Enabling the checker explicitly reproduces the previous report sets exactly and makes the expectations independent of profile membership, which is evaluation-driven data that moves every release. multi_error.en1 already pins it the other way, with an explicit --disable. multi_error_suppress and multi_error_suppress_cstyle kept passing, but the only report their suppress comment covers is the dead store, so without the checker they assert nothing. Pinned for the same reason.
d243341 to
8dc8be0
Compare
barnabasdomozi
left a comment
There was a problem hiding this comment.
Currently, there is a merge conflict. Please rebase to master.
This reverts commit 8dc8be0.
dkrupp
left a comment
There was a problem hiding this comment.
I am afraid that with the current methodology we are moving checkers from and to the profiles back and forth. Let's label up only the new checkers now and start recategorizing the checkers only after the labeling methodology is better developed in a folloup PR.
| "doc_url:https://clang.llvm.org/docs/DiagnosticsReference.html#wconditional-uninitialized", | ||
| "guideline:sei-cert-c", | ||
| "label-tool-skip:severity", | ||
| "profile:default", |
There was a problem hiding this comment.
this looks like a pretty important check for me. using an unitialized variable in a condition... is it that noisy really?
There was a problem hiding this comment.
I have reverted all profile changes for existing checkers.
| ], | ||
| "alpha.unix.cstring.UninitializedRead": [ | ||
| "doc_url:https://clang.llvm.org/docs/analyzer/checkers.html#alpha-unix-cstring-uninitializedread-c", | ||
| "guideline:sei-cert-c", |
There was a problem hiding this comment.
let's not add alpha checker to the sei cert profile, becuase we dont' recommend to run them in production.
There was a problem hiding this comment.
Removed guidelines from alpha checkers.
Until we ensure that the median-based profile categorization does not make the checkers oscillate between profiles, we are not modifying the existing checkers.
Alpha checkers are not recommended for production use, so they should not appear in SEI CERT or CWE guideline mappings.
clang renamed alpha.unix.cstring.UninitializedRead to
unix.cstring.UninitializedRead since the last release. Add an entry for
the new name, and give the alpha one the same EXP33-C mapping, since we
still support the clang versions that only have it.
Three misspellings left the affected checkers without a severity and
outside the CWE Top 25 guideline, with nothing to signal it: a
misspelled key is silently ignored, and so is a guideline name that
descriptions.json does not define.
alpha.cplusplus.MisusedMovedObject carried only a doc url. It is the
pre-rename name of cplusplus.Move, and we still support the clang
versions that only have the old name, so it takes that entry's mapping.
cplusplus. Move itself had no doc url at all, so it gets one.
alpha.core.StdVariant also carried only a doc url. Retrieving an
inactive alternative with std::get throws std::bad_variant_access, so a
true positive means the code will fail at run time, which is HIGH by the
descriptions.json definition.
cppcheck-redundantContinue was in default but in neither sensitive nor
extreme. The profiles are nested by definition.
Eighteen checkers new in clang-tidy and cppcheck had no entry at all,
added those as well.