Make the spam score a real control: weights, thresholds and reporting

[2ndkauboy]

Rules::apply() already computes a score. It is a local float that never leaves the method, the numbers it sums are not a scale, and the thresholds that would act on it are inert. This issue is the plan for turning it into something the plugin, its integrations and a Fail2Ban jail can act on.

Not for 3.0.0. Phase 1 below is safe in isolation, but Phase 2 changes detection behaviour and wants the comment corpus in the loop before any of it ships.

Line references are against feat/spam-log-reasons at 731ad35.

Where we are

It never leaves the method

$score is a local float. It goes to the debug log and is then discarded. Rules exposes get_spam_reasons() and get_no_spam_reasons() but no get_score(); CheckResult (#795) carries is_spam, reasons, payload and was_evaluated but no score; PostProcessors::apply() receives only the reason slugs. Nothing downstream — post-processors, the spam log, comment meta, the admin UI, a third-party integration — can see it.

The numbers are not a scale

src/Handlers/Rules.php:172:

$rule_score = $rule::verify( $item ) * $rule::get_weight();

Base::$weight is 1 and no rule in the tree overrides it, so the multiplier is always one. What verify() actually returns:

Value	Rules	Meaning in practice
999	Honeypot, EmptyData, InvalidRequest, LinkbackPostTitleIsBlogName	"definitely spam" — all four are also $is_final
1	BBCode, TooFastSubmit, RegexpSpam, CountrySpam, DbSpam, LangSpam	one signal fired
-1	ValidGravatar	weak ham signal
-100	ApprovedEmail, LinkbackFromMyself	veto — cancels up to 100 spam signals

That is a boolean OR with two vetoes, wearing arithmetic. Summing 1 + 1 to get 2 says "two rules fired", not "twice as spammy". 999 and -100 are sentinels picked to dominate the sum, not measurements.

The thresholds are inert

antispam_bee_no_spam_threshold (src/Handlers/Rules.php:140) and antispam_bee_spam_threshold (:152) both default to 0.0, which makes both guards unreachable, so the decision collapses to return $score > 0.0. They are filter-only: no UI, no per-reaction-type value, and neither filter is passed the reaction type.

One more wrinkle: the final-rule return true at :179 sits above $score += $rule_score at :188, so at the moment of a honeypot verdict the total is still 0.0. Any score reporting has to fix that first, or a honeypot hit reports score=0. The same early return bypasses the thresholds entirely.

What "control" means here

Today the rules vote and any single spam vote wins. As a control, the weight carries how much a rule's opinion is worth, the threshold is where the verdict flips, and both become data a site owner can tune rather than constants in the source.

That is the SpamAssassin model, and two parts of it are worth copying deliberately.

Ship required= wherever you ship score=. SpamAssassin writes score=7.4 required=5.0 tests=… and never the score alone. When the threshold is configurable, a bare score is uninterpretable — score=2 tells a log consumer nothing without the bar it cleared.

The unary bar is not a gimmick. X-Spam-Level: ******* — one asterisk per point — exists because Fail2Ban regexes cannot compare numbers. score=7 gives a jail no way to say "ban only if the score is at least 5"; level=\*{5,} matches trivially. If the score exists partly to drive Fail2Ban policy, the bar is the field that actually enables it.

The uncomfortable half of the comparison: SpamAssassin's per-rule scores are fitted against a labelled corpus, which is what makes the sum calibrated and the threshold meaningful. Ours are hand-picked sentinels. Shipping configurable weights without calibrating them just moves the arbitrariness into the database.

Phase 1 — plumbing, behaviour-neutral

Expose the score and make weights real, while keeping every verdict bit-for-bit identical to today.

• Move $score += $rule_score above the final-rule early return, so a final hit contributes its 999 instead of reporting 0. The verdict is unaffected — apply() already returns true there.
• Rules gains protected $score and get_score(): float, alongside the existing reason getters.
• PostProcessors::apply() takes the score as a fourth argument and sets $item['asb_score']. Both call sites pass it: Reaction::handle_spam() (src/Handlers/Reaction.php:98) and SpamCheck::post_process() (src/Api/SpamCheck.php:172).
• CheckResult gains get_score() and get_required(), so integrations see what core sees.
• Weights move from a hard-coded 1 to per-rule defaults that reproduce today's effective values exactly.

Gate: the corpus comparison tool reports zero verdict changes across all ~332,000 comments. Any change at all means the plumbing is wrong, not that the scoring improved.

Phase 2 — calibration, changes verdicts

Only once Phase 1 is in and quiet.

• Separate signal from magnitude. verify() should answer "did this rule fire?" (1/0, or -1 for a ham signal) and the weight should carry what that is worth. Today verify() conflates the two, which is the whole reason 999 exists.
• Decide what the vetoes become. -100 is not a weight, it is "this cannot be spam". That is arguably is_final in the ham direction, and modelling it as a large negative number is what forces the scale to be wide and meaningless.
• Fit the weights against the corpus; make the threshold the tuning knob.
• Surface weights and threshold in the admin UI, per reaction type.

Gate: every step carries its own corpus run. This is where the real work — and all of the risk — lives.

Reporting the score

Once Phase 1 lands, the log line gains the fields while keeping the key=value shape #797 established:

2026-01-15T10:23:45+01:00 ip=192.0.2.42 type=comment post=474 score=7 required=5 level=******* reasons=asb-honeypot,asb-regexp

level needs a hard cap on its width — a 999 from a final rule would otherwise write a kilobyte of asterisks into a file that drives firewall bans. The cap is a requirement, not a cosmetic choice.

Beyond the log, the score is worth storing as comment meta next to antispam_bee_reason, so it can be shown in the comment list and queried after the fact rather than only at detection time.

What to reuse

Most of the machinery exists; the implementation should lean on it rather than invent a parallel one.

• Storage and read path. Settings::get_option( $name, $reaction_type ) over the two-level antispam_bee_options array (src/Helpers/Settings.php:19), keyed by ControllableBase::get_option_name( 'weight' ) (src/Rules/ControllableBase.php:51) — giving rule_asb_bbcode_weight inside the reaction-type bucket. No new option row.
• Numeric input. There is no Number field class, but there is a working precedent: DeleteOldSpam injects a Text field with 'input_type' => 'number' into a checkbox label via the inline field type (src/GeneralOptions/DeleteOldSpam.php:66).
• Persistence. Automatic through Sanitize::sanitize_controllables() once an option declares a sanitize callable — including the nested input of an inline option.
• Generating the field once. Section already auto-generates the on/off checkbox for every controllable; a weight box added in the same place gives every rule one for free, instead of editing thirteen get_options() methods.
• Thresholds. Rules already holds $this->reaction_type, so a per-tab threshold option reads naturally at the existing filter sites and can be passed in as the filter default, keeping both hooks intact.

Gaps that have to be built

• get_weight() is declared : int in both Verifiable (:35) and Base (:100). PHP return types are invariant, so widening to float breaks any third-party rule declaring : int. Either fractional weights wait for 4.0, or the scale stays in whole points and simply uses a wider range — the cheaper answer, at negligible cost in precision.
• get_weight() takes no $reaction_type, but options are stored per reaction type. Either the signature gains the parameter (touching the interface, the base class and the call site, which does have the type to hand) or weights are stored reaction-type-agnostically.
• Defaults are not merged. Settings::$defaults is only the fallback for the whole option when the row is absent; once a user saves, it never applies again. A per-rule default weight therefore needs a fallback in the getter, not a defaults entry.
• No default-value support in the field layer, and min/max/step are not emitted by the text field. A sanitizer returning null deletes the key, so a numeric sanitizer must return an int — 0 is fine, null is not.

Open questions

1. What does "offset" mean? This proposal reads it as the threshold the total is compared against, SpamAssassin's required_score. A per-rule additive offset would be a different mechanism and needs its own justification.
2. Vetoes as weights, or as finality? Keeping -100 as a number preserves today's behaviour exactly. Converting it to a ham-direction is_final is cleaner but changes semantics in the case where a veto currently loses to a 999.
3. Per-reaction-type weights, or global weights with a per-type threshold? The latter is far less configuration surface for nearly the same expressive power.
4. Does the score belong in the UI at all? It could stay a constant- and filter-level control for people who know what they are doing. Exposing an uncalibrated scale to every site owner invites support load — and calibration is Phase 2, not Phase 1.

Related

• feat: log the spam reasons in the spam log file #797 — the key=value spam log format this would extend
• feat: resolve both log files from WP_DEBUG_LOG-shaped constants #807 — spam log constants, stacked on feat: log the spam reasons in the spam log file #797
• feat: add a public spam check API for third-party integrations #795 — the public spam check API that CheckResult belongs to