Commit d131b82
committed
Release v3.4.0: serve Qwen's sentiment, and stop implying the panel is evenly covered
qwen3-8-27b joined the generation-2 annotation campaign on 2026-08-25, written
from a self-hosted vLLM run rather than a vendor API, and its columns reached the
public dataset the same morning. Adding it to SENTIMENT_MODELS is three lines.
The rest of this commit exists because Qwen is the first panel member that does
not cover the corpus.
It scores 12,098 articles where the other four score 12,298. The 200-article gap
is deliberate and final rather than a run to repair: each was attempted four
times and then retired. It is also not missing at random. Measured against Luna's
centrality, 6.15% of Marginal articles are unscored against 0.96% of Central and
0.99% of Tres central ones, because the model declines to place a subjectivity
label where Islam is peripheral while the prompt licenses declining only where
Islam is absent.
Two things in the server assumed uniform coverage without ever saying so.
agreement.pairwise is measured on the base of articles every listed model scored,
which is what makes the pairs comparable with each other. Adding a fifth model
therefore changed every pre-existing pairwise number, not only Qwen's: the base
fell from 12,298 to 12,098 and unanimity on polarity from 36.5% to 32.5%. The
common base is kept, because per-pair bases would not be comparable, and it is
disclosed instead. agreement.base states what it counts and agreement.base_caveats
names the short member and why.
rowsToMap drops the empty key, so a model that answered 200 fewer articles
produced a distribution indistinguishable from one that answered all of them.
Every model block now carries coverage {polarity, centrality, subjectivity,
matched_articles}, summed from the distributions already computed rather than
from fresh SQL.
Describing a model stays the registry's job. SentimentModel gains an optional
caveat, so no call site hardcodes "qwen" or "200". It travels into the payload as
model_caveat beside that model's own numbers, and into the agreement block for
any model that shrinks its base.
The chart said "All N scored the same articles", which was now false. The
comparison view captions only a ring falling short of the panel's best coverage,
five identical denominators being noise, and swaps that note for one naming the
short members and the reduced base.
Every figure in the docs was re-measured against the live parquet rather than
carried forward. Subjectivity kappa runs 0.16 to 0.52 across the five, where the
ceiling is one pair, luna and qwen; centrality 0.45 to 0.74; and on polarity Qwen
is now the closest pair to Luna at 0.54, ahead of Gemma's 0.50. Mistral remains
the systematic outlier on all three scales and Qwen joined the non-Mistral
cluster, so a majority on centrality is still the others outvoting Mistral. Qwen
is also the panel's most reluctant to reach for an extreme: 176 Tres positif and
5 Tres negatif against Luna's 425 and 45.
gpt-5-6-luna stays the default, and Qwen made that the closest call so far by
agreeing with Luna more than any other pair on two of the three scales. The
200-row hole is the answer: promoting it would put a silent gap in every inline
polarity this server returns.
Both qwen and alibaba resolve. qwen resolves where gemini is still refused, and
the difference is the point of the registry: the Qwen line scored generation 2,
the Gemini line never did.
The fixture leaves Qwen entirely empty on row 104, the low-centrality one, so
scored_by_all (5) is smaller than the matched set (6). A fixture where the new
model is scored everywhere could not fail when the shrinking base is broken. The
live smoke test asserts the shortfall against the real columns, since only live
data can prove the server discloses it.
Token footprint moves 14,880 to 15,071 of a 16,000 ceiling. 112 of that 191 is
v3.3.0's instructions growth, which was never re-baselined; this change costs 79
on get_sentiment_distribution. The baseline is re-stamped at the current figure.
The combined scoping tool and multi-term keyword probing that v3.3.0's note
assigned to v3.4.0 remain open.1 parent 3539885 commit d131b82
17 files changed
Lines changed: 335 additions & 109 deletions
File tree
- .agents/skills/iwac-mcp
- references
- mcpb
- scripts
- src
- app/views
- tools
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
180 | 180 | | |
181 | 181 | | |
182 | 182 | | |
183 | | - | |
| 183 | + | |
184 | 184 | | |
185 | 185 | | |
186 | 186 | | |
| |||
215 | 215 | | |
216 | 216 | | |
217 | 217 | | |
218 | | - | |
| 218 | + | |
219 | 219 | | |
220 | 220 | | |
221 | 221 | | |
| |||
242 | 242 | | |
243 | 243 | | |
244 | 244 | | |
245 | | - | |
| 245 | + | |
246 | 246 | | |
247 | 247 | | |
248 | 248 | | |
| |||
0 commit comments