Skip to content

SKILL.md

Latest commit

 

History

History
47 lines (36 loc) · 3.45 KB

SKILL.md

File metadata and controls

47 lines (36 loc) · 3.45 KB
Raw
name tournament-judge
description Runs a domain-neutral, evidence-based tournament in which independent candidates are reviewed, anonymized, and scored by a blind judge against a rubric fixed before judging. Use when comparing competing plans, designs, implementations, documents, tools, vendors, strategies, or other substantial options; when the user asks for a tournament, bake-off, best-of-N, adversarial comparison, or independent judging. Do not use for trivial choices, factual lookups, or decisions that cannot be evaluated safely with the available evidence.
license MIT
compatibility Agent Skills compatible assistants; parallel subagents recommended but not required
metadata
version workflow
1.0.0
blind-multi-candidate-tournament

Tournament Judge

Run a fair comparison, not a staged argument for a favorite.

Non-negotiable rules

  1. Freeze requirements, constraints, evidence rules, and the scoring rubric before candidates are generated.
  2. Produce 3 independent candidates by default. Use 2 to 5 only when cost or breadth justifies it.
  3. Keep candidates isolated until all are final. Do not let one team borrow another team's work.
  4. Give each candidate the same inputs, budget, tools, review depth, and output contract.
  5. Separate proposal, adversarial review, and judging roles when the runtime permits.
  6. Anonymize candidates before judging. Remove names, authorship clues, debate history, and ordering signals that do not affect quality.
  7. Score the artifact and its evidence, not confidence, prose polish, or model identity.
  8. Preserve disqualifiers. A weighted score cannot rescue a candidate that violates a hard constraint.
  9. Select one winner only if at least one candidate passes every hard gate and the minimum quality threshold. Otherwise report no acceptable winner.
  10. Never fabricate measurements, tests, citations, prices, or user preferences.

Run the workflow

  1. Read modules/intake.md. Choose either supplied-candidate judging or a generated tournament, then collect missing decision inputs.
  2. Read modules/rubric.md. Create and freeze a domain-specific rubric whose weights total 100.
  3. Read modules/orchestration.md. Generate candidates and run equal adversarial reviews.
  4. Load the role prompt matching each role from prompts/.
  5. Read modules/blind-judging.md. Anonymize, validate, score, compare, and select the winner.
  6. Fill templates/final-report.md using modules/reporting.md.

For a fast, lower-cost run, judge supplied candidates or use 3 candidates and 1 review round. For consequential or hard-to-reverse decisions, use 3 candidates, 2 review rounds, 2 blind judges, and verification with domain tools. If judges disagree on the winner, use a fresh adjudicator with both reasoned verdicts.

Runtime fallbacks

  • Parallel subagents available: run candidate teams concurrently, then start a fresh judge with only the frozen brief, rubric, and anonymized finalists.
  • Only serial agents available: save each team artifact separately and reset to the frozen brief before starting the next team. Start judging in a fresh context if possible.
  • No subagents available: emulate roles in clearly separated passes. State that independence and blindness are weaker.

Safety boundary

This skill structures judgment but does not replace qualified review. For medical, legal, financial, security-critical, or safety-critical decisions, require appropriate expert validation and treat the tournament as decision support only.