[OPIK-8186] opik-instrument: eval harness (verify coverage + no false success)

[aswynz]

What

Adds an eval harness for the opik-instrument skill — it had none (unlike opik-diagnose / opik-explain), so the verify-coverage ability added in OPIK-8185 (#173) was untested beyond CI spec/build checks. Mirrors the opik-diagnose/evals/ layout.

Cases

• Triggering — "add opik tracing" selects opik-instrument, not diagnose/explain/evaluate (judge panel → selection_accuracy).
• Functional / clean — an uninstrumented app (entrypoint → LLM → tool). The skill must instrument, run, and report verified with span-level coverage (all three types present, every span well-formed, code changed).
• Functional / missing_flush (adversarial) — an already-instrumented app with the flush removed, so no complete trace lands. Decisive check no_false_success: the skill must not claim verified / already_verified without a real, complete trace behind it. Passes whether it fixes the flush and verifies, or blocks with a flush next-step; a skill that trusts the decorators and reports success fails.

Run it

cd src/opik_mcp/skills/opik-instrument/evals
uv run --with pyyaml python run_evals.py prepare   # stage fixture apps under _work/
# run /opik-instrument in each _work/<case> dir; it writes result.json there
uv run --with pyyaml python run_evals.py grade     # deterministic scoring

Grading is offline; an optional integrity re-read hits Opik only when a trace_id is present.

Self-tested: correct outputs pass 2/2; a false already_verified with no trace is caught by no_false_success.

Follow-up to #173. HARNESS.md notes a planned partial_trace fixture (batching-race arrives-but-incomplete) as a TODO.

Closes OPIK-8186.

🤖 Generated with Claude Code