↗ Driftproof · Writing · Contributions · Get in touch
I’m Maverick, building Driftproof and contributing to the tools that evaluate AI models and agents.
I work on a deceptively simple question: did the system actually measure what its score says it measured? That has led me from missing execution evidence to inflated metrics, stale grading, and results attributed to the wrong model.
Agent skills change. Models change. Their evaluations should keep up.
Driftproof runs tasks with and without a skill, samples answers and judge scores, and writes dated, hash-verified receipts. It gives you results to inspect and compare when the model underneath a skill changes.
|
Evaluate a skill against a baseline and inspect the evidence behind the result. Explore the source → |
Published runs, measurement limits, and what changed when the instrument was corrected. Three model releases later → |
Selected merged contributions to projects I use and study.
| Where | Why the fix matters |
|---|---|
| NVIDIA · SkillEvaluator | Reading a script must not count as running it. Require invocation evidence ↗ |
| ModelScope · EvalScope | Unrecognized answers must not disappear from recall and F1 denominators. Correct PubMedQA scoring ↗ |
| MLflow | Relative score checks must handle negative baselines correctly. Use the baseline’s magnitude ↗ |
| Alibaba · skill-up | Tool argument rules need the argument structure preserved. Fix judge matching ↗ |
| Addy Osmani · agent-skills | Grades must match declared expectations and the current run. Bind expectation IDs ↗ · Clear stale grading ↗ |
More merged work in agent-skills
|
modelscope/evalscope #1828 alibaba/skill-up #304 mlflow/mlflow #26252 |
BerriAI/litellm #45834 stanfordnlp/dspy #10474 microsoft/vscode #340784 |
Working on agent skills or evaluation infrastructure? I’m interested in reproducible failures, useful tools, and getting the measurement right.
hello@driftproofhq.com · driftproofhq.com
Contribution statuses checked October 10, 2026.