abstention

Algorithms for abstention, calibration and domain adaptation to label shift.

A decision-safety lab for loan approval: trains a baseline classifier, calibrates probabilities (ECE/Brier), sweeps confidence thresholds to build a coverage, quality frontier and outputs a defensible abstention policy (auto-decide vs review). Includes a Streamlit dashboard for report cards, triage UI, and data quality checks.

A practical framework for turning data analysis into decision policies you can defend. Covers risk modeling, thresholding, exception handling, policy cards, monitoring, and update triggers, using real patterns like abstention rules, reorder points, and fairness-aware benchmarking. Built for “ship it” data science.

Longform article reframing abstention (reject option / selective prediction) as product design, not model weakness. Covers coverage as a KPI, calibration as a prerequisite, threshold selection under review capacity and risk, queue/UX design for human-in-the-loop workflows, and anti-patterns that break safety in production.

Decision-safe evaluation + Streamlit dashboard for AI vs Human vs Post-Edited AI text detection. Generates a reliability report card (Accuracy, Macro F1, ECE, Brier), calibration plots, confidence histograms, and a coverage-vs-performance abstention curve. Recommends an operating threshold for human-review routing.

Extracts every assumption your coding agent made, links each one to the code it justifies, and tells you which ones broke. Stdlib only.

An editable, auditable 807K-param byte-level LLM: CRUD single facts with provable per-edit locality, and abstain when unsure instead of guessing. CPU, offline.

Behavioral Trust Clustering a thermodynamic governance layer that reduces LLM hallucination by 52% on HumanEval. Drop-in wrapper for any decoder. MIT.

Dynamic latent-state control heads for LLMs: route each query by actual model capability, not task type, to answer, reason, call tools, abstain, or escalate

TypeScript reference lab for evidence-bounded answers that cite, abstain or downgrade instead of hallucinating.

Memory that abstains instead of guessing: agent memory on your own Postgres with a verdict, confidence and provenance on every hit, and a calibrated refusal when nothing clears the threshold.

We show that a model owner can artificially introduce uncertainty into their model and provide a corresponding detection mechanism.

Reliable medical QA with Mistral-7B, QLoRA, selective prediction, and learned abstention via warm-start SFT + DPO.

Accuracy is not readiness: an open-source robustness stress-test of frontier LLMs (Opus 4.8, GPT-5.5, Grok 4.3, Gemini 3.5 Flash) on medical QA. Re-implements & extends Gu et al. 'The Illusion of Readiness in Health AI'.

Proper scoring rules, reduces LLM overconfidence in multiple-choice QA.

RAG service that treats abstention as a feature: cited answers, a faithfulness gate, and a measured coverage-vs-false-answer curve. Chunking x retriever evaluation grid vs planted ground truth shows why retrieval metrics alone mislead. From-scratch BM25, LSA + RRF hybrid, FastAPI, MLflow, 31 tests, fully offline CI.

probegate

Calibrated trust and abstention for MS/MS molecular annotations

A deterministic honesty layer for LLM agents + a reproducible TruthfulQA eval that measures calibration honestly — reporting the wins and the costs.

A tiny, offline check for abstention in text-to-SQL and RAG-over-warehouse assistants: does it say it cannot answer instead of returning a confident wrong number? No model access, no network.