Removing race alone barely moves the number, because CustodyStatus - a record of prior system contact - carries the same signal: communities that were over-policed historically generate more "custody status" records today, independent of actual reoffense risk. Only after race and CustodyStatus are both dropped does the gap fall to 15.69%, a 71% reduction. The model was never neutral. It was a faithful record of an unequal criminal justice system, expressed as a probability.
Removing race alone barely moves the number, because CustodyStatus - a record of prior system contact - carries the same signal: communities that were over-policed historically generate more "custody status" records today, independent of actual reoffense risk. Only after race and CustodyStatus are both dropped does the gap fall to 15.69%, an 82% reduction. The model was never neutral. It was a faithful record of an unequal criminal justice system, expressed as a probability.
This function checks whether a model's outcome rates differ across protected groups by more than a configurable threshold, and flags features that correlate with the protected attribute strongly enough to explain the gap on their own.
import pandas as pd
diff --git a/explainers/ai-objectivity-myth.md b/explainers/ai-objectivity-myth.md
index e98cbbe..ee9c242 100644
--- a/explainers/ai-objectivity-myth.md
+++ b/explainers/ai-objectivity-myth.md
@@ -40,7 +40,7 @@ model.fit(X_train, y_train)
# Fairness Gap: 86.77%
```
-Removing `race` alone barely moves the number, because `CustodyStatus` - a record of prior system contact - carries the same signal: communities that were over-policed historically generate more "custody status" records today, independent of actual reoffense risk. Only after `race` *and* `CustodyStatus` are both dropped does the gap fall to 15.69%, a 71% reduction. The model was never neutral. It was a faithful record of an unequal criminal justice system, expressed as a probability.
+Removing `race` alone barely moves the number, because `CustodyStatus` - a record of prior system contact - carries the same signal: communities that were over-policed historically generate more "custody status" records today, independent of actual reoffense risk. Only after `race` *and* `CustodyStatus` are both dropped does the gap fall to 15.69%, an 82% reduction. The model was never neutral. It was a faithful record of an unequal criminal justice system, expressed as a probability.
## Detection Code
diff --git a/explainers/confounding-variable.html b/explainers/confounding-variable.html
index 420cb4d..b488406 100644
--- a/explainers/confounding-variable.html
+++ b/explainers/confounding-variable.html
@@ -222,7 +222,7 @@ Concrete Example: COMPAS and
Both arrows are caused by the same systemic factor. CustodyStatus is associated with the recidivism label not because custody history is a reliable individual risk signal, but because the same structural forces that produce elevated custody records also produce more recidivism label events - through monitoring, not through behavior.
Removing both race and CustodyStatus breaks this path:
# fair.py: features include only priors_count + charge_degree + age
-# Black/White fairness gap: 15.69% - 71% reduction
+# Black/White fairness gap: 15.69% - 82% reductionThe residual 15.69% reflects other confounding paths (differential bail rates, charge severity distributions, surveillance-driven label noise) that require changes upstream of the model to eliminate entirely.
1. Stratified analysis cannot distinguish confounding from effect modification. If a feature has a genuinely different causal effect on the outcome across strata - not just a different baseline - that is effect modification, not confounding. The two require different handling. Conflating them produces wrong adjustments.
2. You can only condition on observed confounders. If the confounder is unmeasured - historical policing intensity, neighbourhood-level surveillance, differential healthcare access - no statistical adjustment removes its effect. Causal inference methods (instrumental variables, propensity score matching, difference-in-differences) can partially address unmeasured confounding but require strong, often untestable assumptions about the causal structure.
3. Conditioning on a collider opens new bias. A collider is a variable caused by both the feature and the outcome - the reverse of a confounder. Controlling for a collider introduces a spurious association rather than removing one. Correctly distinguishing confounders from colliders requires a causal graph (a DAG), not statistical testing alone. Chi-squared tests cannot tell you which direction the arrows point.
-4. Confounder removal reduces but does not eliminate bias. Removing CustodyStatus from COMPAS cuts the fairness gap from 86.77% to 15.69% - a 71% reduction. The remaining gap reflects additional confounding paths that cannot be closed by feature removal without changing the label generation process itself.
4. Confounder removal reduces but does not eliminate bias. Removing CustodyStatus from COMPAS cuts the fairness gap from 86.77% to 15.69% - an 82% reduction. The remaining gap reflects additional confounding paths that cannot be closed by feature removal without changing the label generation process itself.
5. Adjustment can introduce its own distortions. Propensity score methods and inverse probability weighting reduce confounding but amplify variance, especially in small subgroups. In high-stakes settings, an overcorrected model may perform worse for the groups it was adjusted to protect.
When a confounded model is deployed and its outputs influence future labels - recidivism surveillance, credit monitoring, healthcare resource allocation - the confounding strengthens across retraining cycles. The model's outputs become part of the data-generating process, reinforcing the spurious association with each iteration. See feedback-loop-bias.md.
COMPAS/ - the primary worked example. CustodyStatus confounds the race→recidivism path, driving 71% of the Black/White fairness gap. Removing it alongside race reduces the gap from 86.77% to 15.69%.Benefits Denial/ - relationship and marital-status act as confounders for sex: historical gender norms independently elevated male-coded relationship statuses and income levels in the census data, creating a spurious association the model amplifies.Healthcare Readmission/ - payer_code is confounded by race: differential insurance access is caused by structural factors that also independently predict readmission risk, not only by individual health status.COMPAS/ - the primary worked example. CustodyStatus confounds the race→recidivism path, driving 82% of the Black/White fairness gap. Removing it alongside race reduces the gap from 86.77% to 15.69%.Benefits Denial/ - relationship and marital-status act as confounders for sex: historical gender norms independently elevated male-coded relationship statuses and income levels in the census data, creating a spurious association the model amplifies.Healthcare Readmission/ - payer_code is confounded by race: differential insurance access is caused by structural factors that also independently predict readmission risk, not only by individual health status.The COMPAS audit in this repository demonstrates a proxy form of target leakage. CustodyStatus - a feature describing whether a defendant is currently in custody - correlates with race at p < 0.001 and with the recidivism label at p < 0.001. It is not a neutral operational variable. It encodes the outcome of a prior criminal justice interaction, which itself encodes historical over-policing of Black communities. Including it in the model achieves an 86.77% Black/White fairness gap in positive prediction rates.
This is the structural form of leakage: a feature that looks like a legitimate input but whose predictive power comes from downstream effects of the very outcome being predicted.
-Removing CustodyStatus alongside the protected race attribute reduces the gap to 15.69% - a 71% reduction.
Removing CustodyStatus alongside the protected race attribute reduces the gap to 15.69% - an 82% reduction.
import os
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
diff --git a/explainers/data-leakage.md b/explainers/data-leakage.md
index f11f86c..bb9a366 100644
--- a/explainers/data-leakage.md
+++ b/explainers/data-leakage.md
@@ -47,7 +47,7 @@ The COMPAS audit in this repository demonstrates a proxy form of target leakage.
This is the structural form of leakage: a feature that looks like a legitimate input but whose predictive power comes from downstream effects of the very outcome being predicted.
-Removing `CustodyStatus` alongside the protected race attribute reduces the gap to 15.69% - a 71% reduction.
+Removing `CustodyStatus` alongside the protected race attribute reduces the gap to 15.69% - an 82% reduction.
```python
import os
diff --git a/explainers/equalized-odds.html b/explainers/equalized-odds.html
index 86d5ddb..5b50a14 100644
--- a/explainers/equalized-odds.html
+++ b/explainers/equalized-odds.html
@@ -287,7 +287,7 @@ Proxy Variables
Even after correcting for Equalized Odds, models can still encode protected attributes through proxy variables - features like zip code, custody status, or employment tenure that correlate with race, age, or class in the training data. Equalized Odds doesn't detect these; proxy variable auditing does. See the proxy variables explainer.
Related Projects in This Repo
-COMPAS/ - Full COMPAS analysis: biased model → fair model → 71% gap reduction. Equalized Odds violations visible in the raw model outputs.explainers/proxy-variables.md - Why AI stays biased even after you remove protected attributesexplainers/shap-values.md - How to see exactly what drove an AI decision - and use that to catch bias
+COMPAS/ - Full COMPAS analysis: biased model → fair model → 82% gap reduction. Equalized Odds violations visible in the raw model outputs.explainers/proxy-variables.md - Why AI stays biased even after you remove protected attributesexplainers/shap-values.md - How to see exactly what drove an AI decision - and use that to catch bias
Further Reading
- Hardt, Price, Srebro (2016): Equality of Opportunity in Supervised Learning - the paper that formally defined Equalized Odds
- Chouldechova (2017): Fair Prediction with Disparate Impact - the mathematical proof that calibration and Equalized Odds conflict when base rates differ
- Fairlearn documentation
- ProPublica: Machine Bias (2016) - the original COMPAS investigation
diff --git a/explainers/equalized-odds.md b/explainers/equalized-odds.md
index 50cbeb0..706d756 100644
--- a/explainers/equalized-odds.md
+++ b/explainers/equalized-odds.md
@@ -184,7 +184,7 @@ Even after correcting for Equalized Odds, models can still encode protected attr
## Related Projects in This Repo
-- [`COMPAS/`](../COMPAS/) - Full COMPAS analysis: biased model → fair model → 71% gap reduction. Equalized Odds violations visible in the raw model outputs.
+- [`COMPAS/`](../COMPAS/) - Full COMPAS analysis: biased model → fair model → 82% gap reduction. Equalized Odds violations visible in the raw model outputs.
- [`explainers/proxy-variables.md`](proxy-variables.md) - Why AI stays biased even after you remove protected attributes
- [`explainers/shap-values.md`](shap-values.md) - How to see exactly what drove an AI decision - and use that to catch bias
diff --git a/explainers/how-ai-detects-patterns.html b/explainers/how-ai-detects-patterns.html
index 40a7920..43c1f50 100644
--- a/explainers/how-ai-detects-patterns.html
+++ b/explainers/how-ai-detects-patterns.html
@@ -219,7 +219,7 @@ Concrete Example: COMPAS - Audit 01
When race is dropped, the gap barely moves - CustodyStatus alone reconstructs most of the pattern, because it correlates with race at 0.31 importance even on its own. Only when both race and CustodyStatus are removed does the gap fall to 15.69%, a 71% reduction. The pattern the model was detecting was never really about behavior - it was about which neighborhoods got patrolled.
When race is dropped, the gap barely moves - CustodyStatus alone reconstructs most of the pattern, because it correlates with race at 0.31 importance even on its own. Only when both race and CustodyStatus are removed does the gap fall to 15.69%, an 82% reduction. The pattern the model was detecting was never really about behavior - it was about which neighborhoods got patrolled.
The following functions inspect what patterns a trained model relied on, and flag features whose importance may be inflated by correlation with a protected attribute.
import pandas as pd
diff --git a/explainers/how-ai-detects-patterns.md b/explainers/how-ai-detects-patterns.md
index 229b1bc..30b73da 100644
--- a/explainers/how-ai-detects-patterns.md
+++ b/explainers/how-ai-detects-patterns.md
@@ -52,7 +52,7 @@ model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)
```
-When `race` is dropped, the gap barely moves - `CustodyStatus` alone reconstructs most of the pattern, because it correlates with `race` at 0.31 importance even on its own. Only when both `race` and `CustodyStatus` are removed does the gap fall to 15.69%, a 71% reduction. The pattern the model was detecting was never really about behavior - it was about which neighborhoods got patrolled.
+When `race` is dropped, the gap barely moves - `CustodyStatus` alone reconstructs most of the pattern, because it correlates with `race` at 0.31 importance even on its own. Only when both `race` and `CustodyStatus` are removed does the gap fall to 15.69%, an 82% reduction. The pattern the model was detecting was never really about behavior - it was about which neighborhoods got patrolled.
## Detection Code
diff --git a/explainers/ml-bias.html b/explainers/ml-bias.html
index c19e448..8def3cb 100644
--- a/explainers/ml-bias.html
+++ b/explainers/ml-bias.html
@@ -228,7 +228,7 @@ Concrete Example: COMPAS - Audit 01Labels: The label is is_recid - whether a defendant reoffended within two years. Reoffending is measured by re-arrest, not by actual criminal behaviour. In a jurisdiction where Black defendants are more likely to be stopped, searched, and arrested for equivalent behaviour, re-arrest is a biased label. The model trains to predict re-arrest and produces a proxy for policing intensity, not criminal propensity.
Proxy variable: Custody status correlates with race because pretrial detention patterns reflect unequal bail access, which reflects income inequality, which is racially structured. Removing race while leaving custody status in the model leaves most of the racial signal intact.
Feedback loop: COMPAS scores influence bail and sentencing decisions. A defendant flagged as high-risk is more likely to be detained pretrial. Pretrial detention increases the probability of conviction and reoffending (due to job loss, housing instability, and network effects). The model produces the outcome it predicted, and the outcome enters the next training set as evidence the prediction was correct.
-The biased model flags Black defendants as high-risk at 86.77%. The fair model - with race and custody status removed - reduces that gap to 15.69%. A 71% reduction from one targeted intervention on one proxy variable.
+The biased model flags Black defendants as high-risk at 86.77%. The fair model - with race and custody status removed - reduces that gap to 15.69%. An 82% reduction from one targeted intervention on one proxy variable.
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
diff --git a/explainers/ml-bias.md b/explainers/ml-bias.md
index b3ac0f2..5ae0b0b 100644
--- a/explainers/ml-bias.md
+++ b/explainers/ml-bias.md
@@ -76,7 +76,7 @@ The COMPAS audit is the sharpest illustration of how all four entry points conve
**Feedback loop:** COMPAS scores influence bail and sentencing decisions. A defendant flagged as high-risk is more likely to be detained pretrial. Pretrial detention increases the probability of conviction and reoffending (due to job loss, housing instability, and network effects). The model produces the outcome it predicted, and the outcome enters the next training set as evidence the prediction was correct.
-The biased model flags Black defendants as high-risk at 86.77%. The fair model - with race and custody status removed - reduces that gap to 15.69%. A 71% reduction from one targeted intervention on one proxy variable.
+The biased model flags Black defendants as high-risk at 86.77%. The fair model - with race and custody status removed - reduces that gap to 15.69%. An 82% reduction from one targeted intervention on one proxy variable.
```python
import pandas as pd
diff --git a/explainers/proxy-variables.html b/explainers/proxy-variables.html
index fabf1c8..0a7551a 100644
--- a/explainers/proxy-variables.html
+++ b/explainers/proxy-variables.html
@@ -230,7 +230,7 @@ What We Did
Results:
Group High-Risk Flag Rate Black defendants 84.71% White defendants 69.02% Fairness gap 15.69%
Summary
-Approach Fairness Gap Reduction Biased model 86.77% - Remove race only ~80%+ Minimal Remove race + proxy 15.69% 71%
+Approach Fairness Gap Reduction Biased model 86.77% - Remove race only ~80%+ Minimal Remove race + proxy 15.69% 82%
Removing the protected attribute alone is not enough. You must audit every feature for correlation with protected attributes.
How to Detect Proxy Variables
@@ -264,7 +264,7 @@ The Bigger Picture
This is why algorithmic auditing is not optional. It is a prerequisite for deployment.
Related Projects in This Repo
-COMPAS/ - Full COMPAS analysis: biased model → fair model → 71% gap reductionAI Fair Recruitment/ - AI recruitment bias: 97.3% gap reduction after feature audit- Coming soon: Facial recognition bias, HMDA loan bias, healthcare AI
+COMPAS/ - Full COMPAS analysis: biased model → fair model → 82% gap reductionAI Fair Recruitment/ - AI recruitment bias: 97.3% gap reduction after feature audit- Coming soon: Facial recognition bias, HMDA loan bias, healthcare AI
Further Reading
- ProPublica: Machine Bias (2016)
- Barocas & Hardt: Fairness and Machine Learning
- Obermeyer et al.: Dissecting racial bias in an algorithm used to manage the health of populations (Science, 2019)
diff --git a/explainers/proxy-variables.md b/explainers/proxy-variables.md
index 3d4e935..e9582b8 100644
--- a/explainers/proxy-variables.md
+++ b/explainers/proxy-variables.md
@@ -89,7 +89,7 @@ X = pd.get_dummies(df[[
|---|---|---|
| Biased model | 86.77% | - |
| Remove race only | ~80%+ | Minimal |
-| Remove race + proxy | 15.69% | **71%** |
+| Remove race + proxy | 15.69% | **82%** |
**Removing the protected attribute alone is not enough. You must audit every feature for correlation with protected attributes.**
@@ -138,7 +138,7 @@ Data doesn't exist in a vacuum. It reflects the society that generated it. A mod
## Related Projects in This Repo
-- [`COMPAS/`](../COMPAS/) - Full COMPAS analysis: biased model → fair model → 71% gap reduction
+- [`COMPAS/`](../COMPAS/) - Full COMPAS analysis: biased model → fair model → 82% gap reduction
- [`AI Fair Recruitment/`](../AI%20Fair%20Recruitment/) - AI recruitment bias: 97.3% gap reduction after feature audit
- Coming soon: Facial recognition bias, HMDA loan bias, healthcare AI
diff --git a/explainers/reinforcement-learning.html b/explainers/reinforcement-learning.html
index 8e295df..f379eb9 100644
--- a/explainers/reinforcement-learning.html
+++ b/explainers/reinforcement-learning.html
@@ -411,7 +411,7 @@ Step 3 - Remove Proxies and Retrain
Results - mitigated policy (race + proxy removed):
Group High-Risk Flag Rate Black defendants 53.43% White defendants 37.74% New fairness gap 15.69%
-Approach Fairness Gap Reduction Biased policy 86.77% - Remove race only ~60% Partial Remove race + custody_status 15.69% 71%
+Approach Fairness Gap Reduction Biased policy 86.77% - Remove race only ~60% Partial Remove race + custody_status 15.69% 82%
The policy architecture did not change. The training procedure did not change. Only the state representation changed - and most of the discriminatory behaviour disappeared.
Second Case: Recommendation Systems and Feedback Loops
@@ -443,13 +443,13 @@ Second Case: Reco
Limitations
RL is rarely deployed explicitly in high-stakes settings. The COMPAS case is RL-adjacent, not pure RL. The patterns - reward misspecification, credit assignment failure, proxy exploitation - appear in supervised systems too. The explainer uses the RL frame because it makes these failure modes most legible, not because RL is uniquely responsible.
-Removing proxies reduces but does not eliminate the gap. The 71% reduction in the COMPAS case leaves a 15.69-point fairness gap. Remaining disparity reflects features that correlate with race for legitimate predictive reasons (prior arrests reflect real behaviour differences produced by structural conditions) or proxies not yet identified. Proxy removal is necessary but not sufficient.
+Removing proxies reduces but does not eliminate the gap. The 82% reduction in the COMPAS case leaves a 15.69-point fairness gap. Remaining disparity reflects features that correlate with race for legitimate predictive reasons (prior arrests reflect real behaviour differences produced by structural conditions) or proxies not yet identified. Proxy removal is necessary but not sufficient.
Defining the reward function is unavoidably political. There is no neutral reward signal for a parole decision. Choosing to penalise false releases more than wrongful holds is a value judgment about whose safety matters more. This explainer cannot resolve that question. It can only make it visible.
Credit assignment failure is not fully solvable. No discounting scheme correctly attributes a 6-month outcome to the exact decision that caused it when hundreds of intervening variables - housing, employment, family, neighbourhood - all contribute. This is a fundamental limit of sequential decision-making under delayed feedback, not an implementation flaw.
Watch time as a proxy for value is increasingly contested. Platforms have introduced secondary signals (surveys, explicit ratings) to supplement watch time. Whether these corrections are sufficient, or whether they introduce new biases, is an open empirical question.
Related Concepts
-feedback-loop-bias.md - How retraining on RL-generated decisions amplifies bias across cyclesproxy-variables.md - Why the state representation is where most RL bias enterslabel-bias.md - How the reward signal inherits bias from historical outcomesneural-networks.md - How the policy function learns from state-reward pairsCOMPAS/ - Full audit of the COMPAS dataset: 71% gap reduction after removing race + custody_status proxy
+feedback-loop-bias.md - How retraining on RL-generated decisions amplifies bias across cyclesproxy-variables.md - Why the state representation is where most RL bias enterslabel-bias.md - How the reward signal inherits bias from historical outcomesneural-networks.md - How the policy function learns from state-reward pairsCOMPAS/ - Full audit of the COMPAS dataset: 82% gap reduction after removing race + custody_status proxy
Further Reading
- Dressel & Farid: The Accuracy, Fairness, and Limits of Predicting Recidivism (Science Advances, 2018) - the peer-reviewed analysis showing COMPAS performs no better than untrained humans and exhibits racial disparities
- Sutton & Barto: Reinforcement Learning: An Introduction (MIT Press, 2nd ed., 2018) - the canonical RL textbook, free online; Chapter 1 covers the core loop; Chapter 17 covers applications
- Krakovna et al.: Specification Gaming: The Flip Side of AI Ingenuity (DeepMind, 2020) - the canonical catalogue of reward hacking cases, from simulated robots to production systems
diff --git a/explainers/reinforcement-learning.md b/explainers/reinforcement-learning.md
index 146f71c..37d5f2d 100644
--- a/explainers/reinforcement-learning.md
+++ b/explainers/reinforcement-learning.md
@@ -309,7 +309,7 @@ model_fair.fit(X_train_f, y_train_f)
|---|---|---|
| Biased policy | 86.77% | - |
| Remove race only | ~60% | Partial |
-| Remove race + custody_status | 15.69% | **71%** |
+| Remove race + custody_status | 15.69% | **82%** |
The policy architecture did not change. The training procedure did not change. Only the state representation changed - and most of the discriminatory behaviour disappeared.
@@ -353,7 +353,7 @@ The demographic consequences of this reward function are asymmetric: recommendat
**RL is rarely deployed explicitly in high-stakes settings.** The COMPAS case is RL-adjacent, not pure RL. The patterns - reward misspecification, credit assignment failure, proxy exploitation - appear in supervised systems too. The explainer uses the RL frame because it makes these failure modes most legible, not because RL is uniquely responsible.
-**Removing proxies reduces but does not eliminate the gap.** The 71% reduction in the COMPAS case leaves a 15.69-point fairness gap. Remaining disparity reflects features that correlate with race for legitimate predictive reasons (prior arrests reflect real behaviour differences produced by structural conditions) or proxies not yet identified. Proxy removal is necessary but not sufficient.
+**Removing proxies reduces but does not eliminate the gap.** The 82% reduction in the COMPAS case leaves a 15.69-point fairness gap. Remaining disparity reflects features that correlate with race for legitimate predictive reasons (prior arrests reflect real behaviour differences produced by structural conditions) or proxies not yet identified. Proxy removal is necessary but not sufficient.
**Defining the reward function is unavoidably political.** There is no neutral reward signal for a parole decision. Choosing to penalise false releases more than wrongful holds is a value judgment about whose safety matters more. This explainer cannot resolve that question. It can only make it visible.
@@ -369,7 +369,7 @@ The demographic consequences of this reward function are asymmetric: recommendat
- [`proxy-variables.md`](proxy-variables.md) - Why the state representation is where most RL bias enters
- [`label-bias.md`](label-bias.md) - How the reward signal inherits bias from historical outcomes
- [`neural-networks.md`](neural-networks.md) - How the policy function learns from state-reward pairs
-- [`COMPAS/`](../COMPAS/) - Full audit of the COMPAS dataset: 71% gap reduction after removing race + custody_status proxy
+- [`COMPAS/`](../COMPAS/) - Full audit of the COMPAS dataset: 82% gap reduction after removing race + custody_status proxy
---
diff --git a/faircode/_explainers/ai-objectivity-myth.md b/faircode/_explainers/ai-objectivity-myth.md
index e98cbbe..ee9c242 100644
--- a/faircode/_explainers/ai-objectivity-myth.md
+++ b/faircode/_explainers/ai-objectivity-myth.md
@@ -40,7 +40,7 @@ model.fit(X_train, y_train)
# Fairness Gap: 86.77%
```
-Removing `race` alone barely moves the number, because `CustodyStatus` - a record of prior system contact - carries the same signal: communities that were over-policed historically generate more "custody status" records today, independent of actual reoffense risk. Only after `race` *and* `CustodyStatus` are both dropped does the gap fall to 15.69%, a 71% reduction. The model was never neutral. It was a faithful record of an unequal criminal justice system, expressed as a probability.
+Removing `race` alone barely moves the number, because `CustodyStatus` - a record of prior system contact - carries the same signal: communities that were over-policed historically generate more "custody status" records today, independent of actual reoffense risk. Only after `race` *and* `CustodyStatus` are both dropped does the gap fall to 15.69%, an 82% reduction. The model was never neutral. It was a faithful record of an unequal criminal justice system, expressed as a probability.
## Detection Code
diff --git a/faircode/_explainers/confounding-variable.md b/faircode/_explainers/confounding-variable.md
index 46ad0b5..94586e7 100644
--- a/faircode/_explainers/confounding-variable.md
+++ b/faircode/_explainers/confounding-variable.md
@@ -69,7 +69,7 @@ Removing both `race` and `CustodyStatus` breaks this path:
```python
# fair.py: features include only priors_count + charge_degree + age
-# Black/White fairness gap: 15.69% - 71% reduction
+# Black/White fairness gap: 15.69% - 82% reduction
```
The residual 15.69% reflects other confounding paths (differential bail rates, charge severity distributions, surveillance-driven label noise) that require changes upstream of the model to eliminate entirely.
@@ -181,7 +181,7 @@ For continuous features, replace `chi2_contingency` with a Pearson correlation o
3. **Conditioning on a collider opens new bias.** A collider is a variable caused by both the feature and the outcome - the reverse of a confounder. Controlling for a collider introduces a spurious association rather than removing one. Correctly distinguishing confounders from colliders requires a causal graph (a DAG), not statistical testing alone. Chi-squared tests cannot tell you which direction the arrows point.
-4. **Confounder removal reduces but does not eliminate bias.** Removing `CustodyStatus` from COMPAS cuts the fairness gap from 86.77% to 15.69% - a 71% reduction. The remaining gap reflects additional confounding paths that cannot be closed by feature removal without changing the label generation process itself.
+4. **Confounder removal reduces but does not eliminate bias.** Removing `CustodyStatus` from COMPAS cuts the fairness gap from 86.77% to 15.69% - an 82% reduction. The remaining gap reflects additional confounding paths that cannot be closed by feature removal without changing the label generation process itself.
5. **Adjustment can introduce its own distortions.** Propensity score methods and inverse probability weighting reduce confounding but amplify variance, especially in small subgroups. In high-stakes settings, an overcorrected model may perform worse for the groups it was adjusted to protect.
@@ -205,7 +205,7 @@ When a confounded model is deployed and its outputs influence future labels - re
## Related Projects in This Repo
-- [`COMPAS/`](../COMPAS/) - the primary worked example. `CustodyStatus` confounds the race→recidivism path, driving 71% of the Black/White fairness gap. Removing it alongside `race` reduces the gap from 86.77% to 15.69%.
+- [`COMPAS/`](../COMPAS/) - the primary worked example. `CustodyStatus` confounds the race→recidivism path, driving 82% of the Black/White fairness gap. Removing it alongside `race` reduces the gap from 86.77% to 15.69%.
- [`Benefits Denial/`](../Benefits%20Denial/) - `relationship` and `marital-status` act as confounders for sex: historical gender norms independently elevated male-coded relationship statuses and income levels in the census data, creating a spurious association the model amplifies.
- [`Healthcare Readmission/`](../Healthcare%20Readmission/) - `payer_code` is confounded by race: differential insurance access is caused by structural factors that also independently predict readmission risk, not only by individual health status.
diff --git a/faircode/_explainers/data-leakage.md b/faircode/_explainers/data-leakage.md
index f11f86c..bb9a366 100644
--- a/faircode/_explainers/data-leakage.md
+++ b/faircode/_explainers/data-leakage.md
@@ -47,7 +47,7 @@ The COMPAS audit in this repository demonstrates a proxy form of target leakage.
This is the structural form of leakage: a feature that looks like a legitimate input but whose predictive power comes from downstream effects of the very outcome being predicted.
-Removing `CustodyStatus` alongside the protected race attribute reduces the gap to 15.69% - a 71% reduction.
+Removing `CustodyStatus` alongside the protected race attribute reduces the gap to 15.69% - an 82% reduction.
```python
import os
diff --git a/faircode/_explainers/equalized-odds.md b/faircode/_explainers/equalized-odds.md
index 50cbeb0..706d756 100644
--- a/faircode/_explainers/equalized-odds.md
+++ b/faircode/_explainers/equalized-odds.md
@@ -184,7 +184,7 @@ Even after correcting for Equalized Odds, models can still encode protected attr
## Related Projects in This Repo
-- [`COMPAS/`](../COMPAS/) - Full COMPAS analysis: biased model → fair model → 71% gap reduction. Equalized Odds violations visible in the raw model outputs.
+- [`COMPAS/`](../COMPAS/) - Full COMPAS analysis: biased model → fair model → 82% gap reduction. Equalized Odds violations visible in the raw model outputs.
- [`explainers/proxy-variables.md`](proxy-variables.md) - Why AI stays biased even after you remove protected attributes
- [`explainers/shap-values.md`](shap-values.md) - How to see exactly what drove an AI decision - and use that to catch bias
diff --git a/faircode/_explainers/how-ai-detects-patterns.md b/faircode/_explainers/how-ai-detects-patterns.md
index 229b1bc..30b73da 100644
--- a/faircode/_explainers/how-ai-detects-patterns.md
+++ b/faircode/_explainers/how-ai-detects-patterns.md
@@ -52,7 +52,7 @@ model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)
```
-When `race` is dropped, the gap barely moves - `CustodyStatus` alone reconstructs most of the pattern, because it correlates with `race` at 0.31 importance even on its own. Only when both `race` and `CustodyStatus` are removed does the gap fall to 15.69%, a 71% reduction. The pattern the model was detecting was never really about behavior - it was about which neighborhoods got patrolled.
+When `race` is dropped, the gap barely moves - `CustodyStatus` alone reconstructs most of the pattern, because it correlates with `race` at 0.31 importance even on its own. Only when both `race` and `CustodyStatus` are removed does the gap fall to 15.69%, an 82% reduction. The pattern the model was detecting was never really about behavior - it was about which neighborhoods got patrolled.
## Detection Code
diff --git a/faircode/_explainers/ml-bias.md b/faircode/_explainers/ml-bias.md
index b3ac0f2..5ae0b0b 100644
--- a/faircode/_explainers/ml-bias.md
+++ b/faircode/_explainers/ml-bias.md
@@ -76,7 +76,7 @@ The COMPAS audit is the sharpest illustration of how all four entry points conve
**Feedback loop:** COMPAS scores influence bail and sentencing decisions. A defendant flagged as high-risk is more likely to be detained pretrial. Pretrial detention increases the probability of conviction and reoffending (due to job loss, housing instability, and network effects). The model produces the outcome it predicted, and the outcome enters the next training set as evidence the prediction was correct.
-The biased model flags Black defendants as high-risk at 86.77%. The fair model - with race and custody status removed - reduces that gap to 15.69%. A 71% reduction from one targeted intervention on one proxy variable.
+The biased model flags Black defendants as high-risk at 86.77%. The fair model - with race and custody status removed - reduces that gap to 15.69%. An 82% reduction from one targeted intervention on one proxy variable.
```python
import pandas as pd
diff --git a/faircode/_explainers/proxy-variables.md b/faircode/_explainers/proxy-variables.md
index 3d4e935..e9582b8 100644
--- a/faircode/_explainers/proxy-variables.md
+++ b/faircode/_explainers/proxy-variables.md
@@ -89,7 +89,7 @@ X = pd.get_dummies(df[[
|---|---|---|
| Biased model | 86.77% | - |
| Remove race only | ~80%+ | Minimal |
-| Remove race + proxy | 15.69% | **71%** |
+| Remove race + proxy | 15.69% | **82%** |
**Removing the protected attribute alone is not enough. You must audit every feature for correlation with protected attributes.**
@@ -138,7 +138,7 @@ Data doesn't exist in a vacuum. It reflects the society that generated it. A mod
## Related Projects in This Repo
-- [`COMPAS/`](../COMPAS/) - Full COMPAS analysis: biased model → fair model → 71% gap reduction
+- [`COMPAS/`](../COMPAS/) - Full COMPAS analysis: biased model → fair model → 82% gap reduction
- [`AI Fair Recruitment/`](../AI%20Fair%20Recruitment/) - AI recruitment bias: 97.3% gap reduction after feature audit
- Coming soon: Facial recognition bias, HMDA loan bias, healthcare AI
diff --git a/faircode/_explainers/reinforcement-learning.md b/faircode/_explainers/reinforcement-learning.md
index 146f71c..37d5f2d 100644
--- a/faircode/_explainers/reinforcement-learning.md
+++ b/faircode/_explainers/reinforcement-learning.md
@@ -309,7 +309,7 @@ model_fair.fit(X_train_f, y_train_f)
|---|---|---|
| Biased policy | 86.77% | - |
| Remove race only | ~60% | Partial |
-| Remove race + custody_status | 15.69% | **71%** |
+| Remove race + custody_status | 15.69% | **82%** |
The policy architecture did not change. The training procedure did not change. Only the state representation changed - and most of the discriminatory behaviour disappeared.
@@ -353,7 +353,7 @@ The demographic consequences of this reward function are asymmetric: recommendat
**RL is rarely deployed explicitly in high-stakes settings.** The COMPAS case is RL-adjacent, not pure RL. The patterns - reward misspecification, credit assignment failure, proxy exploitation - appear in supervised systems too. The explainer uses the RL frame because it makes these failure modes most legible, not because RL is uniquely responsible.
-**Removing proxies reduces but does not eliminate the gap.** The 71% reduction in the COMPAS case leaves a 15.69-point fairness gap. Remaining disparity reflects features that correlate with race for legitimate predictive reasons (prior arrests reflect real behaviour differences produced by structural conditions) or proxies not yet identified. Proxy removal is necessary but not sufficient.
+**Removing proxies reduces but does not eliminate the gap.** The 82% reduction in the COMPAS case leaves a 15.69-point fairness gap. Remaining disparity reflects features that correlate with race for legitimate predictive reasons (prior arrests reflect real behaviour differences produced by structural conditions) or proxies not yet identified. Proxy removal is necessary but not sufficient.
**Defining the reward function is unavoidably political.** There is no neutral reward signal for a parole decision. Choosing to penalise false releases more than wrongful holds is a value judgment about whose safety matters more. This explainer cannot resolve that question. It can only make it visible.
@@ -369,7 +369,7 @@ The demographic consequences of this reward function are asymmetric: recommendat
- [`proxy-variables.md`](proxy-variables.md) - Why the state representation is where most RL bias enters
- [`label-bias.md`](label-bias.md) - How the reward signal inherits bias from historical outcomes
- [`neural-networks.md`](neural-networks.md) - How the policy function learns from state-reward pairs
-- [`COMPAS/`](../COMPAS/) - Full audit of the COMPAS dataset: 71% gap reduction after removing race + custody_status proxy
+- [`COMPAS/`](../COMPAS/) - Full audit of the COMPAS dataset: 82% gap reduction after removing race + custody_status proxy
---
diff --git a/llms-full.txt b/llms-full.txt
index a99002b..e11ad4a 100644
--- a/llms-full.txt
+++ b/llms-full.txt
@@ -103,7 +103,7 @@ X = pd.get_dummies(df[[
|---|---|---|
| Biased model | 86.77% | - |
| Remove race only | ~80%+ | Minimal |
-| Remove race + proxy | 15.69% | **71%** |
+| Remove race + proxy | 15.69% | **82%** |
**Removing the protected attribute alone is not enough. You must audit every feature for correlation with protected attributes.**
@@ -152,7 +152,7 @@ Data doesn't exist in a vacuum. It reflects the society that generated it. A mod
## Related Projects in This Repo
-- [`COMPAS/`](../COMPAS/) - Full COMPAS analysis: biased model → fair model → 71% gap reduction
+- [`COMPAS/`](../COMPAS/) - Full COMPAS analysis: biased model → fair model → 82% gap reduction
- [`AI Fair Recruitment/`](../AI%20Fair%20Recruitment/) - AI recruitment bias: 97.3% gap reduction after feature audit
- Coming soon: Facial recognition bias, HMDA loan bias, healthcare AI
@@ -360,7 +360,7 @@ Even after correcting for Equalized Odds, models can still encode protected attr
## Related Projects in This Repo
-- [`COMPAS/`](../COMPAS/) - Full COMPAS analysis: biased model → fair model → 71% gap reduction. Equalized Odds violations visible in the raw model outputs.
+- [`COMPAS/`](../COMPAS/) - Full COMPAS analysis: biased model → fair model → 82% gap reduction. Equalized Odds violations visible in the raw model outputs.
- [`explainers/proxy-variables.md`](proxy-variables.md) - Why AI stays biased even after you remove protected attributes
- [`explainers/shap-values.md`](shap-values.md) - How to see exactly what drove an AI decision - and use that to catch bias
@@ -4413,7 +4413,7 @@ model_fair.fit(X_train_f, y_train_f)
|---|---|---|
| Biased policy | 86.77% | - |
| Remove race only | ~60% | Partial |
-| Remove race + custody_status | 15.69% | **71%** |
+| Remove race + custody_status | 15.69% | **82%** |
The policy architecture did not change. The training procedure did not change. Only the state representation changed - and most of the discriminatory behaviour disappeared.
@@ -4457,7 +4457,7 @@ The demographic consequences of this reward function are asymmetric: recommendat
**RL is rarely deployed explicitly in high-stakes settings.** The COMPAS case is RL-adjacent, not pure RL. The patterns - reward misspecification, credit assignment failure, proxy exploitation - appear in supervised systems too. The explainer uses the RL frame because it makes these failure modes most legible, not because RL is uniquely responsible.
-**Removing proxies reduces but does not eliminate the gap.** The 71% reduction in the COMPAS case leaves a 15.69-point fairness gap. Remaining disparity reflects features that correlate with race for legitimate predictive reasons (prior arrests reflect real behaviour differences produced by structural conditions) or proxies not yet identified. Proxy removal is necessary but not sufficient.
+**Removing proxies reduces but does not eliminate the gap.** The 82% reduction in the COMPAS case leaves a 15.69-point fairness gap. Remaining disparity reflects features that correlate with race for legitimate predictive reasons (prior arrests reflect real behaviour differences produced by structural conditions) or proxies not yet identified. Proxy removal is necessary but not sufficient.
**Defining the reward function is unavoidably political.** There is no neutral reward signal for a parole decision. Choosing to penalise false releases more than wrongful holds is a value judgment about whose safety matters more. This explainer cannot resolve that question. It can only make it visible.
@@ -4473,7 +4473,7 @@ The demographic consequences of this reward function are asymmetric: recommendat
- [`proxy-variables.md`](proxy-variables.md) - Why the state representation is where most RL bias enters
- [`label-bias.md`](label-bias.md) - How the reward signal inherits bias from historical outcomes
- [`neural-networks.md`](neural-networks.md) - How the policy function learns from state-reward pairs
-- [`COMPAS/`](../COMPAS/) - Full audit of the COMPAS dataset: 71% gap reduction after removing race + custody_status proxy
+- [`COMPAS/`](../COMPAS/) - Full audit of the COMPAS dataset: 82% gap reduction after removing race + custody_status proxy
---
@@ -4825,7 +4825,7 @@ The COMPAS audit is the sharpest illustration of how all four entry points conve
**Feedback loop:** COMPAS scores influence bail and sentencing decisions. A defendant flagged as high-risk is more likely to be detained pretrial. Pretrial detention increases the probability of conviction and reoffending (due to job loss, housing instability, and network effects). The model produces the outcome it predicted, and the outcome enters the next training set as evidence the prediction was correct.
-The biased model flags Black defendants as high-risk at 86.77%. The fair model - with race and custody status removed - reduces that gap to 15.69%. A 71% reduction from one targeted intervention on one proxy variable.
+The biased model flags Black defendants as high-risk at 86.77%. The fair model - with race and custody status removed - reduces that gap to 15.69%. An 82% reduction from one targeted intervention on one proxy variable.
```python
import pandas as pd
@@ -5049,7 +5049,7 @@ The COMPAS audit in this repository demonstrates a proxy form of target leakage.
This is the structural form of leakage: a feature that looks like a legitimate input but whose predictive power comes from downstream effects of the very outcome being predicted.
-Removing `CustodyStatus` alongside the protected race attribute reduces the gap to 15.69% - a 71% reduction.
+Removing `CustodyStatus` alongside the protected race attribute reduces the gap to 15.69% - an 82% reduction.
```python
import os
@@ -5289,7 +5289,7 @@ model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)
```
-When `race` is dropped, the gap barely moves - `CustodyStatus` alone reconstructs most of the pattern, because it correlates with `race` at 0.31 importance even on its own. Only when both `race` and `CustodyStatus` are removed does the gap fall to 15.69%, a 71% reduction. The pattern the model was detecting was never really about behavior - it was about which neighborhoods got patrolled.
+When `race` is dropped, the gap barely moves - `CustodyStatus` alone reconstructs most of the pattern, because it correlates with `race` at 0.31 importance even on its own. Only when both `race` and `CustodyStatus` are removed does the gap fall to 15.69%, an 82% reduction. The pattern the model was detecting was never really about behavior - it was about which neighborhoods got patrolled.
## Detection Code
@@ -5629,7 +5629,7 @@ model.fit(X_train, y_train)
# Fairness Gap: 86.77%
```
-Removing `race` alone barely moves the number, because `CustodyStatus` - a record of prior system contact - carries the same signal: communities that were over-policed historically generate more "custody status" records today, independent of actual reoffense risk. Only after `race` *and* `CustodyStatus` are both dropped does the gap fall to 15.69%, a 71% reduction. The model was never neutral. It was a faithful record of an unequal criminal justice system, expressed as a probability.
+Removing `race` alone barely moves the number, because `CustodyStatus` - a record of prior system contact - carries the same signal: communities that were over-policed historically generate more "custody status" records today, independent of actual reoffense risk. Only after `race` *and* `CustodyStatus` are both dropped does the gap fall to 15.69%, an 82% reduction. The model was never neutral. It was a faithful record of an unequal criminal justice system, expressed as a probability.
## Detection Code
@@ -5813,7 +5813,7 @@ Removing both `race` and `CustodyStatus` breaks this path:
```python
# fair.py: features include only priors_count + charge_degree + age
-# Black/White fairness gap: 15.69% - 71% reduction
+# Black/White fairness gap: 15.69% - 82% reduction
```
The residual 15.69% reflects other confounding paths (differential bail rates, charge severity distributions, surveillance-driven label noise) that require changes upstream of the model to eliminate entirely.
@@ -5925,7 +5925,7 @@ For continuous features, replace `chi2_contingency` with a Pearson correlation o
3. **Conditioning on a collider opens new bias.** A collider is a variable caused by both the feature and the outcome - the reverse of a confounder. Controlling for a collider introduces a spurious association rather than removing one. Correctly distinguishing confounders from colliders requires a causal graph (a DAG), not statistical testing alone. Chi-squared tests cannot tell you which direction the arrows point.
-4. **Confounder removal reduces but does not eliminate bias.** Removing `CustodyStatus` from COMPAS cuts the fairness gap from 86.77% to 15.69% - a 71% reduction. The remaining gap reflects additional confounding paths that cannot be closed by feature removal without changing the label generation process itself.
+4. **Confounder removal reduces but does not eliminate bias.** Removing `CustodyStatus` from COMPAS cuts the fairness gap from 86.77% to 15.69% - an 82% reduction. The remaining gap reflects additional confounding paths that cannot be closed by feature removal without changing the label generation process itself.
5. **Adjustment can introduce its own distortions.** Propensity score methods and inverse probability weighting reduce confounding but amplify variance, especially in small subgroups. In high-stakes settings, an overcorrected model may perform worse for the groups it was adjusted to protect.
@@ -5949,7 +5949,7 @@ When a confounded model is deployed and its outputs influence future labels - re
## Related Projects in This Repo
-- [`COMPAS/`](../COMPAS/) - the primary worked example. `CustodyStatus` confounds the race→recidivism path, driving 71% of the Black/White fairness gap. Removing it alongside `race` reduces the gap from 86.77% to 15.69%.
+- [`COMPAS/`](../COMPAS/) - the primary worked example. `CustodyStatus` confounds the race→recidivism path, driving 82% of the Black/White fairness gap. Removing it alongside `race` reduces the gap from 86.77% to 15.69%.
- [`Benefits Denial/`](../Benefits%20Denial/) - `relationship` and `marital-status` act as confounders for sex: historical gender norms independently elevated male-coded relationship statuses and income levels in the census data, creating a spurious association the model amplifies.
- [`Healthcare Readmission/`](../Healthcare%20Readmission/) - `payer_code` is confounded by race: differential insurance access is caused by structural factors that also independently predict readmission risk, not only by individual health status.