Merge pull request #558 from propcgamer20-png/docs/results-glance-corrections

docs: correct COMPAS reduction figure repo-wide (71% -> 82%) + Benefits Denial proxy count

Author: yakew7

.github/PULL_REQUEST_TEMPLATE.md

@@ -31,7 +31,7 @@
 
 **After fairness gap:** <!-- e.g. 15.69% -->
 
-**Reduction:** <!-- e.g. 71% -->
+**Reduction:** <!-- e.g. 82% -->
 
 **Protected attribute(s):** <!-- e.g. Race -->
 

Benefits Denial/README.md

@@ -19,7 +19,7 @@ python3 "Benefits Denial/fair.py"     # mitigated (protected attribute + proxies
 ## What the audit controls
 
 - Protected attribute(s): Sex, Race, Origin, Age
-- Proxy feature(s) removed in `fair.py`: Relationship, Marital Status, Hours, Occupation, fnlwgt
+- Proxy feature(s) removed in `fair.py`: Relationship, Marital Status, Hours, Occupation, fnlwgt (census sampling weight)
 - Fairness metric: Demographic Parity (difference in positive-prediction rate between groups)
 
 ## Expected result (published, paper-aligned)

CHANGELOG.md

@@ -717,7 +717,7 @@ First release since **v1.1.0** (9 Jun 2026). The headline is the **Open Dataset
 ### Added
 - Explainer: Reinforcement Learning - `reinforcement-learning.md` created by evanjain-dot (PR #48, commit a785ea95), added to `index.html`, `README.md`, and `CONTRIBUTING.md` (commit e3928af7)
   - Full explainer covering the three-part RL loop (state → action → reward → policy), reward function design as a political act, reward hacking, and the credit assignment problem
-  - Real-world proof using COMPAS as an RL-adjacent system: biased policy produces 86.77% Black/White fairness gap; removing race + `CustodyStatus` proxy reduces gap to 15.69% (71% reduction)
+  - Real-world proof using COMPAS as an RL-adjacent system: biased policy produces 86.77% Black/White fairness gap; removing race + `CustodyStatus` proxy reduces gap to 15.69% (82% reduction)
   - Results table: biased policy vs. race-only removal vs. race + proxy removal
   - Second case: YouTube recommendation engine using watch time as reward signal - documents asymmetric demographic consequences and outrage optimisation
   - `fairness_gap()` detection code with chi-squared proxy check for state representation audit

README.md

@@ -546,7 +546,7 @@ Automated welfare and benefits systems use income-prediction models to screen ap
 
 #### The Problem - `unfair.py`
 
-Trained with sex, race, age, and national origin directly, plus four proxy variables that reconstruct those attributes even after the protected columns are removed.
+Trained with sex, race, age, and national origin directly, plus five proxy variables that reconstruct those attributes even after the protected columns are removed.
 
 | Group | Ineligibility Flag Rate |
 |-------|:-----------------------:|
@@ -568,7 +568,7 @@ Trained with sex, race, age, and national origin directly, plus four proxy varia
 
 #### The Fix - `fair.py`
 
-Dropped all four protected attributes and all four proxy variables. Retained only the features a means-tested programme can legitimately consult under equality law.
+Dropped all four protected attributes and all five proxy variables. Retained only the features a means-tested programme can legitimately consult under equality law.
 
 ```python
 # THE FIX: Policy-defined economic signals only
@@ -586,6 +586,7 @@ features = [
     # marital.status removed ✓  (proxy: encodes sex via spousal status)
     # hours.per.week removed ✓  (proxy: encodes sex via caregiving gap)
     # occupation     removed ✓  (proxy: encodes race via occupational segregation)
+    # fnlwgt         removed ✓  (proxy: census sampling weight, no causal link)
 ]
 ```
 

explainers/ai-objectivity-myth.html

@@ -211,7 +211,7 @@ <h3 id="concrete-example-compas-audit-01">Concrete Example: COMPAS - Audit 01</h
 model = RandomForestClassifier(n_estimators=100, random_state=42)
 model.fit(X_train, y_train)
 # Fairness Gap: 86.77%</code></pre>
-<p>Removing <code>race</code> alone barely moves the number, because <code>CustodyStatus</code> - a record of prior system contact - carries the same signal: communities that were over-policed historically generate more &quot;custody status&quot; records today, independent of actual reoffense risk. Only after <code>race</code> <em>and</em> <code>CustodyStatus</code> are both dropped does the gap fall to 15.69%, a 71% reduction. The model was never neutral. It was a faithful record of an unequal criminal justice system, expressed as a probability.</p>
+<p>Removing <code>race</code> alone barely moves the number, because <code>CustodyStatus</code> - a record of prior system contact - carries the same signal: communities that were over-policed historically generate more &quot;custody status&quot; records today, independent of actual reoffense risk. Only after <code>race</code> <em>and</em> <code>CustodyStatus</code> are both dropped does the gap fall to 15.69%, an 82% reduction. The model was never neutral. It was a faithful record of an unequal criminal justice system, expressed as a probability.</p>
 <h3 id="detection-code">Detection Code</h3>
 <p>This function checks whether a model&#x27;s outcome rates differ across protected groups by more than a configurable threshold, and flags features that correlate with the protected attribute strongly enough to explain the gap on their own.</p>
 <pre><code class="language-python">import pandas as pd

explainers/ai-objectivity-myth.md

@@ -40,7 +40,7 @@ model.fit(X_train, y_train)
 # Fairness Gap: 86.77%
 ```
 
-Removing `race` alone barely moves the number, because `CustodyStatus` - a record of prior system contact - carries the same signal: communities that were over-policed historically generate more "custody status" records today, independent of actual reoffense risk. Only after `race` *and* `CustodyStatus` are both dropped does the gap fall to 15.69%, a 71% reduction. The model was never neutral. It was a faithful record of an unequal criminal justice system, expressed as a probability.
+Removing `race` alone barely moves the number, because `CustodyStatus` - a record of prior system contact - carries the same signal: communities that were over-policed historically generate more "custody status" records today, independent of actual reoffense risk. Only after `race` *and* `CustodyStatus` are both dropped does the gap fall to 15.69%, an 82% reduction. The model was never neutral. It was a faithful record of an unequal criminal justice system, expressed as a probability.
 
 ## Detection Code
 

explainers/confounding-variable.html

@@ -222,7 +222,7 @@ <h3 id="concrete-example-compas-and-custodystatus">Concrete Example: COMPAS and
 <p>Both arrows are caused by the same systemic factor. <code>CustodyStatus</code> is associated with the recidivism label not because custody history is a reliable individual risk signal, but because the same structural forces that produce elevated custody records also produce more recidivism label events - through monitoring, not through behavior.</p>
 <p>Removing both <code>race</code> and <code>CustodyStatus</code> breaks this path:</p>
 <pre><code class="language-python"># fair.py: features include only priors_count + charge_degree + age
-# Black/White fairness gap: 15.69% - 71% reduction</code></pre>
+# Black/White fairness gap: 15.69% - 82% reduction</code></pre>
 <p>The residual 15.69% reflects other confounding paths (differential bail rates, charge severity distributions, surveillance-driven label noise) that require changes upstream of the model to eliminate entirely.</p>
 <hr>
 <h3 id="detection-code">Detection Code</h3>
@@ -315,7 +315,7 @@ <h3 id="limitations">Limitations</h3>
 <p>1. <strong>Stratified analysis cannot distinguish confounding from effect modification.</strong> If a feature has a genuinely different causal effect on the outcome across strata - not just a different baseline - that is effect modification, not confounding. The two require different handling. Conflating them produces wrong adjustments.</p>
 <p>2. <strong>You can only condition on observed confounders.</strong> If the confounder is unmeasured - historical policing intensity, neighbourhood-level surveillance, differential healthcare access - no statistical adjustment removes its effect. Causal inference methods (instrumental variables, propensity score matching, difference-in-differences) can partially address unmeasured confounding but require strong, often untestable assumptions about the causal structure.</p>
 <p>3. <strong>Conditioning on a collider opens new bias.</strong> A collider is a variable caused by both the feature and the outcome - the reverse of a confounder. Controlling for a collider introduces a spurious association rather than removing one. Correctly distinguishing confounders from colliders requires a causal graph (a DAG), not statistical testing alone. Chi-squared tests cannot tell you which direction the arrows point.</p>
-<p>4. <strong>Confounder removal reduces but does not eliminate bias.</strong> Removing <code>CustodyStatus</code> from COMPAS cuts the fairness gap from 86.77% to 15.69% - a 71% reduction. The remaining gap reflects additional confounding paths that cannot be closed by feature removal without changing the label generation process itself.</p>
+<p>4. <strong>Confounder removal reduces but does not eliminate bias.</strong> Removing <code>CustodyStatus</code> from COMPAS cuts the fairness gap from 86.77% to 15.69% - an 82% reduction. The remaining gap reflects additional confounding paths that cannot be closed by feature removal without changing the label generation process itself.</p>
 <p>5. <strong>Adjustment can introduce its own distortions.</strong> Propensity score methods and inverse probability weighting reduce confounding but amplify variance, especially in small subgroups. In high-stakes settings, an overcorrected model may perform worse for the groups it was adjusted to protect.</p>
 <hr>
 <h3 id="related-concepts">Related Concepts</h3>
@@ -329,7 +329,7 @@ <h4 id="feedback-loop-bias">Feedback Loop Bias</h4>
 <p>When a confounded model is deployed and its outputs influence future labels - recidivism surveillance, credit monitoring, healthcare resource allocation - the confounding strengthens across retraining cycles. The model&#x27;s outputs become part of the data-generating process, reinforcing the spurious association with each iteration. See <a href="feedback-loop-bias.html">feedback-loop-bias.md</a>.</p>
 <hr>
 <h3 id="related-projects-in-this-repo">Related Projects in This Repo</h3>
-<ul><li><a href="../index.html#project-compas"><code>COMPAS/</code></a> - the primary worked example. <code>CustodyStatus</code> confounds the race→recidivism path, driving 71% of the Black/White fairness gap. Removing it alongside <code>race</code> reduces the gap from 86.77% to 15.69%.</li><li><a href="../index.html#project-benefits"><code>Benefits Denial/</code></a> - <code>relationship</code> and <code>marital-status</code> act as confounders for sex: historical gender norms independently elevated male-coded relationship statuses and income levels in the census data, creating a spurious association the model amplifies.</li><li><a href="../index.html#project-readmission"><code>Healthcare Readmission/</code></a> - <code>payer_code</code> is confounded by race: differential insurance access is caused by structural factors that also independently predict readmission risk, not only by individual health status.</li></ul>
+<ul><li><a href="../index.html#project-compas"><code>COMPAS/</code></a> - the primary worked example. <code>CustodyStatus</code> confounds the race→recidivism path, driving 82% of the Black/White fairness gap. Removing it alongside <code>race</code> reduces the gap from 86.77% to 15.69%.</li><li><a href="../index.html#project-benefits"><code>Benefits Denial/</code></a> - <code>relationship</code> and <code>marital-status</code> act as confounders for sex: historical gender norms independently elevated male-coded relationship statuses and income levels in the census data, creating a spurious association the model amplifies.</li><li><a href="../index.html#project-readmission"><code>Healthcare Readmission/</code></a> - <code>payer_code</code> is confounded by race: differential insurance access is caused by structural factors that also independently predict readmission risk, not only by individual health status.</li></ul>
 <hr>
 <h3 id="further-reading">Further Reading</h3>
 <ul><li><a href="https://doi.org/10.1017/CBO9780511803161" target="_blank" rel="noreferrer noopener">Pearl, J. (2009). <em>Causality: Models, Reasoning and Inference</em> (2nd ed.). Cambridge University Press.</a> - the foundational text on causal graphs, the do-calculus, and the formal definitions of confounders, mediators, and colliders that underpin modern causal fairness work.</li><li><a href="https://doi.org/10.1126/science.aax2342" target="_blank" rel="noreferrer noopener">Obermeyer, Z., Powers, B., Vogeli, C., &amp; Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. <em>Science</em>, 366(6464), 447–453.</a> - a documented case of confounding-driven racial bias: healthcare cost (the proxy label) was confounded by differential access, making Black patients appear healthier than white patients with the same conditions, and the algorithm allocated less care as a result.</li><li><a href="https://doi.org/10.1214/12-AOS1058" target="_blank" rel="noreferrer noopener">VanderWeele, T. J., &amp; Shpitser, I. (2013). On the definition of a confounder. <em>Annals of Statistics</em>, 41(1), 196–220.</a> - a rigorous definition of confounding that distinguishes it from colliders and mediators, resolving long-standing disagreements in the epidemiological and statistical literature that carry directly into algorithmic fairness auditing.</li></ul>

explainers/confounding-variable.md

@@ -69,7 +69,7 @@ Removing both `race` and `CustodyStatus` breaks this path:
 
 ```python
 # fair.py: features include only priors_count + charge_degree + age
-# Black/White fairness gap: 15.69% - 71% reduction
+# Black/White fairness gap: 15.69% - 82% reduction
 ```
 
 The residual 15.69% reflects other confounding paths (differential bail rates, charge severity distributions, surveillance-driven label noise) that require changes upstream of the model to eliminate entirely.
@@ -181,7 +181,7 @@ For continuous features, replace `chi2_contingency` with a Pearson correlation o
 
 3. **Conditioning on a collider opens new bias.** A collider is a variable caused by both the feature and the outcome - the reverse of a confounder. Controlling for a collider introduces a spurious association rather than removing one. Correctly distinguishing confounders from colliders requires a causal graph (a DAG), not statistical testing alone. Chi-squared tests cannot tell you which direction the arrows point.
 
-4. **Confounder removal reduces but does not eliminate bias.** Removing `CustodyStatus` from COMPAS cuts the fairness gap from 86.77% to 15.69% - a 71% reduction. The remaining gap reflects additional confounding paths that cannot be closed by feature removal without changing the label generation process itself.
+4. **Confounder removal reduces but does not eliminate bias.** Removing `CustodyStatus` from COMPAS cuts the fairness gap from 86.77% to 15.69% - an 82% reduction. The remaining gap reflects additional confounding paths that cannot be closed by feature removal without changing the label generation process itself.
 
 5. **Adjustment can introduce its own distortions.** Propensity score methods and inverse probability weighting reduce confounding but amplify variance, especially in small subgroups. In high-stakes settings, an overcorrected model may perform worse for the groups it was adjusted to protect.
 
@@ -205,7 +205,7 @@ When a confounded model is deployed and its outputs influence future labels - re
 
 ## Related Projects in This Repo
 
-- [`COMPAS/`](../COMPAS/) - the primary worked example. `CustodyStatus` confounds the race→recidivism path, driving 71% of the Black/White fairness gap. Removing it alongside `race` reduces the gap from 86.77% to 15.69%.
+- [`COMPAS/`](../COMPAS/) - the primary worked example. `CustodyStatus` confounds the race→recidivism path, driving 82% of the Black/White fairness gap. Removing it alongside `race` reduces the gap from 86.77% to 15.69%.
 - [`Benefits Denial/`](../Benefits%20Denial/) - `relationship` and `marital-status` act as confounders for sex: historical gender norms independently elevated male-coded relationship statuses and income levels in the census data, creating a spurious association the model amplifies.
 - [`Healthcare Readmission/`](../Healthcare%20Readmission/) - `payer_code` is confounded by race: differential insurance access is caused by structural factors that also independently predict readmission risk, not only by individual health status.
 

explainers/data-leakage.html

@@ -210,7 +210,7 @@ <h4 id="train-test-contamination">Train-Test Contamination</h4>
 <h3 id="concrete-example">Concrete Example</h3>
 <p>The COMPAS audit in this repository demonstrates a proxy form of target leakage. <code>CustodyStatus</code> - a feature describing whether a defendant is currently in custody - correlates with race at p &lt; 0.001 and with the recidivism label at p &lt; 0.001. It is not a neutral operational variable. It encodes the outcome of a prior criminal justice interaction, which itself encodes historical over-policing of Black communities. Including it in the model achieves an 86.77% Black/White fairness gap in positive prediction rates.</p>
 <p>This is the structural form of leakage: a feature that looks like a legitimate input but whose predictive power comes from downstream effects of the very outcome being predicted.</p>
-<p>Removing <code>CustodyStatus</code> alongside the protected race attribute reduces the gap to 15.69% - a 71% reduction.</p>
+<p>Removing <code>CustodyStatus</code> alongside the protected race attribute reduces the gap to 15.69% - an 82% reduction.</p>
 <pre><code class="language-python">import os
 import pandas as pd
 from sklearn.ensemble import RandomForestClassifier

explainers/data-leakage.md

@@ -47,7 +47,7 @@ The COMPAS audit in this repository demonstrates a proxy form of target leakage.
 
 This is the structural form of leakage: a feature that looks like a legitimate input but whose predictive power comes from downstream effects of the very outcome being predicted.
 
-Removing `CustodyStatus` alongside the protected race attribute reduces the gap to 15.69% - a 71% reduction.
+Removing `CustodyStatus` alongside the protected race attribute reduces the gap to 15.69% - an 82% reduction.
 
 ```python
 import os