Skip to content

Commit 3a3e29b

Browse files
committed
Portfolio upgrade: Toulmin benchmark, academic frame, worked examples
- Add 10-case claim-lattice vs baseline Q&A benchmark with measurable lifts - Document Toulmin argument model mapping (academic citation) - Commit full philosophy / kaggle / freelance worked outputs - Rewrite README: problem-first, before/after, benchmark table
1 parent 77d6e50 commit 3a3e29b

36 files changed

Lines changed: 3217 additions & 42 deletions

README.md

Lines changed: 128 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,108 @@
11
# Epistemic Forge
22

3-
**An ARSENAL-powered kit for turning messy questions into claim lattices, dialectic briefs, freelance scopes, and Kaggle notebook spines.**
3+
**Stop shipping confident mush.**
4+
Turn messy research, philosophy, freelance, and Kaggle questions into **claim lattices**—structured arguments with evidence slots, objections, confidence, and next actions.
45

56
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
67
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
8+
[![Tests](https://img.shields.io/badge/tests-6%20passed-brightgreen.svg)](tests/)
79
[![Built with ARSENAL](https://img.shields.io/badge/built%20with-ARSENAL%20L0--L6-purple.svg)](https://github.com/faresrafat3/arsenal-unified-master-pipeline)
10+
[![Benchmark](https://img.shields.io/badge/Toulmin%20coverage-+231%25%20vs%20baseline-orange.svg)](docs/benchmark/BENCHMARK.md)
811

9-
Epistemic Forge is a **local-first Python package** (no paid API required) that implements a faithful, practical slice of the [ARSENAL](https://github.com/faresrafat3/arsenal-unified-master-pipeline) unified agent pipeline:
12+
Local-first Python package (**no paid API required**). Implements a practical slice of the [ARSENAL](https://github.com/faresrafat3/arsenal-unified-master-pipeline) L0–L6 pipeline.
1013

11-
| Layer | ARSENAL idea | In this repo |
14+
---
15+
16+
## The problem
17+
18+
Experts and freelancers constantly answer hard questions under uncertainty:
19+
20+
- a founder wants a **2-week research sprint** scoped *now*
21+
- a Kaggle teammate wants a **leakage-safe baseline**, not vibes
22+
- a seminar needs a **dialectic**, not a blog take
23+
24+
Default LLM / one-shot Q&A style fails the same way every time:
25+
26+
| Failure mode | What you get |
27+
|---|---|
28+
| Hidden assumptions | Smooth prose, no load-bearing claims listed |
29+
| No objections | One-sided answers that collapse in review |
30+
| False precision | Confident tone without confidence labels |
31+
| Weak packaging | No acceptance criteria, checklist, or next action |
32+
33+
**Epistemic Forge** forces a shared intermediate representation—a **claim lattice**—then packages it for the domain you’re in.
34+
35+
Academic anchor: **Toulmin’s model of argument** (claim · grounds · warrant · rebuttal · qualifier).
36+
See [`docs/ACADEMIC_FRAMEWORK.md`](docs/ACADEMIC_FRAMEWORK.md) · Toulmin, *The Uses of Argument* (1958/2003).
37+
38+
---
39+
40+
## Before / after (real run)
41+
42+
**Question:** *If the brain is a prediction machine, what happens to moral responsibility?*
43+
44+
### Before — baseline one-shot Q&A
45+
**Overall 0.37** · **Toulmin coverage 0.40**
46+
47+
```text
48+
Predictive processing and moral responsibility: On the question «If the brain
49+
is a prediction machine, what happens to moral responsibility?», a reasonable
50+
view is that responsibility still applies when agents can track norms, even if
51+
cognition is predictive. Keywords: predictive processing, responsibility, agency.
52+
In short, keep blame practices but update the metaphysics of agency.
53+
```
54+
55+
### After — Epistemic Forge claim lattice
56+
**Overall 1.00** · **Toulmin coverage 1.00** on this memo
57+
(full folder: [`examples/worked/01_philosophy_predictive_responsibility/`](examples/worked/01_philosophy_predictive_responsibility/))
58+
59+
The forge memo adds: explicit **claim IDs**, **supports**, **objections**, **confidence**, **dialectic (thesis/antithesis/steelman/synthesis)**, **limits & unknowns**, and **next actions**—the slots baseline Q&A skips.
60+
61+
Side-by-side writeup: [`examples/worked/BEFORE_AFTER_philosophy.md`](examples/worked/BEFORE_AFTER_philosophy.md)
62+
63+
---
64+
65+
## Benchmark (10 examples, measurable)
66+
67+
We compare **claim-lattice pipeline output** vs a **simple Q&A baseline** on **10 fixed cases** (philosophy ×3, kaggle ×3, freelance ×3, hybrid ×1).
68+
69+
Scoring is **deterministic** (no paid judge model): weighted Toulmin completeness + structure / actionability / humility.
70+
71+
| Metric | Baseline Q&A | Epistemic Forge | Lift |
72+
|---|---:|---:|---:|
73+
| **Overall quality (0–1)** | 0.319 | **0.951** | **+0.633 (~+199%)** |
74+
| **Toulmin coverage (0–1)** | 0.260 | **0.860** | **+0.600 (~+231%)** |
75+
| **Wins (overall)** || **10 / 10** ||
76+
| **Wins (Toulmin)** || **10 / 10** ||
77+
78+
Per-case table and method notes: [`docs/benchmark/BENCHMARK.md`](docs/benchmark/BENCHMARK.md) · raw JSON: [`docs/benchmark/benchmark_results.json`](docs/benchmark/benchmark_results.json)
79+
80+
```bash
81+
python scripts/run_benchmark.py --out docs/benchmark
82+
```
83+
84+
| ID | Domain | Baseline | Forge | Δ Overall |
85+
|---|---|---:|---:|---:|
86+
| p1–p3 | philosophy | ~0.37 | ~1.00 | ~+0.63 |
87+
| k1–k3 | kaggle | ~0.30 | ~0.94 | ~+0.64 |
88+
| f1–f3 | freelance | ~0.23 | ~0.90 | ~+0.67 |
89+
| h1 | hybrid | 0.50 | 1.00 | +0.50 |
90+
91+
> Interpretation for portfolio readers: the gain is not “smarter vibes”—it is **argument slot coverage + packaging**. That is what clients, reviewers, and hiring managers can inspect.
92+
93+
---
94+
95+
## Worked examples (full outputs)
96+
97+
| Domain | Example | Score / review |
1298
|---|---|---|
13-
| **L0** | Technique routing | Domain → families + layer flags |
14-
| **L1** | APE + OPRO | Seed instructions + score-history climb |
15-
| **L2** | Meta conductor | Claim / dialectic / writing / freelance / Kaggle experts |
16-
| **L3** | ToT (+ LATS cascade) | Beam search over framings; optional rollout polish |
17-
| **L4** | Self-Refine | Multi-aspect critique → revise loop |
18-
| **L5** | Reflexion + Voyager | Verbal trial memory + skill library |
19-
| **L6** | Stage shell | Artifacts, executive summary, peer-review rubric |
99+
| **Philosophy** | [Predictive processing & responsibility](examples/worked/01_philosophy_predictive_responsibility/) | 0.66 · minor revisions |
100+
| **Kaggle** | [Imbalanced tabular baseline](examples/worked/02_kaggle_imbalanced_baseline/) | 0.71 · minor revisions |
101+
| **Freelance** | [Climate-tech 2-week sprint](examples/worked/03_freelance_climate_sprint/) | 0.71 · minor revisions |
102+
103+
Index: [`examples/worked/README.md`](examples/worked/README.md)
20104

21-
> Novel angle: a **claim lattice** as the shared intermediate representation across philosophy, research writing, freelancing, and Kaggle planning—so epistemic humility travels with the deliverable.
105+
Each folder includes `executive_summary.md`, `research_memo.md`, domain packs (`client_brief.json` / `kaggle_spine.md`), and full `result.json` (route, trials, search nodes).
22106

23107
---
24108

@@ -28,9 +112,10 @@ Epistemic Forge is a **local-first Python package** (no paid API required) that
28112
git clone https://github.com/faresrafat3/epistemic-forge.git
29113
cd epistemic-forge
30114
pip install -e ".[dev]"
115+
pytest -q
31116
```
32117

33-
## Quick start (CLI)
118+
## Quick start
34119

35120
```bash
36121
epistemic-forge \
@@ -59,7 +144,7 @@ epistemic-forge \
59144
--out runs/freelance_demo
60145
```
61146

62-
## Python API
147+
### Python API
63148

64149
```python
65150
from epistemic_forge import run_pipeline
@@ -80,57 +165,58 @@ export_result(result, "runs/hybrid_demo")
80165
| File | Purpose |
81166
|---|---|
82167
| `executive_summary.md` | One-pager |
83-
| `research_memo.md` | Full lattice + framing + limits + actions |
168+
| `research_memo.md` | Claim lattice + framing + limits + actions |
84169
| `client_brief.json` | Freelance pack (when domain fits) |
85170
| `kaggle_spine.md` | Notebook spine (when domain fits) |
86-
| `result.json` | Full machine-readable trace (route, trials, search nodes) |
171+
| `result.json` | Full trace (route, trials, search nodes) |
87172
| `MANIFEST.json` | Score, review, file list |
88173

89174
## Domains
90175

91176
`research` · `philosophy` · `writing` · `freelance` · `kaggle` · `hybrid`
92177

93-
## Why this is not “another prompt dump”
178+
---
94179

95-
1. **Executable pipeline** with tests (`pytest`)
96-
2. **Shared IR** (claims) across domains
97-
3. **Explicit ARSENAL mapping** with a run log: [`docs/ARSENAL_RUN_LOG.md`](docs/ARSENAL_RUN_LOG.md)
98-
4. **Memory**: verbal lessons + reusable skills
99-
5. **Publishable packaging**: `pyproject.toml`, CLI entrypoint, MIT license
180+
## ARSENAL mapping (how the sausage is made)
181+
182+
| Layer | ARSENAL idea | In this repo |
183+
|---|---|---|
184+
| **L0** | Technique routing | Domain → families + layer flags |
185+
| **L1** | APE + OPRO | Seed instructions + score-history climb |
186+
| **L2** | Meta conductor | Claim / dialectic / writing / freelance / Kaggle experts |
187+
| **L3** | ToT (+ LATS cascade) | Beam search over framings; optional rollout polish |
188+
| **L4** | Self-Refine | Multi-aspect critique → revise loop |
189+
| **L5** | Reflexion + Voyager | Verbal trial memory + skill library |
190+
| **L6** | Stage shell | Artifacts, executive summary, peer-review rubric |
191+
192+
Design log: [`docs/ARSENAL_RUN_LOG.md`](docs/ARSENAL_RUN_LOG.md)
100193

101194
## Project layout
102195

103196
```text
104197
epistemic_forge/
105-
pipeline/ # L0–L6 orchestration
106-
experts/ # L2 specialist modules
107-
memory/ # Reflexion + skill library
108-
io/ # export
109-
data/samples/ # demo specs
198+
pipeline/ # L0–L6 orchestration
199+
experts/ # domain specialists
200+
memory/ # Reflexion + skills
201+
benchmark/ # baseline vs forge metrics
202+
io/ # export
203+
examples/worked/ # full philosophy / kaggle / freelance runs
204+
docs/benchmark/ # numbers + method
205+
docs/ACADEMIC_FRAMEWORK.md
110206
tests/
111-
docs/ARSENAL_RUN_LOG.md
112-
examples/
113207
```
114208

115-
## Tests
209+
## Tests & regenerate portfolio assets
116210

117211
```bash
118212
pytest -q
213+
python scripts/run_benchmark.py --out docs/benchmark
214+
python scripts/generate_worked_examples.py
119215
```
120216

121-
## Design principles (from ARSENAL)
122-
123-
1. Router first
124-
2. Optimize the instruction (APE / OPRO cascade)
125-
3. Conduct via experts
126-
4. Search framings deliberately (ToT; cascade when code-like)
127-
5. Refine with multi-aspect feedback
128-
6. Remember failures and skills
129-
7. Ship staged artifacts + review
130-
131-
## Related work
217+
## Related
132218

133-
- ARSENAL master pipeline: https://github.com/faresrafat3/arsenal-unified-master-pipeline
219+
- ARSENAL: https://github.com/faresrafat3/arsenal-unified-master-pipeline
134220
- Extractions archive: https://github.com/faresrafat3/llm-agent-research-extractions
135221

136222
## License
@@ -139,4 +225,4 @@ MIT — see [LICENSE](LICENSE).
139225

140226
## Disclaimer
141227

142-
Epistemic Forge **scaffolds thinking and packaging**. It does not replace domain expertise, human ethics review, or competition rules. Treat outputs as **provisional** claims with explicit limits.
228+
Epistemic Forge **scaffolds thinking and packaging**. It does not replace domain expertise, ethics review, or competition rules. Benchmark scores are **automatic structural metrics**, not human truth judgments. Treat outputs as **provisional** claims with explicit limits.

docs/ACADEMIC_FRAMEWORK.md

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Academic framework: Claim lattices and Toulmin’s model of argument
2+
3+
Epistemic Forge treats a **claim lattice** as a lightweight, multi-domain intermediate representation. The design is deliberately aligned with classical **argumentation theory**, especially Stephen Toulmin’s model of argument.
4+
5+
## Primary citation
6+
7+
> Toulmin, S. E. (1958/2003). *The Uses of Argument* (Updated ed.). Cambridge University Press.
8+
9+
Toulmin proposed that everyday and scientific arguments are better analyzed as **functional roles** than as pure syllogisms. The core layout (commonly taught in writing & critical reasoning) is:
10+
11+
| Toulmin element | Role | Epistemic Forge mapping |
12+
|---|---|---|
13+
| **Claim** | The conclusion being advanced | `Claim.text` (thesis node) |
14+
| **Data / Grounds** | Evidence supporting the claim | `Claim.support[]` |
15+
| **Warrant** | Principle linking data → claim | Instruction + expert integration narrative (“why this structure improves decisions”) |
16+
| **Backing** | Support for the warrant | Domain experts (methods notes, baselines, client constraints) |
17+
| **Qualifier** | Strength / modality of the claim | `Claim.confidence` (`sure` / `likely` / `possible` / `weak`) |
18+
| **Rebuttal** | Conditions of exception / objections | `Claim.objections[]` + dialectic antithesis/steelman |
19+
20+
## Why this fits the product
21+
22+
1. **Portable across domains** — The same slots structure a philosophy thesis, a freelance acceptance criterion, or a Kaggle “honest baseline” claim.
23+
2. **Humility is first-class** — Qualifiers and rebuttals are not optional decoration; they are scored in the benchmark.
24+
3. **Compatible with modern AI scaffolding** — Self-critique loops (Self-Refine / Reflexion) naturally *fill* rebuttal and qualifier slots rather than only polishing style.
25+
26+
## Related academic threads (secondary)
27+
28+
- **Argumentation schemes & critical questions** — Walton, Reed, Macagno (*Argumentation Schemes*, 2008): each scheme comes with defeaters; our objections list is a practical subset.
29+
- **IBIS / design rationale** — Kunz & Rittel (1970): issues–positions–arguments; our lattice is a compressed position/argument graph for delivery, not a full IBIS database.
30+
- **Computational argumentation** — Dung’s abstract argumentation frameworks (1995) formalize attack relations; we stay at the *informal-logic packaging* layer for human-readable deliverables.
31+
32+
## How the benchmark uses Toulmin
33+
34+
The benchmark metrics in `epistemic_forge/benchmark/metrics.py` score documents for presence of:
35+
36+
- claim, grounds, warrant, rebuttal, qualifier
37+
38+
plus packaging features (structure, actionability, humility).
39+
**Toulmin coverage** = fraction of the five core slots detected at score ≥ 0.5.
40+
41+
This is an **automatic, deterministic proxy**—useful for regression testing and portfolio evidence—not a substitute for human argument evaluation.
42+
43+
## Practical reading
44+
45+
- Toulmin (2003), ch. on the layout of arguments.
46+
- Introductory teaching notes on Toulmin in composition studies (claim–evidence–warrant).
47+
- For AI context: connect to self-critique / debate papers only as *process*, while Toulmin remains the *structure* of the artifact.
48+
49+
## One-sentence thesis
50+
51+
> Epistemic Forge operationalizes Toulmin’s layout as a **shared claim lattice IR**, then runs an ARSENAL-style pipeline to fill, stress-test, and package those slots for research, freelancing, and applied ML planning.

docs/benchmark/BENCHMARK.md

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Claim Lattice vs Baseline Q&A — Benchmark Report
2+
3+
Cases: **10**
4+
5+
## Summary
6+
7+
| Metric | Baseline | Epistemic Forge | Lift |
8+
|---|---:|---:|---:|
9+
| Overall quality (0–1) | 0.319 | 0.951 | **+0.633 (199%)** |
10+
| Toulmin coverage (0–1) | 0.260 | 0.860 | **+0.600 (231%)** |
11+
| Wins (overall) || **10/10** ||
12+
| Wins (Toulmin) || **10/10** ||
13+
14+
## Per-case results
15+
16+
| ID | Domain | Baseline | Forge | Δ Overall | Δ Toulmin |
17+
|---|---|---:|---:|---:|---:|
18+
| p1 | philosophy | 0.375 | 1.000 | +0.625 | +0.600 |
19+
| p2 | philosophy | 0.361 | 0.999 | +0.637 | +0.600 |
20+
| p3 | philosophy | 0.361 | 1.000 | +0.639 | +0.600 |
21+
| k1 | kaggle | 0.329 | 0.935 | +0.606 | +0.600 |
22+
| k2 | kaggle | 0.229 | 0.935 | +0.706 | +0.600 |
23+
| k3 | kaggle | 0.329 | 0.935 | +0.606 | +0.600 |
24+
| f1 | freelance | 0.131 | 0.837 | +0.706 | +0.600 |
25+
| f2 | freelance | 0.236 | 0.935 | +0.699 | +0.600 |
26+
| f3 | freelance | 0.329 | 0.935 | +0.606 | +0.600 |
27+
| h1 | hybrid | 0.504 | 1.000 | +0.496 | +0.600 |
28+
29+
## Method
30+
31+
Overall = weighted Toulmin completeness + structure/actionability/humility. Toulmin coverage = fraction of {claim, grounds, warrant, rebuttal, qualifier} present at score>=0.5. Deterministic, no external LLM judge.
32+
33+
Baseline = short unstructured Q&A template (no lattice, no objections section, no staged packaging).
34+
Forge = full ARSENAL-mapped pipeline output (`research_memo`).
35+
36+
## Academic anchor
37+
38+
Metrics map to Toulmin's argument model (claim, data/grounds, warrant, rebuttal, qualifier).
39+
See `docs/ACADEMIC_FRAMEWORK.md`.

0 commit comments

Comments
 (0)