You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Browse filesBrowse the repository at this point in the historyBrowse files
faresrafat3
committed
README: replace unverified marketing claims with benchmark-anchored copy
Removed or qualified the following unsupported claims:
- '100% predictable output shapes' -> 'deterministic output schemas'
- 'No confident mush. No hallucinations.' -> remove; replaced
with caveat 'reduces ungrounded claims; does not eliminate
hallucinations'
- 'Zero-Cost RAG' -> 'RAG' (DuckDuckGo is not zero-cost)
- '85% on API inference costs' -> not benchmarked, removed
- '80% fewer tokens per query' -> not benchmarked, removed
- 'Saves 2-3 months of Lead AI Engineer payroll' -> speculative,
removed
- 'dominates the 2026 landscape' -> 'Measured vs Marketing'
Added status note at the top pointing readers to
docs/benchmark/BENCHMARK.md for actual measured numbers (10-case
benchmark, +0.633 overall quality, +0.600 Toulmin coverage).
Copy file name to clipboardExpand all lines: README.md
+15-13Lines changed: 15 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,9 @@
8
8
9
9
**A Neuro-Symbolic State Machine for LLMs.**
10
10
11
-
Instead of chatty, unpredictable multi-agent systems that burn tokens, we use a deterministic **L0–L6 pipeline**. We enforce strict Toulmin-based Pydantic schemas at every node, allowing you to extract GPT-4 level deep tree-search and expert dialectics from *any* model with **100% predictable output shapes**.
11
+
Instead of chatty, unpredictable multi-agent systems that burn tokens, we use a deterministic **L0–L6 pipeline**. We enforce strict Toulmin-based Pydantic schemas at every node, allowing you to extract GPT-4 level deep tree-search and expert dialectics from *any* model with **deterministic output schemas**.
12
+
13
+
> **Status:** v1.0.0-rc1. The package is installable; tested end-to-end on 10 benchmark cases. Cost / token numbers in older marketing-style copy are *not* benchmarked — see `docs/benchmark/BENCHMARK.md` for what was actually measured (overall quality +0.633 vs baseline, Toulmin coverage +0.600, on 10 cases).
12
14
13
15
</div>
14
16
@@ -21,7 +23,7 @@ This engine was brutally refactored to represent the bleeding edge of AI researc
21
23
- 🌐 **Hermes Universal Routing:** Powered by `LiteLLM` and `Instructor`, you can swap the cognitive engine instantly. Use `openai/gpt-4o-mini`, `anthropic/claude-3-5-sonnet`, or `ollama/llama3` without changing a line of code.
22
24
- 🧬 **ADAS (Automated Design of Agentic Systems):** If your query is too niche, the L1.5 layer dynamically programs a brand-new Pydantic Schema and Python Expert Class at runtime to solve it.
23
25
- ⚖️ **Process Reward Models (PRM):** Our L3 Tree Search doesn't just guess. It uses an LLM-as-a-Judge to evaluate its own thoughts step-by-step, executing immediate rollbacks if it detects a logical fallacy (Test-Time Compute).
24
-
- 🌍 **Live Empirical Grounding (Zero-Cost RAG):** Before reasoning, the system executes a live web search (via DuckDuckGo) to fetch the scientific consensus. **No confident mush. No hallucinations.**Every claim requires an `epistemic_warrant` and a `potential_falsifier`.
26
+
- 🌍 **Live Empirical Grounding (RAG):** Before reasoning, the system executes a live web search (via DuckDuckGo) to fetch the scientific consensus. Every claim requires an `epistemic_warrant` and a `potential_falsifier` (reduces ungrounded claims; does not eliminate hallucinations).
25
27
- 🛡️ **Adversarial Self-Refine:** The L4 layer acts as a ruthless NeurIPS-level peer reviewer, aggressively critiquing the draft and forcing rewrites until it passes the threshold.
26
28
- 💰 **Cognitive Economy Tracker:** Built-in Token Budgeting. If the system approaches your budget limit mid-search, it halts the expensive tree expansion and safely falls back to the crystallization phase.
27
29
@@ -60,24 +62,24 @@ Please see our [Contributing Guidelines](CONTRIBUTING.md) and [PR Template](.git
60
62
<em>Built with precision — from Cairo to the open web.</em>
61
63
</div>
62
64
63
-
## 🏢 Why Epistemic Forge? (Enterprise Metrics vs Alternatives)
65
+
## 🏢 Why Epistemic Forge? (Measured vs Marketing)
64
66
65
-
When evaluating AI reasoning architectures for production, executives and researchers must balance **Intelligence, Determinism, and Cost**. Here is how Epistemic Forge dominates the 2026 landscape:
67
+
What was actually benchmarked vs what is qualitative opinion.
66
68
67
69
### 1. vs. Vanilla LLMs (ChatGPT / Claude 3.5 Sonnet directly)
68
-
***The Problem:** Single-shot LLMs suffer from "Confident Mush." They will confidently hallucinate an answer without rigorous warrants.
69
-
***The Epistemic Forge Edge:** We enforce **Toulmin Lattices**. The system will hard-fail rather than lie. It requires an `epistemic_warrant` and a `potential_falsifier` for every node.
70
-
***Cost Difference:**EF achieves GPT-4-level reasoning using cheap models (e.g., `gpt-4o-mini` or `Llama 3 8B`) via our PRM Tree Search, **saving up to 85% on API inference costs**.
70
+
***The Problem:** Single-shot LLMs can produce confident but ungrounded answers.
71
+
***The Epistemic Forge Edge:** We enforce **Toulmin Lattices**. The system requires an `epistemic_warrant` and a `potential_falsifier` for every node.
72
+
***Measured:**On 10 benchmark cases, EF outputs averaged overall quality **0.951** vs the baseline **0.319** (+0.633 lift, full report: `docs/benchmark/BENCHMARK.md`).
71
73
72
74
### 2. vs. Conversational Agents (CrewAI / AutoGen)
73
-
***The Problem:** Chat-based multi-agent systems burn massive context windows as agents "talk" to each other in natural language.
74
-
***The Epistemic Forge Edge:****Token Economy & Determinism.**We use Neuro-Symbolic State Machines. Experts communicate strictly via Pydantic JSON schemas, eliminating conversational bloat.
75
-
***Cost Difference:**EF consumes **~80% fewer tokens per query**, preventing API bankruptcy.
75
+
***The Problem:** Chat-based multi-agent systems burn context windows as agents "talk" in natural language.
76
+
***The Epistemic Forge Edge:****Token Economy & Determinism.** Neuro-Symbolic State Machines; experts communicate via Pydantic JSON schemas.
77
+
***Not measured:**specific token / cost ratios vs CrewAI or AutoGen on the same workload — qualitative opinion only.
76
78
77
79
### 3. vs. Build-it-Yourself Frameworks (LangGraph / DSPy)
78
-
***The Problem:** LangGraph is a powerful graph framework, but you start with a blank slate. You have to spend months writing the cognitive logic.
79
-
***The Epistemic Forge Edge:****Out-of-the-box SOTA.** EF comes pre-installed with bleeding-edge cognitive nodes: *Automated Design of Agentic Systems (ADAS)*, *Process Reward Models (PRM)*, and *Hegelian Synthesis*.
80
-
***Cost Difference:**Saves **2-3 months of Lead AI Engineer payroll**.
80
+
***The Problem:** LangGraph is a powerful graph framework, but you start with a blank slate.
0 commit comments