Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
<a href="https://github.com/proteus-evolve/Proteus/actions/workflows/release-smoke.yml"><img src="https://github.com/proteus-evolve/Proteus/actions/workflows/release-smoke.yml/badge.svg" alt="release smoke"></a>
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green.svg" alt="MIT License"></a>
<img src="https://img.shields.io/badge/python-3.10%2B-blue.svg" alt="Python 3.10+">
<img src="https://img.shields.io/badge/version-0.2.0-informational.svg" alt="v0.2.0">
<img src="https://img.shields.io/badge/version-0.3.0-informational.svg" alt="v0.3.0">
<img src="https://img.shields.io/badge/status-research%20preview-orange.svg" alt="research preview">
</p>

Expand All @@ -26,7 +26,7 @@
<a href="docs/RECIPES.md">Recipes</a> •
<a href="docs/BENCHMARKS.md">Bring a Benchmark</a> •
<a href="docs/MEASUREMENTS.md">Add a Measurement</a> •
<a href="docs/releases/v0.2.0.md">v0.2.0 Notes</a> •
<a href="docs/releases/v0.3.0.md">v0.3.0 Notes</a> •
<a href="environments/README.md">Environments</a> •
<a href="#-measurement">Measurement</a>
</p>
Expand Down Expand Up @@ -273,7 +273,7 @@ roots, so the evolving agent can never read its own condition.

## 📊 Status

`v0.2.0` (research preview). Working today: the offline `minimal` harness; the live `llm`
`v0.3.0` (research preview). Working today: the offline `minimal` harness; the live `llm`
harness; pinned, source-evolving DeepSeek Harness and Pi adapters with frozen per-episode
activation, automatic rollback, exact-tree boundary gates, rebuild caching, turn budgets,
phase-aware act-priority budget plans and agent-authored checkpoint tracking, and task
Expand Down
86 changes: 86 additions & 0 deletions docs/releases/v0.3.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Proteus v0.3.0

Proteus v0.3.0 makes the *episode* — not just the harness snapshot — a controlled,
versioned experimental object. It ships a phase-aware turn-budget protocol and a default
epistemic episode protocol, both recorded in the manifest and locked across resume, and it
grows the lightweight benchmark shelf (HumanEval, MBPP) on a shared, hardened
process-isolation harness with reproducibly pinned datasets.

## Highlights

### Phase-aware episode budgets

- A harness-neutral `BudgetPlan` (`proteus.core.budget`) turns the public knobs into one
execution policy. Alongside the legacy `max_turns` / `min_turns_per_phase` behavior it
adds an explicit per-phase allocation (`--phase-turns observe=…,propose=…,act=…,reflect=…`),
a separate burst ceiling (`--hard-max-turns`), and an agent-authored checkpoint reserve
(`--checkpoint-turns`).
- Unused early-phase quota and the burst allowance are prioritized for act, while reflect
stays bounded by its own allocation — the implementation window can expand without making
the hard ceiling the expected cost.
- With `--announce-budget`, each phase begins with a live used/remaining budget header the
subject can plan against. The full budget condition (version, normal/hard limits,
per-phase plan, checkpoint reserve) is recorded in `manifest.json` and locked across
resume.
- The bundled `dsh`, `pi`, `minimal`, and `llm` adapters consume the plan through the core
helpers (`budget_plan`, `phase_prompt`); the copy-paste adapter template does too. The
`aki` research adapter keeps its own native turn policy and is the documented exception.

### Default epistemic episode protocol

- Every default episode now carries a versioned Proteus protocol: external evaluators are
treated as *evidence*, not automatically as a complete definition of success. An
evaluator may fully operationalize a narrow goal or only partially cover a broad one, and
the harness is asked to judge sufficiency and evolve its own tests or evaluators when that
reduces uncertainty — never merely to satisfy the protocol.
- No-goal runs get neutral exploration prompts. They are no longer silently framed as
"improve reliability" runs, and the harness may formulate and revise its own provisional
goals and evaluation machinery as part of its evolving state.
- The protocol wording is deliberately conditional, so it reveals neither the existence of a
HIDDEN evaluator nor an unstated objective. Its version is recorded in the manifest, so a
run cannot resume across a changed protocol condition.

### Lightweight benchmarks on a hardened isolation harness

- New `humaneval:<id>` and `mbpp:<id>` `BenchTask` packs, wired into the CLI and gradable
offline.
- Both packs share one parent/worker process-isolation harness (`proteus.bench._isolation`)
for running agent-authored code: candidate results travel over a dedicated inherited file
descriptor rather than parseable stdout, worker and executor subprocesses are reaped by
process group, and the driver is installed with `O_EXCL | O_NOFOLLOW`. Grading still routes
through the episode sandbox (`run_python`) and never falls back to host execution.
- Official datasets are downloaded on first use, **commit-pinned and sha256-verified** before
an atomic publish into the cache (`proteus.bench._datasets`); `PROTEUS_HUMANEVAL_PATH` /
`PROTEUS_MBPP_PATH` bypass the download for offline or vendored use.

### DSH cold-start hardening

- A DSH candidate that changes `package.json` or the workspace topology is re-resolved
against the image's offline pnpm store with a frozen lockfile before it can build. A new
workspace package gets real runtime links; an undeclared or unavailable dependency is
rejected without network access. Pure code edits keep the baked links and skip the work.

## Compatibility and migration

- **Do not resume a v0.2.x sweep with v0.3.0.** v0.3 records a `default_episode_protocol_version`
(and, for explicit budget runs, a `budget_protocol` block) in the manifest's locked
condition. Because the episode protocol a v0.2 sweep ran under differs, v0.3 refuses to
resume it rather than joining episodes from two protocols into one trajectory. Finish the
run with v0.2.x, or start a new `--out`. `--on-existing overwrite` remains available after
backing up the old sweep and intentionally removes the complete previous sweep state.
- The default episode phase prompts changed (goal-run wording plus a neutral no-goal
variant). Announcing a budget also changes phase prompts. Both are experimental
conditions by design; they are recorded, not silent.
- `--phase-turns` requires a positive `--max-turns` it sums to; `--checkpoint-turns`
requires `--announce-budget` and a harness with native or framework continuity. Existing
`max_turns` / `min_turns_per_phase` configurations are unchanged.

## Install

```bash
python -m pip install --upgrade proteus-evolve==0.3.0
```

Proteus remains a research preview and requires Python 3.10+. The v0.3.0 release gate uses
the pinned DSH `dsh-v0.1.0-rc.7` and Pi `v0.84.2` source environments so published results
remain reproducible; the separate upstream canary tracks newer upstream versions.
2 changes: 1 addition & 1 deletion proteus/__init__.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"""Proteus — a harness-agnostic framework for agent self-evolution."""
__version__ = "0.2.0"
__version__ = "0.3.0"
27 changes: 15 additions & 12 deletions proteus/examples/adapter_template.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@
from typing import Sequence

from proteus.core.adapter import ActionEvent, EpisodeResult, EpisodeSpec, Surface
from proteus.core.budget import PHASES, budget_plan, phase_prompt
from proteus.core.disposition import Disposition
from proteus.core.episode import PHASES # ("observe", "propose", "act", "reflect")


class TemplateHarness:
Expand Down Expand Up @@ -122,10 +122,12 @@ def run_episode(self, spec: EpisodeSpec) -> EpisodeResult:

The writable harness is at `spec.root / "harness"`. If you declared
`staged_activation = True`, execute from `spec.active_root` instead and keep the
candidate separate. Turn budget: `spec.max_turns` bounds the episode;
`spec.min_turns_per_phase` reserves turns for later phases (a phase that reaches
its reserved line ends early — only a spent budget ends the episode). Honour it or
the mid-phase cap is untestable.
candidate separate. Turn budget: let `proteus.core.budget` own the allocation —
`budget_plan(spec)` validates the legacy `max_turns` / `min_turns_per_phase` knobs
and the explicit phase plan alike; `plan.stop_at(phase, used)` is the cumulative
call at which this phase must stop, and `phase_prompt(spec, phase, used)` prepends
the live budget header when the run announces it. Reaching a phase's stop line ends
that phase; only the hard limit ends the episode.
"""
harness = spec.root / "harness"
(harness / "notes").mkdir(parents=True, exist_ok=True) # restore may drop empty dirs
Expand All @@ -136,17 +138,18 @@ def run_episode(self, spec: EpisodeSpec) -> EpisodeResult:
turn = 0
writes = 0
capped = False
min_pp = int(getattr(spec, "min_turns_per_phase", 0) or 0)
plan = budget_plan(spec)
with trace_path.open("w", encoding="utf-8") as sink:
for idx, phase in enumerate(PHASES):
for phase in PHASES:
if plan.enabled and turn >= plan.hard_limit:
capped = True
if capped:
break
stop_at = (spec.max_turns - min_pp * (len(PHASES) - idx - 1)
if spec.max_turns else 0)
prompt = spec.phase_prompts.get(phase, "")
stop_at = plan.stop_at(phase, turn)
prompt = phase_prompt(spec, phase, turn)
for tool, surface, text in self._stub_policy(phase, prompt, spec.episode, rng):
if spec.max_turns and turn >= stop_at:
capped = turn >= spec.max_turns
if plan.enabled and turn >= stop_at:
capped = turn >= plan.hard_limit
break
turn += 1
if tool == "write_note":
Expand Down
Loading