v100.38: factual training evidence, diverse fresh tasks and snapshot upgrades

Author: Mantiee

V100_AUDIT.md

@@ -1,4 +1,84 @@
-# V100 mission release v100.24
+# V100 mission audit v100.38
+
+This inventory separates source implementation from configuration, execution and
+demonstrated improvement. Passing controller tests does not certify GPU stability,
+profitable trading or a successful model promotion. It is not a declaration that
+every requested feature is complete or optimal.
+
+## Current requirements inventory
+
+| Requirement | Source mechanism | Limit or evidence still needed |
+|---|---|---|
+| Autonomous self-upgrade loop | `mission.py`, `continuous.py`, `supervisor.py` | Accepted updates and numerical health, not loop count alone |
+| Operator-owned long-term objective | `goals.py`, `chat_goals.py`, `planning.py` | Only a clear current operator request can change it; upgrades preserve old goals |
+| Short/mid-term plans from ordinary chat | `mission_chat.py`, `planning.py` | Versioned host receipts; a written plan does not mean its experiment ran |
+| Chat while research continues | Persistent queue, accepted master and explicit RTX delegate | One V100 slot shares inference; exclusive training prevents accepted-master chat temporarily |
+| Factual status in chat | `mission_evidence.py`, supplied automatically and as a research tool | Report paths, successful optimizer counters and accepted versions are distinct; missing counters mean unknown |
+| Long context, Flash Attention, KV | Native configuration and fit checks | Device execution and long-input accuracy; free VRAM alone does not establish speed |
+| Adaptive thinking/output | Validated `research_policy.py` budgets | Research only; fixed evaluations retain comparable settings |
+| Adaptive batching/speed | MTP/calibration/performance measurements | Master batch is a stopped-server benchmark proposal, not unrestricted live reconfiguration |
+| Parallel master/helper research | `paper_agents.py`, resident drones | One RTX request and bounded CPU jobs; concurrent master training/inference models on the single V100 are not enabled |
+| Independent A/B evolution/crossbreeding | `lineages.py`, `competition.py`, `crossbreeding.py` | Accepted separate ancestors require passing updates; initial branches share the base |
+| Architecture activation/rollback | `architecture_promotion.py`, foundation/custom trials | Fixed/public/fresh gates and actual inference probe; no hardware promotion inferred |
+| New independent training/audit tasks | `curriculum.py`, split ledgers, `fresh_audit.py` | Fresh calculations, sequences and typed extraction; finite synthetic coverage is not all-domain robustness |
+| Official AI benchmark comparisons | Pinned LiveBench tasks/scorers, same-question published references | A selected panel is not the full leaderboard; provider settings differ; scorer errors block certification |
+| Source/CPU/RTX drones | Bounded persistent job queue | Configured and running jobs differ; limits and game pauses can defer work |
+| Full tool evidence and compact context | SHA256 archives, paged reads and compact capability routing | Previews are incomplete; oversized original input can still fail explicitly |
+| Original memory and summaries | Archive, hierarchical summaries, CPU lexical/embedding retrieval | Retrieval/provenance quality; recursive model transcripts are not facts |
+| Read/copy own code, sandbox files | `self_code.py`, private namespace and guest | Host controller protected; arbitrary shell and modified copies operate in private guest |
+| Linux GUI/browser/tools/vision | Debian VM, public HTTP(S) broker, RTX vision | Guest/device verification; GUI is not required for native training or calculators |
+| Resource/error recovery and uptime | Bounded resources, deadlines, restart backoff, rollback | Best-effort recovery, not unconditional 24/7 uptime or stability |
+| Restrict helper only on Windows | Isolated runtime, smaller batches/context, pacing, LoL pause | No hard per-process GPU-utilization/power cap; blackscreen cause unproven |
+| Use Windows CPU/RAM/owned machines | Authenticated shared-folder workers, leases and tensor validation | Worker launcher must run there; mailbox configuration alone is insufficient |
+| Additional Colab compute | Bounded interactive notebook and safe result import | No autonomous managed-free worker farm or cookie/account quota evasion |
+| Free external model consultation | Documented services, zero-price catalog, guest browser, explicit key routing | Existing session/key and allowance needed; stop or hand off at authentication/payment/limit walls |
+| Research beyond fixed assets | Catalog discovery, model-selected spot and configurable feeds | Equities/sports require verified fees, rights, liquidity and settlement semantics |
+| Historical/event-timed backtests | Chronological splits, events, cash/buy-hold and cost stress | Historical availability/leakage; past sample performance does not prove future income |
+| Learn from profitable/losing outcomes | Audited paper accounting, causal DPO, shadow reward head | Resolved valid outcomes required; not unbiased profit-maximizing policy RL |
+| Net results and local alerts | Audited paper reports and labeled rejected/accepted proposals | No real orders or demonstrated income edge inferred from signals |
+
+## v100.38 additions
+
+Campaign setup automatically creates a separate public benchmark snapshot when
+the installed official-grading adapter changed. A baseline from a different
+snapshot is retained as historical evidence, while accepted weights must be
+evaluated on the new snapshot before comparison/promotion. Existing input hashes
+are checked first; corrupted benchmark inputs are not silently replaced.
+
+`mission_evidence` reads bounded current-run health records and aggregate baseline
+summaries. Hidden audit answers never enter this tool. Attempted/global steps are
+not successful optimizer updates; old benchmark errors are labeled historical.
+Ordinary chat receives evidence even when it does not select the tool. The prompt
+rejects invented scorer causes, a GUI prerequisite for unrelated work, and claims
+that arithmetic practice is necessary for income.
+
+`read_tool_result` reads complete archived outputs in UTF-8-safe pages, checking
+SHA256 on every page. Arbitrary file paths and symlinks are rejected. A preview is
+still explicitly incomplete evidence. Original task/resource limits remain enforced.
+
+Fresh training and post-freeze audits add independently checked sequence operations
+and typed JSON extraction to the existing three calculation domains. Generation
+never reads benchmark answers. Audits stay single-use, drawn after candidate
+identities are frozen and reserved against training. Owned-worker examples are
+tested to fit without truncation. This adds coverage, not an income policy or
+universal no-forgetting proof.
+
+Earlier device logs established native inference, a small CUDA backward probe,
+private code namespaces and RTX identity/loading. They did not establish complete
+GUI readiness, master architecture promotion, latest-release GPU training or
+repeatable net profit. Those claims require fresh device reports.
+
+The official [NVIDIA MPS documentation](https://docs.nvidia.com/deploy/mps/when-to-use-mps.html)
+limits MPS support to Linux/QNX; it is not a Windows per-helper utilization cap.
+The [Colab FAQ](https://research.google.com/colaboratory/faq.html) distinguishes
+ordinary interactive work from distributed workers on free managed runtimes and
+disallows multiple accounts to bypass limits. Owned workers and interactive
+notebooks remain the implemented routes; no quota-evasion mechanism is added.
+
+## Historical release notes and earlier scope
+
+The sections below record earlier releases. Their test counts and resource defaults
+are historical, not evidence that every current feature is active.
 
 v100.24 normalizes Transformers 5 chat-tokenization mappings and handles Gemma's
 exact empty-thought inference suffix when encoding answer-only training records.

pyproject.toml

@@ -3,7 +3,7 @@ name = "rlms"
 authors = [
     {name = "Alex Zhang", email = "altzhang@mit.edu"},
 ]
-version = "0.1.3+v100.37"
+version = "0.1.3+v100.38"
 description = "Recursive Language Models."
 readme = "README.md"
 license = "MIT"

rlm/v100/campaign.py

@@ -106,7 +106,7 @@ def prepare(
 ) -> Path:
     from rlm.v100.mission import status
     from rlm.v100.public_benchmarks import current
-    from rlm.v100.public_benchmarks import prepare as prepare_benchmarks
+    from rlm.v100.public_benchmarks import prepare_current as prepare_benchmarks
     from rlm.v100.self_code import prepare as prepare_code
 
     if status(root)["running"]:
@@ -135,12 +135,28 @@ def stage(name, function):
     stage("helper-identity", lambda: configure_helper(root))
     stage("own-source-and-cpu-sandbox", lambda: prepare_code(root))
     if benchmarks:
-        stage("official-public-benchmarks", lambda: current(root) or prepare_benchmarks(root, 20))
+        stage("official-public-benchmarks", lambda: prepare_benchmarks(root, 20))
     if desktop:
         from rlm.v100.desktop import prepare as prepare_desktop
 
         stage("private-desktop", lambda: prepare_desktop(root))
     resources = profile.setdefault("resources", {})
+    if benchmarks and current(root) and resources.get("public_baseline"):
+        from rlm.v100.protection import file_hash
+
+        baseline = Path(resources["public_baseline"])
+        if file_hash(baseline) != resources["public_baseline_sha256"]:
+            raise ValueError("Pinned public baseline evidence changed")
+        old = json.loads(baseline.read_text())
+        snapshot_hash = file_hash(current(root) / "manifest.json")
+        if old.get("snapshot_sha256") != snapshot_hash:
+            resources["public_baseline_requires_refresh"] = {
+                "previous_report": str(baseline),
+                "new_snapshot_sha256": snapshot_hash,
+                "reason": "Evaluate accepted weights again; do not compare different public task snapshots",
+            }
+            resources.pop("public_baseline")
+            resources.pop("public_baseline_sha256")
     resources.update(
         fresh_audit_required=True,
         paper_research_enabled=resources.get("paper_research_enabled", False)

rlm/v100/capabilities.py

@@ -29,6 +29,7 @@ def audit(root: Path) -> dict:
         "last_cycle": evidence["last_learning_cycle"],
         "accepted_updates_this_run": evidence["accepted_weight_updates_this_run"],
         "training_metrics": evidence["training_metrics"],
+        "mission_evidence": evidence["mission_evidence"],
         "paper_blockers": evidence["paper_blockers"],
         "audited_paper_outcomes": evidence["available_audited_paper_outcomes"],
         "research_policy": settings(root),
@@ -88,6 +89,9 @@ def audit(root: Path) -> dict:
         ],
         "not_implemented": [
             "profit-maximizing policy RL",
+            "hard per-helper GPU utilization or power cap on Windows WDDM",
+            "accepted-master chat inference during exclusive master training on the single V100; RTX delegation is used instead",
+            "autonomous distributed free managed Colab workers",
             "automatic certification of sports/equities provider fees, account rights and settlement semantics",
             "automatic account creation or bypassing browser authentication/quotas",
         ],

rlm/v100/curriculum.py

@@ -1,32 +1,47 @@
 """Fresh training tasks from trusted calculators, independently of sealed audits."""
 
+import json
 import secrets
 import sqlite3
 from pathlib import Path
 
-from rlm.v100.insights import InsightQueue, verified_record
+from rlm.v100.insights import PROOF_DOMAINS, InsightQueue, verified_record
+
+
+def draw_task(domain: str, rng) -> dict:
+    if domain not in PROOF_DOMAINS:
+        raise ValueError("Unsupported independent proof domain")
+    a, b, c = (rng.randint(2, 999) for _ in range(3))
+    if domain == "sequence_transform":
+        payload = {
+            "operation": rng.choice(("sort", "reverse", "unique_sorted")),
+            "values": [rng.randint(-999, 999) for _ in range(rng.randint(4, 8))],
+        }
+        expression = json.dumps(payload, separators=(",", ":"))
+    elif domain == "structured_extraction":
+        data = {"label": "item-" + str(a), "count": b, "code": c, "enabled": bool(rng.randrange(2))}
+        expression = json.dumps(
+            {"data": data, "field": rng.choice(tuple(data))}, separators=(",", ":")
+        )
+    else:
+        expression = (
+            f"({a}+{b})*{c}"
+            if domain == "arithmetic"
+            else f"{a}*x+{b}={c}"
+            if domain == "linear_equation"
+            else f"{a}.{b:03d}*{c}/10000"
+        )
+    return {"kind": domain, "expression": expression}
 
 
 def request(root: Path, branch: str, domain: str, count: int) -> dict:
-    if (
-        domain not in ("arithmetic", "linear_equation", "decimal_calculation")
-        or type(count) is not int
-        or not 1 <= count <= 32
-    ):
+    if domain not in PROOF_DOMAINS or type(count) is not int or not 1 <= count <= 32:
         raise ValueError("Choose a supported proof domain and 1-32 new examples")
     queue = InsightQueue(root)
     rng, added = secrets.SystemRandom(), 0
     try:
         for _ in range(count * 10):
-            a, b, c = (rng.randint(2, 999) for _ in range(3))
-            expression = (
-                f"({a}+{b})*{c}"
-                if domain == "arithmetic"
-                else f"{a}*x+{b}={c}"
-                if domain == "linear_equation"
-                else f"{a}.{b:03d}*{c}/10000"
-            )
-            task = {"kind": domain, "expression": expression}
+            task = draw_task(domain, rng)
             row = verified_record(task)
             ledger = root / "research/state/splits.sqlite3"
             if ledger.exists():
@@ -44,5 +59,5 @@ def request(root: Path, branch: str, domain: str, count: int) -> dict:
         "new_verified_examples": added,
         "domain": domain,
         "weights_changed": False,
-        "scope": "New calculator-verified training examples; no benchmark or sealed audit answers",
+        "scope": "New independent proof-verified training examples; no benchmark or sealed audit answers. Extraction/sequence fixtures are synthetic, not income labels.",
     }

rlm/v100/distributed_compute.py

@@ -45,24 +45,14 @@ def mailbox_path(root: Path) -> Path:
 
 
 def fresh_records(domain: str, count: int = 64) -> list[dict]:
-    from rlm.v100.insights import verified_record
+    from rlm.v100.curriculum import draw_task
+    from rlm.v100.insights import PROOF_DOMAINS, verified_record
 
-    if (
-        domain not in ("arithmetic", "linear_equation", "decimal_calculation")
-        or not 32 <= count <= 128
-    ):
+    if domain not in PROOF_DOMAINS or not 32 <= count <= 128:
         raise ValueError("Compute examples require a trusted proof domain and 32-128 records")
     rng, rows = secrets.SystemRandom(), {}
     while len(rows) < count:
-        a, b, c = (rng.randint(2, 999) for _ in range(3))
-        expression = (
-            f"({a}-{b})*{c}"
-            if domain == "arithmetic"
-            else f"{a}*x+{b}={c}"
-            if domain == "linear_equation"
-            else f"{a}.{b:03d}*{c}/10000"
-        )
-        row = verified_record({"kind": domain, "expression": expression})
+        row = verified_record(draw_task(domain, rng))
         rows[row["group"]] = row
     return list(rows.values())
 

rlm/v100/drones.py

@@ -151,6 +151,8 @@ def execute(root: Path, job: dict) -> dict:
     client.identity()
     client.activity_actor = "resident-" + job["kind"]
     client.research_tool_names = {
+        "mission_evidence",
+        "read_tool_result",
         "read_public_page",
         "parallel_source_research",
         "search_memory",

rlm/v100/fresh_audit.py

@@ -5,7 +5,8 @@
 from pathlib import Path
 
 from rlm.v100.common import atomic_json
-from rlm.v100.insights import verified_record
+from rlm.v100.curriculum import draw_task
+from rlm.v100.insights import PROOF_DOMAINS, verified_record
 from rlm.v100.protection import compare_reports, file_hash, reserve_audit_sources
 
 
@@ -20,15 +21,7 @@ def create(root: Path, parent: dict, candidate: Path, folder: Path, count: int =
     folder.mkdir(parents=True, exist_ok=False)
     rng, rows, sources = secrets.SystemRandom(), [], []
     while len(rows) < count:
-        a, b, c = (rng.randint(2, 999) for _ in range(3))
-        kind = len(rows) % 3
-        task = (
-            {"kind": "arithmetic", "expression": f"({a}-{b})*{c}"}
-            if kind == 0
-            else {"kind": "linear_equation", "expression": f"{a}*x+{b}={c}"}
-            if kind == 1
-            else {"kind": "decimal_calculation", "expression": f"{a}.{b:03d}*{c}/10000"}
-        )
+        task = draw_task(PROOF_DOMAINS[len(rows) % len(PROOF_DOMAINS)], rng)
         record = verified_record(task)
         if record["group"] in sources:
             continue
@@ -57,7 +50,7 @@ def create(root: Path, parent: dict, candidate: Path, folder: Path, count: int =
             **identities,
             "suite_sha256": file_hash(suite),
             "count": count,
-            "scope": "Fresh finite arithmetic/equation/decimal checks, not proof of any user goal or universal retention",
+            "scope": "Fresh finite arithmetic/equation/decimal/sequence/extraction checks, not proof of any user goal or universal retention",
         },
     )
     return folder