v59: retain dashboard evidence, recover goal work and commission verified future labels

Author: Mantiee

SYNTA_RECOVERY.md

@@ -0,0 +1,34 @@
+# Synta recovery, v59
+
+This fixes the uploaded dashboard snapshot's demonstrated failures, not a claim
+that every strategy is profitable or every candidate must be accepted.
+
+- Aggregate timeout retains only the latest same-run report, with explicit stale
+  state and timestamp. New-run optimizer counters never inherit old-run values.
+- Verified goal/plans and worker file snapshots remain readable independently.
+  Missing ledger/blocker data is unavailable, not an empty ledger or no blockers.
+- Dedicated report entry point avoids inference CLI initialization. Goal status
+  uses a bounded read-only SQLite snapshot, without competing writer transactions.
+  Failure to collect goal metrics does not discard other mission evidence.
+- Public tool receipts show actual state/ID/value/process exit code. Desktop
+  completion means process execution, with task outcome explicitly unverified.
+  Nonzero desktop exit is failed, not completed. The known placeholder/pass
+  simulation pattern is refused before queuing; other code still needs validation.
+- Upgrade archives UI-dominated income plans and restores observable financial
+  research tasks while retaining the operator goal. Model researchers cannot
+  replace financial plans with UI-only work. Explicit operator UI tasks stay allowed.
+- Financial missions collect up to two public reference series per observer tick,
+  rotated across eight separate Coinbase spot-price paths. Uniform controller
+  forecasts are precommitted for 300 seconds and resolved using independently
+  fetched future prices. They are labelled commissioning baselines, not model
+  discoveries, actionable signals or profit. Source outages create failure receipts,
+  never fabricated prices or labels. Other goals need their own measurable adapters.
+- Model forecasts and observation discovery remain available. The independent
+  reference loop supplies task labels even when a model makes invalid tool calls.
+  Existing source-disjoint CPU/master admission, retention and independent quality
+  gates remain mandatory. Insufficient/division-inadequate labels, unavailable
+  workers, fees or failed candidate gates remain explicit legitimate blockers.
+
+No change executes real orders, purchases, paid APIs or account creation. It does
+not certify the historical placeholder scalping/arbitrage scripts. Production
+execution and measured progress must be checked after installation on Debian.

SYNTA_REQUIREMENTS_AUDIT.md

@@ -1,4 +1,4 @@
-# Synta: requirement and verification audit, v58
+# Synta: requirement and verification audit, v59
 
 This audits the visible conversation. "Implemented" means code exists and the
 listed checks pass, not that it has run successfully on the operator's computers.
@@ -63,3 +63,10 @@ proposals and compatible identity-initialized residual growth. Existing queued
 master trials execute them in isolation. Activation additionally requires verified
 goal outcomes and strict measured task improvement. Native GGUF architecture is
 not reshaped in place. See `SYNTA_MORPHOLOGY.md` for the executable contract.
+
+## v59 runtime recovery
+
+See `SYNTA_RECOVERY.md`: same-run stale-report preservation, independent verified
+goal reads, read-only goal-status SQLite, actual process/forecast receipts, archived
+UI-plan recovery and independent forward-only financial reference labels. These
+labels commission supervised task data, not model skill or profitable strategies.

pyproject.toml

@@ -3,7 +3,7 @@ name = "rlms"
 authors = [
     {name = "Alex Zhang", email = "altzhang@mit.edu"},
 ]
-version = "0.1.3+v100.58"
+version = "0.1.3+v100.59"
 description = "Recursive Language Models."
 readme = "README.md"
 license = "MIT"

rlm/v100/background_observer.py

@@ -20,7 +20,20 @@ def tick(root: Path) -> None:
         ActivityLog(root, "controller", "goal-learning").write(
             "errors", "goal-observer-failed", {"detail": str(error)[:400]}
         )
+    from rlm.v100.goal_observer import tick as reference_tick
+
+    try:
+        reference_tick(root)
+    except (OSError, ValueError, RuntimeError, requests.RequestException) as error:
+        ActivityLog(root, "controller", "goal-observer").write(
+            "errors", "reference-collection-failed", {"detail": str(error)[:400]}
+        )
     compute_tick(root)
+    from rlm.v100.goal_learning import status as goal_status
+
+    atomic_json(
+        root / "research/goal-learning/status.json", {**goal_status(root), "updated": time.time()}
+    )
     ledger = root / "research/paper/ledger.sqlite3"
     if not ledger.exists():
         return

rlm/v100/dashboard_layout.py

@@ -85,14 +85,14 @@
 for(const e of result.events||[]){const d=document.createElement('details'),title=document.createElement('summary'),body=document.createElement('pre');title.textContent=[e.time,e.branch,e.actor,e.kind].filter(Boolean).join(' | ');body.textContent=JSON.stringify({context:e.context,payload:e.payload,id:e.id},null,2);d.append(title,body);rows.append(d)}cursor=result.next_cursor;status.textContent=`${rows.children.length} events shown | ${cursor} / ${result.archive_bytes||0} archive bytes`;button.textContent=result.has_more?'Load next 100 actions':'Check for new actions';}catch(error){status.textContent='Archive unavailable: '+error.message}finally{button.disabled=false}}
 button.onclick=()=>load();select.onchange=()=>load(true);
 }
-function learningPanel(data){rememberDisclosures();const p=panel('goal-learning-panel','Goal-directed ML pipeline',$('plan-panel'));let body=$('goal-learning-view');if(!body){body=document.createElement('div');body.id='goal-learning-view';p.append(body)}body.replaceChildren();const evidence=data.report?.mission_evidence||{},g=evidence.goal_learning||{};textField(body,'Active goal ID',g.goal_id||'No goal recorded');for(const [k,v] of Object.entries(g.learning_task||{}))textField(body,k,v);textField(body,'Recent forecasts',fmt(g.forecasts?.length));textField(body,'Recent resolved forecasts',g.recent_resolved);textField(body,'Recent Brier score',g.recent_brier);textField(body,'Admitted training examples',g.training_admission?.admitted);textField(body,'Confirmed optimizer updates',fmt(evidence.optimizer_updates_observed));textField(body,'Accepted weight updates',fmt(evidence.accepted_weight_updates_this_run));detailsField(body,'Training admission evidence',JSON.stringify(g.training_admission||{}),'goal-admission');const dispatch=g.owned_cpu_dispatch||{};textField(body,'Owned CPU dispatch',dispatch.state||'Not observed');textField(body,'Dispatch blocker',dispatch.reason);for(const branch of dispatch.branches||[])textField(body,'CPU trial',`${branch.id||'Not allocated'} | ${branch.state} | ${branch.reason||branch.scope||''}`);detailsField(body,'Owned CPU provenance and receipts',JSON.stringify(dispatch),'goal-compute');restoreDisclosures()}
+function learningPanel(data){rememberDisclosures();const p=panel('goal-learning-panel','Goal-directed ML pipeline',$('plan-panel'));let body=$('goal-learning-view');if(!body){body=document.createElement('div');body.id='goal-learning-view';p.append(body)}body.replaceChildren();const evidence=data.report?.mission_evidence||{},g=evidence.goal_learning||{};textField(body,'Active goal ID',g.goal_id||(data.report?.plans?.long?.id?'Goal metrics unavailable':'Goal status unavailable'));for(const [k,v] of Object.entries(g.learning_task||{}))textField(body,k,v);textField(body,'Recent forecasts',fmt(g.forecasts?.length));textField(body,'Recent resolved forecasts',g.recent_resolved);textField(body,'Recent Brier score',g.recent_brier);textField(body,'Admitted training examples',g.training_admission?.admitted);textField(body,'Confirmed optimizer updates',fmt(evidence.optimizer_updates_observed));textField(body,'Accepted weight updates',fmt(evidence.accepted_weight_updates_this_run));detailsField(body,'Training admission evidence',JSON.stringify(g.training_admission||{}),'goal-admission');textField(body,'Verified total outcomes',g.resolved_total);detailsField(body,'Independent observation pipeline',JSON.stringify(g.observer||{}),'goal-observer');const dispatch=g.owned_cpu_dispatch||{};textField(body,'Owned CPU dispatch',dispatch.state||'Not observed');textField(body,'Dispatch blocker',dispatch.reason);for(const branch of dispatch.branches||[])textField(body,'CPU trial',`${branch.id||'Not allocated'} | ${branch.state} | ${branch.reason||branch.scope||''}`);detailsField(body,'Owned CPU provenance and receipts',JSON.stringify(dispatch),'goal-compute');restoreDisclosures()}
 function svg(tag,attrs,text){let node=document.createElementNS('http://www.w3.org/2000/svg',tag);for(const [key,val] of Object.entries(attrs))node.setAttribute(key,val);if(text!==undefined)node.textContent=text;return node}
 function render(data){if(latestInference&&inferenceStamp(latestInference)>=inferenceStamp(data.live))data=mergeInference(data,latestInference);latestSnapshot=data;const m=data.mission||{},r=data.report||{},ev=(m.state?.phase==='official-public-baseline'?r.mission_evidence?.official_benchmark:null)||m.evaluation||{},learn=m.learning||{},gpu=data.gpu||{};
 if(data.layout?.active_sha256&&layoutVersion!==data.layout.active_sha256){location.reload();return}$('layout').textContent='HTML mastera: /workspace/dashboard/index.html | '+(data.layout?.state||'oczekiwanie')+(data.layout?.error?' | '+data.layout.error:'');
-$('fresh').textContent='Zebrano: '+new Date(data.collected_at*1000).toLocaleString()+' | Raport zbierany co 20 s | Przebieg: '+fmt(m.run);
+$('fresh').textContent='Zebrano: '+new Date(data.collected_at*1000).toLocaleString()+' | Raport zbierany co 20 s | Przebieg: '+fmt(m.run)+(data.report_freshness?.state==='stale'?' | STALE REPORT: '+fmt(data.report_freshness.updated_at):'');
 workView(data);learningPanel(data);archivePanel();$('warning').textContent=(data.errors||[]).join('\n');$('cards').replaceChildren();card('Misja',m.running?'działa':'zatrzymana');card('Faza',m.state?.phase);card('Cykle uczenia',learn.completed_cycles);card('Zaakceptowane aktualizacje wag',r.accepted_weight_updates_this_run);card('Potwierdzone kroki optymalizatora',r.mission_evidence?.optimizer_updates_observed);card('Kontekst',m.state?.context_window);card('GPU - wykorzystanie',gpu.utilization===undefined?null:gpu.utilization+'%');card('VRAM zajęty / razem',gpu.memory_used===undefined?null:gpu.memory_used+' / '+gpu.memory_total+' MiB');card('GPU - temperatura / moc',gpu.temperature===undefined?null:gpu.temperature+' °C / '+gpu.power+' W');
 $('evaluation').textContent=ev.total?`${ev.completed}/${ev.total} ukończonych, ${ev.passed===undefined?'ocenianie oficjalnym graderem':ev.passed+' zaliczonych'} (${ev.state})`:'Brak bieżącego licznika ewaluacji';$('progress').max=ev.total||1;$('progress').value=ev.completed||0;$('case').textContent=ev.current_case?('Obecny przypadek: '+ev.current_case):'';
-const ledger=r.paper||{};$('paper').textContent=Object.entries(ledger.branches||{}).map(([name,b])=>`${name} | Equity ${fmt(b.equity)} ${ledger.currency||''} | Net P&L ${fmt(b.net_pnl_before_personal_tax)} | Costs ${fmt(b.modeled_costs)} | Resolved fills ${fmt(b.resolved_fills)}`).join('\n\n')||'No recorded paper ledger';$('blockers').textContent=(r.paper_blockers||[]).join('\n')||'No reported paper blockers';$('workers').textContent=JSON.stringify({drones:r.drones,external_compute:r.external_compute,desktop:r.desktop,official_benchmark:r.official_benchmark},null,2);$('log').textContent=data.controller_log||'Brak logu';$('evidence').textContent=JSON.stringify({last_cycle:learn.last_cycle,evidence:r.mission_evidence,collection_errors:data.errors},null,2);
+const ledger=r.paper||{};$('paper').textContent=Object.entries(ledger.branches||{}).map(([name,b])=>`${name} | Equity ${fmt(b.equity)} ${ledger.currency||''} | Net P&L ${fmt(b.net_pnl_before_personal_tax)} | Costs ${fmt(b.modeled_costs)} | Resolved fills ${fmt(b.resolved_fills)}`).join('\n\n')||'Paper ledger status unavailable';$('blockers').textContent=(r.paper_blockers||[]).join('\n')||(Array.isArray(r.paper_blockers)?'No blockers in recorded snapshot':'Paper blockers unavailable');$('workers').textContent=JSON.stringify({drones:r.drones,external_compute:r.external_compute,desktop:r.desktop,official_benchmark:r.official_benchmark},null,2);$('log').textContent=data.controller_log||'Brak logu';$('evidence').textContent=JSON.stringify({report_freshness:data.report_freshness,last_cycle:learn.last_cycle,evidence:r.mission_evidence,collection_errors:data.errors},null,2);
 $('backtests').replaceChildren();$('chart').replaceChildren();const tests=r.recent_exploratory_backtests||[];let largest=Math.max(.01,...tests.map(t=>Math.abs(t.development_test?.net_return||0)));$('chart').append(svg('line',{x1:400,y1:0,x2:400,y2:220,stroke:'#8296b1'}));tests.forEach((test,i)=>{const result=test.development_test||{},tr=document.createElement('tr');[`${i+1}. ${test.parameters?.rule||''}`,pct(result.net_return),pct(result.max_sampled_drawdown),fmt(result.fills)].forEach(v=>{let td=document.createElement('td');td.textContent=v;tr.append(td)});$('backtests').append(tr);if(typeof result.net_return==='number'&&Number.isFinite(result.net_return)){let width=Math.abs(result.net_return)/largest*280,y=15+i*48;$('chart').append(svg('rect',{x:result.net_return<0?400-width:400,y,width,height:26,fill:result.net_return<0?'#e68d80':'#75c5ae'}),svg('text',{x:8,y:y+19,fill:'#dce6f5','font-size':14},`${i+1}. ${pct(result.net_return)}`))}});if(!tests.length)$('chart').append(svg('text',{x:20,y:50,fill:'#adbdd1'},'Brak zapisanych wyników'));
 $('reports').replaceChildren();for(const id of data.paper_reports||[]){const row=document.createElement('p');row.append(document.createTextNode(id+' '));for(const [name,label] of [['report.html','HTML i wykresy'],['report.json','JSON'],['trades.csv','CSV']]){const a=document.createElement('a');a.href='/paper/'+encodeURIComponent(id)+'/'+name;a.target='_blank';a.rel='noopener';a.textContent=label;row.append(a,document.createTextNode(' · '))}$('reports').append(row)}
 }

rlm/v100/dashboard_snapshot.py

@@ -0,0 +1,59 @@
+"""Fast independent file snapshots when an aggregate report is unavailable.
+
+Never reuse previous-run training counters or invent empty financial ledgers.
+"""
+
+import json
+from pathlib import Path
+
+
+def supplement(root: Path, mission: dict, report: dict, errors: list) -> dict:
+    from rlm.v100.goals import load_goal
+    from rlm.v100.planning import read
+
+    result = dict(report)
+    for key, filename in (
+        ("drones", "research/drones-status.json"),
+        ("external_compute", "research/compute-status.json"),
+        ("desktop", "research/desktop/status.json"),
+    ):
+        path = root / filename
+        if path.exists():
+            try:
+                if path.stat().st_size > 2 * 2**20:
+                    raise ValueError("Snapshot exceeds 2 MiB")
+                value = json.loads(path.read_text())
+                if not isinstance(value, dict):
+                    raise ValueError("Snapshot must be an object")
+                result[key] = value
+            except (OSError, ValueError) as error:
+                errors.append(f"{key} snapshot unavailable: {str(error)[:200]}")
+    try:
+        goal = load_goal(root)
+        if goal:
+            result["plans"] = read(root)
+            evidence = dict(result.get("mission_evidence", {}))
+            evidence.setdefault("run", mission.get("run"))
+            learning = dict(evidence.get("goal_learning", {}))
+            if learning.get("goal_id") != goal["id"]:
+                learning = {
+                    "goal_id": goal["id"],
+                    "scope": "Goal read from its verified file; forecast and training metrics unavailable",
+                }
+            for name, filename in (
+                ("cached", "research/goal-learning/status.json"),
+                ("observer", "research/goal-observer/status.json"),
+                ("owned_cpu_dispatch", "research/goal-compute/status.json"),
+            ):
+                path = root / filename
+                if path.exists() and path.stat().st_size <= 2 * 2**20:
+                    snapshot = json.loads(path.read_text())
+                    if name == "cached" and snapshot.get("goal_id") == goal["id"]:
+                        learning.update(snapshot)
+                    elif name != "cached" and snapshot.get("goal_id", goal["id"]) == goal["id"]:
+                        learning[name] = snapshot
+            evidence["goal_learning"] = learning
+            result["mission_evidence"] = evidence
+    except (OSError, ValueError) as error:
+        errors.append("Goal snapshot unavailable: " + str(error)[:200])
+    return result

rlm/v100/drones.py

@@ -53,6 +53,14 @@ def schedule(root: Path, branch: str, kind: str, payload: str, interval: int) ->
     maximum = 12000 if kind in ("python", "desktop") else 1500 if kind == "source" else 400
     if not isinstance(payload, str) or not 1 <= len(payload) <= maximum:
         raise ValueError("Drone payload exceeds its budget")
+    if (
+        kind in ("desktop", "python")
+        and "placeholder" in payload.lower()
+        and re.search(r"(?m)^\s*pass\s*(?:#.*)?$", payload)
+    ):
+        raise ValueError(
+            "Placeholder script refused: implement the experiment or report its missing inputs; printing a simulation banner is not execution"
+        )
     if type(interval) is not int or interval != 0 and not 300 <= interval <= 86400:
         raise ValueError("Drone interval: 0 once, or 300-86400 seconds")
     if kind == "source":
@@ -125,7 +133,10 @@ def execute(root: Path, job: dict) -> dict:
     if job["kind"] == "desktop":
         from rlm.v100.desktop import run
 
-        return run(root, job["payload"])
+        return {
+            **run(root, job["payload"]),
+            "completion_scope": "Guest process execution only; task correctness and income effects are not independently verified",
+        }
     if job["kind"] == "python":
         from rlm.v100.research_sandbox import run
 
@@ -194,6 +205,25 @@ def finish(root: Path, identity: str, value: dict) -> None:
         state = (
             "cancelled" if row["state"] == "cancelled" else "queued" if interval else "completed"
         )
+        if (
+            row["kind"] != "compute-audit"
+            and value.get("status") == "failed"
+            and not interval
+            and state != "cancelled"
+        ):
+            state = "failed"
+        if (
+            row["kind"] in ("desktop", "python")
+            and type(value.get("exit_code")) is int
+            and value["exit_code"] != 0
+        ):
+            value = {
+                **value,
+                "status": "execution failed",
+                "completion_scope": "Process failed; no task success receipt",
+            }
+            if not interval and state != "cancelled":
+                state = "failed"
         if (
             row["kind"] == "compute-audit"
             and value.get("status") == "failed"