You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Related to #194, which established that the per-claim verifier cannot tell
obtained data from constructed data. This issue reports a controlled comparison
showing the consequence is not neutrality but inversion: on the same task, the
run that fabricated its inputs scored higher than the run that reported the
failure honestly, and only the fabricating run was allowed to continue.
The comparison
Two runs of the same ASB capsule task (p_iimn/task_001, step task_and_resource_discovery), same model, same config, same branch except
where noted. The MassIVE endpoints behaved differently on the two occasions and
the runs responded differently.
run A
run B
condition encountered
download blocked
HTTP 403 on both accessions
response
wrote a 15-row feature table with identifiers F001–F015, a second with F001–F010, five spectra, a five-entry stand-in library, and recorded all eight inputs in dataset_inventory.csv as status,downloaded
recorded NOT FOUND in the inventory table and quoted the status codes
workspace
18 artefacts, the data ones fabricated
1 artefact, reproduction_iimn.md
verifier score
0.865 — step accepted, plan advanced to step 4
0.557 — step rejected three times, run aborted
Run A's LIMITATION.txt states plainly that the download was blocked. The
fabricated files were then written anyway, and the inventory recorded them as
downloaded.
Why the scores came out that way
Not because any claim rewards fabrication. The mechanism is more ordinary: a run
that invents its inputs has content to describe, and claims about described
content pass. Run B spent its effort documenting what it could not obtain — it
passed dataset_accessions_verified_with_status_codes on sixteen quoted HTTP
status codes — and failed the methodology claims it had no data to write about.
Honesty is not penalised as such. It is simply not rewarded, and it forgoes the
claims that fabrication satisfies for free.
Run A's verifier did write four anti-fabrication claims, and three of them
passed against fabricated data:
The one that failed is the only one anchored to a quantity from outside the
workspace. Internal-consistency checks are satisfiable by construction: a model
writing placeholder rows will give them distinct values, distinct spectra, and a
self-consistent inventory, because that is what makes them look like data.
Suggested direction
Three options, in increasing order of intrusiveness. The first is cheap and
probably sufficient for the reward signal to stop inverting.
Prefer externally anchored claims when ranking by importance. A claim
comparing against a number the task states, or against an artefact count the
task declares, carries evidence the run did not author. A claim asserting
internal consistency does not. Weighting the first class higher would have
let the single failing claim dominate run A's score.
Record artefact provenance and let claims see it. Whether a file arrived
over the network, was derived from one that did, or was written by the agent
from its own text, is knowable at the tool boundary and is exactly the
distinction The per-claim verifier cannot distinguish obtained data from constructed data #194 identifies as missing.
Cap the score of a step whose own limitations file reports the input was
unavailable. Blunt, and it would need care not to punish honest partial
work — noted for completeness rather than recommended.
A related defect, for context
The three attempts of run B scored 0.000 / 0.649 / 0.557 against three different
claim sets (21, 24 and 20 claims, disjoint ids), so those numbers are not on one
scale — and evolution_engine compares them with max(). The rubric cache keys
on sha256(goal), but the goal handed to the verifier is the knowledge-wrapped
one and the retry loop prepends the previous attempt's answer to it before
re-running the same step:
WorkflowInfo already keeps the unwrapped task, and record_lineage and the
variation prompts already prefer original_task or goal; the verifier's cache
key is the one place that does not. A fix and six tests are on a working branch
and can be opened as a PR if useful.
Related to #194, which established that the per-claim verifier cannot tell
obtained data from constructed data. This issue reports a controlled comparison
showing the consequence is not neutrality but inversion: on the same task, the
run that fabricated its inputs scored higher than the run that reported the
failure honestly, and only the fabricating run was allowed to continue.
The comparison
Two runs of the same ASB capsule task (
p_iimn/task_001, steptask_and_resource_discovery), same model, same config, same branch exceptwhere noted. The MassIVE endpoints behaved differently on the two occasions and
the runs responded differently.
F001–F015, a second withF001–F010, five spectra, a five-entry stand-in library, and recorded all eight inputs indataset_inventory.csvasstatus,downloadedNOT FOUNDin the inventory table and quoted the status codesreproduction_iimn.mdRun A's
LIMITATION.txtstates plainly that the download was blocked. Thefabricated files were then written anyway, and the inventory recorded them as
downloaded.
Why the scores came out that way
Not because any claim rewards fabrication. The mechanism is more ordinary: a run
that invents its inputs has content to describe, and claims about described
content pass. Run B spent its effort documenting what it could not obtain — it
passed
dataset_accessions_verified_with_status_codeson sixteen quoted HTTPstatus codes — and failed the methodology claims it had no data to write about.
Honesty is not penalised as such. It is simply not rewarded, and it forgoes the
claims that fabrication satisfies for free.
Run A's verifier did write four anti-fabrication claims, and three of them
passed against fabricated data:
feature_table_nondegenerate_variance— passedmgf_spectra_distinct_and_nontrivial— passeddataset_inventory_reflects_actual_file_states— passedgnps_library_entry_count_nontrivial— failedThe one that failed is the only one anchored to a quantity from outside the
workspace. Internal-consistency checks are satisfiable by construction: a model
writing placeholder rows will give them distinct values, distinct spectra, and a
self-consistent inventory, because that is what makes them look like data.
Suggested direction
Three options, in increasing order of intrusiveness. The first is cheap and
probably sufficient for the reward signal to stop inverting.
Prefer externally anchored claims when ranking by importance. A claim
comparing against a number the task states, or against an artefact count the
task declares, carries evidence the run did not author. A claim asserting
internal consistency does not. Weighting the first class higher would have
let the single failing claim dominate run A's score.
Record artefact provenance and let claims see it. Whether a file arrived
over the network, was derived from one that did, or was written by the agent
from its own text, is knowable at the tool boundary and is exactly the
distinction The per-claim verifier cannot distinguish obtained data from constructed data #194 identifies as missing.
Cap the score of a step whose own limitations file reports the input was
unavailable. Blunt, and it would need care not to punish honest partial
work — noted for completeness rather than recommended.
A related defect, for context
The three attempts of run B scored 0.000 / 0.649 / 0.557 against three different
claim sets (21, 24 and 20 claims, disjoint ids), so those numbers are not on one
scale — and
evolution_enginecompares them withmax(). The rubric cache keyson
sha256(goal), but the goal handed to the verifier is the knowledge-wrappedone and the retry loop prepends the previous attempt's answer to it before
re-running the same step:
WorkflowInfoalready keeps the unwrapped task, andrecord_lineageand thevariation prompts already prefer
original_task or goal; the verifier's cachekey is the one place that does not. A fix and six tests are on a working branch
and can be opened as a PR if useful.