Skip to content

Experiment: does declared intent improve an authorization decision, or does argument-aware policy already? #5065

Description

@chernistry

This is an experiment, not a feature. It has a kill criterion. A result that says "do not build this" is a successful outcome.

Thesis

Identity plus role plus resource is not enough to authorize an agent, because the same principal legitimately does opposite things in different tasks. Reading a customer record is routine during a support task and alarming during a marketing task.

Hypothesis

Adding a declared, propagated intent to the authorization input catches a class of harmful action that identity-based authorization allows — without a false-deny rate that makes operators turn it off.

Why now, and why cheap here

src/bernstein/core/security/toolcall_identity.py:26-45 already computes an intent_digest alongside args_digest and run_journal_head. The plumbing exists and nothing reads it. Testing this idea here costs a corpus and a policy condition, not an architecture.

Existing alternatives

Argument-aware policy is the cheaper rival hypothesis: if a policy can see the tool's arguments, perhaps it does not need a separate notion of intent. This experiment must measure against that baseline, not against role-only authorization, or it will flatter itself.

Experiment

Build adversarial pairs: same agent, same user, same resource, same entitlement, different declared intent. Run three policies over the corpus:

  1. role-only
  2. argument-aware
  3. argument-aware + intent

Measure:

  • harmful actions caught (2) vs (3)
  • false denies introduced by (3)
  • injection resistance: does a hostile tool result change the recorded intent mid-run?
  • latency and cost of evaluating intent

Success metric

Intent catches materially more harmful actions than argument-aware policy alone, at a false-deny rate an operator would accept.

Kill criterion

Drop it if either holds:

  • intent adds less than ~2 percentage points of caught-harmful over the argument-aware baseline; or
  • a prompt-injected intent flips a decision in more than a negligible fraction of adversarial cases.

Then argument-aware policy is the whole answer, and it is simpler.

Hard boundary

A model may produce an intent. A model must never be the authorization authority. Intent is one input to a deterministic decision, and the decision record must show it was one input among several. A PR that makes a model's judgement the final verdict will be rejected regardless of its numbers.

Candidate primitive

  • intent on the action context
  • intent propagated across a delegation hop
  • an intent/action consistency check expressed as a policy condition, never as a verdict

Deliverable

A written result with the numbers, and a recommendation. Code only as far as the experiment needs. If the kill criterion fires, the deliverable is the negative result and this issue closes as answered.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Priority 2 - NormalevaluationEvaluation harness and quality scoringresearchA falsifiable experiment with a kill criterion, not a featuresecuritySecurity hardeningsize/m

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions