Skip to content

FEAT: Add supports_prefill capability and PrefillAttack strategy - #3283

Open
Dev3 ^~^ (shashank03-dev) wants to merge 5 commits into
microsoft:mainfrom
shashank03-dev:feat/prefill-attack
Open

Dev3 ^~^ (shashank03-dev) wants to merge 5 commits into
microsoft:mainfrom
shashank03-dev:feat/prefill-attack

Conversation

@shashank03-dev

Copy link
Copy Markdown
Contributor

Fixes #3270

Adds a supports_prefill target capability and a PrefillAttack that ends the conversation on an assistant turn carrying prefill_text, so the target continues it. The attack requires the capability natively, so a target that would drop the assistant turn raises at construction instead of looking like it resisted.

  • HuggingFaceChatTarget sets the flag and now passes continue_final_message when the last message is from the assistant.
  • OpenAICompletionTarget does not set it, even though the issue suggested completion-style targets. It forwards only the last message's text, so the user turn carrying the objective would be lost.
  • The prefill is stored with the simulated assistant role, and request converters apply only to the objective.

Tests: unit tests for the attack, the capability round-trip and per-target defaults, and the HF chat template switch; the doc example is run against Qwen2.5-0.5B-Instruct.

@ManoharPaturi

Copy link
Copy Markdown
Contributor

This is a great implementation of the prefill attack. Requiring supports_prefill natively in TargetRequirements is the right design decision because attempting to adapt or fall back on targets that drop trailing assistant messages would disguise a target limitation as an attack refusal. Scoping request converters strictly to the prepended user objective preserves the exact prompt tokens intended for continuation, and tagging the prefill as a simulated assistant role keeps memory history truthful. The HuggingFaceChatTarget chat template continuation tests cover the mechanics nicely.

@shashank03-dev

Copy link
Copy Markdown
Contributor Author

Manohar Paturi (@ManoharPaturi) Thank you :)

Comment thread pyrit/executor/attack/single_turn/prefill.py
Comment thread pyrit/executor/attack/single_turn/prefill.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FEAT: Add supports_prefill capability and PrefillAttack strategy

3 participants