Skip to content

Data Platform: Skip system messages in LiteLLM guardrail evaluations - #18714

Draft
muhammad9970 wants to merge 5 commits into
mainfrom
ai_gw_guardrail_config
Draft

Data Platform: Skip system messages in LiteLLM guardrail evaluations#18714
muhammad9970 wants to merge 5 commits into
mainfrom
ai_gw_guardrail_config

Conversation

@muhammad9970

@muhammad9970 muhammad9970 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • set skip_system_message_in_guardrail: true globally for the public and admin LiteLLM proxies
  • re-enable the Bedrock prompt-attack guardrail by setting default_on: true
  • exclude role: system content from compatible guardrail evaluations while continuing to send the complete message list to the model
  • manage LiteLLM unified access-group team assignments from the existing team configuration
  • grant the Justice Data Platform team access to the generally-available-models group

Relates to the work being done here

Signed-off-by: Muhammad Ahmad <muhammad.ahmad@justice.gov.uk>
@muhammad9970 muhammad9970 self-assigned this Sep 1, 2026
@github-actions github-actions Bot added the environments-repository Used to exclude PRs from this repo in our Slack PR update label Sep 1, 2026
@muhammad9970
muhammad9970 had a problem deploying to data-platform-test-ai-gateway September 1, 2026 10:42 — with GitHub Actions Failure
@muhammad9970
muhammad9970 had a problem deploying to data-platform-development-ai-gateway September 1, 2026 10:42 — with GitHub Actions Failure
@muhammad9970
muhammad9970 had a problem deploying to data-platform-development-ai-gateway September 1, 2026 13:02 — with GitHub Actions Failure
@muhammad9970
muhammad9970 had a problem deploying to data-platform-test-ai-gateway September 1, 2026 13:02 — with GitHub Actions Failure
muhammad9970 and others added 2 commits September 1, 2026 14:03
Signed-off-by: Muhammad Ahmad <muhammad.ahmad@justice.gov.uk>
@muhammad9970
muhammad9970 had a problem deploying to data-platform-test-ai-gateway September 1, 2026 13:06 — with GitHub Actions Failure
@muhammad9970
muhammad9970 had a problem deploying to data-platform-development-ai-gateway September 1, 2026 13:07 — with GitHub Actions Failure
@muhammad9970
muhammad9970 had a problem deploying to data-platform-development-ai-gateway September 1, 2026 13:10 — with GitHub Actions Failure
@muhammad9970
muhammad9970 had a problem deploying to data-platform-test-ai-gateway September 1, 2026 13:10 — with GitHub Actions Failure
@muhammad9970
muhammad9970 had a problem deploying to data-platform-test-ai-gateway September 1, 2026 13:51 — with GitHub Actions Failure
Signed-off-by: Muhammad Ahmad <muhammad.ahmad@justice.gov.uk>
@muhammad9970
muhammad9970 had a problem deploying to data-platform-test-ai-gateway September 1, 2026 14:12 — with GitHub Actions Failure
@muhammad9970
muhammad9970 had a problem deploying to data-platform-development-ai-gateway September 1, 2026 14:12 — with GitHub Actions Failure
@muhammad9970
muhammad9970 deployed to data-platform-development-ai-gateway September 1, 2026 14:44 — with GitHub Actions Active
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Checkov Scan Success

Show Output
*****************************

Checkov will check the following folders:
terraform/environments/data-platform/ai-gateway

*****************************

Running Checkov in terraform/environments/data-platform/ai-gateway
terraform scan results:

Passed checks: 91, Failed checks: 0, Skipped checks: 0, Parsing errors: 1

secrets scan results:

Passed checks: 0, Failed checks: 0, Skipped checks: 1


checkov_exitcode=0

TFLint Scan Success

Show Output
*****************************

Using default config
Installing "terraform" plugin...
The plugin was signed using a legacy PGP signing key. Please update the plugin to the latest version
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.15.0)
tflint checking:
terraform/environments/data-platform/ai-gateway

*****************************

Running tflint in terraform/environments/data-platform/ai-gateway
tflint_exitcode=0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

environments-repository Used to exclude PRs from this repo in our Slack PR update

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant