test: add policy compiler round-trip tests (YAML -> Cedar -> evaluate) - #1595
Conversation
Adds 36 evaluation-level tests in lib/policy/src/compiler.rs covering all 10 built-in policy templates / all 8 PolicySpec fields: each template is compiled to Cedar, loaded into a real PolicyEngine, and evaluated against representative requests with an expected decision asserted, including forbid-overrides-permit and quarantine/require_approval annotation precedence cases. production-baseline additionally evaluates the same requests against an independently hand-written equivalent Cedar policy and asserts identical decisions — the literal "compiled YAML == equivalent Cedar" comparison from the issue. Also adds one test in src/src/routes/policy.rs proving a signed policy bundle, once uploaded/verified/hot-reloaded, evaluates identically to loading that same Cedar text standalone (no signing pipeline involved). Tampered-bundle rejection was already covered by an existing test. Closes #1328
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Warning Review limit reached
More reviews will be available in 35 minutes and 22 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive round-trip and integration tests for the Cedar policy compiler and engine, ensuring that compiled YAML templates and uploaded signed policy bundles evaluate correctly against a real PolicyEngine. The feedback suggests refactoring a test in compiler.rs to group parallel arrays into a single array of tuples, which improves safety and avoids potential out-of-bounds panics.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| let scenarios = [ | ||
| request("github", "read", false, "trusted_internal_signed", "production", false), // unknown tool case driven by is_tool_known below | ||
| request("github", "merge", true, "trusted_internal_signed", "production", false), | ||
| request("github", "merge", true, "untrusted_external", "production", false), | ||
| request("github", "read", false, "trusted_internal_signed", "production", false), | ||
| request("github", "read", false, "trusted_internal_signed", "staging", false), | ||
| request("github", "read", false, "trusted_internal_signed", "production", false), | ||
| ]; | ||
| let is_tool_known = [false, true, true, true, true, true]; | ||
| let is_mtls = [true, true, true, false, true, true]; | ||
|
|
||
| for i in 0..scenarios.len() { | ||
| let compiled_decision = | ||
| decide(&compiled_engine, &scenarios[i], is_tool_known[i], is_mtls[i]).await; | ||
| let hand_written_decision = decide( | ||
| &hand_written_engine, | ||
| &scenarios[i], | ||
| is_tool_known[i], | ||
| is_mtls[i], | ||
| ) | ||
| .await; | ||
| assert_eq!( | ||
| compiled_decision, hand_written_decision, | ||
| "scenario {i}: compiled YAML->Cedar and hand-written Cedar must agree" | ||
| ); | ||
| } |
There was a problem hiding this comment.
Using multiple parallel arrays (scenarios, is_tool_known, is_mtls) that must be kept in sync by index is error-prone and can easily lead to out-of-bounds panics if one array is updated without the others. Grouping the request and its associated parameters into a single tuple or struct array is much safer and more idiomatic.
let scenarios = [
(request("github", "read", false, "trusted_internal_signed", "production", false), false, true),
(request("github", "merge", true, "trusted_internal_signed", "production", false), true, true),
(request("github", "merge", true, "untrusted_external", "production", false), true, true),
(request("github", "read", false, "trusted_internal_signed", "production", false), true, false),
(request("github", "read", false, "trusted_internal_signed", "staging", false), true, true),
(request("github", "read", false, "trusted_internal_signed", "production", false), true, true),
];
for (i, (req, is_tool_known, is_mtls)) in scenarios.into_iter().enumerate() {
let compiled_decision =
decide(&compiled_engine, &req, is_tool_known, is_mtls).await;
let hand_written_decision = decide(
&hand_written_engine,
&req,
is_tool_known,
is_mtls,
)
.await;
assert_eq!(
compiled_decision,
hand_written_decision,
"scenario {i}: compiled YAML->Cedar and hand-written Cedar must agree"
);
}
1 participant
Summary
lib/policy/src/compiler.rs(mod round_trip) covering all 10 built-in policy templates / all 8PolicySpecfields: each template is compiled YAML->Cedar, loaded into a realPolicyEngine, and evaluated against representative requests with an expected decision asserted — including forbid-overrides-permit and quarantine/require_approval annotation precedence cases.production_baseline_compiled_matches_hand_written_equivalentadditionally evaluates the same 6 requests against an independently hand-written equivalent Cedar policy and asserts identical decisions to the compiled output — the literal "compiled YAML == equivalent Cedar" comparison from the issue's acceptance criteria.upload_policy_bundle_evaluation_matches_standalone_cedar_loadinsrc/src/routes/policy.rs: a signed policy bundle, once uploaded/verified/hot-reloaded, evaluates identically to loading that same Cedar text standalone (no signing pipeline involved) — closes the "signed bundle -> load -> evaluate -> same results" criterion. Tampered-bundle rejection was already covered by the existingupload_policy_bundle_rejects_tampered_bundletest.Closes #1328
Test plan
lib/policy/src/cedar.rs's actualauthorize()implementation for every new assertion before writing it.cargo test --workspace(cannot run locally — no Rust toolchain in this environment; relying on CI, per project convention).