I transform compliance findings from my existing audit tools (s3-audit, sg-audit, cloudtrail-audit, secret-scanner, evidence-logger) into OSCAL Assessment Results (SAR) JSON. That is the machine-readable evidence format FedRAMP 20x wants, and what federal assessors increasingly expect for FedRAMP High and CJIS v6.1 packages.
Built on IBM Compliance Trestle (orchestration / CLI) and oscal-pydantic (typed transform of audit-tool JSON into OSCAL models).
Status: v1.0 shipped 2026-06-15. SAR generation from the portfolio audit tools. POA&M (v1.1) and Component Definitions (v1.2) come later.
graph LR
A[Audit tools<br/>s3 / sg / cloudtrail<br/>secret-scanner / evidence-logger] --> B[Ingest + adapt<br/>native JSON]
B --> C[Transform<br/>oscal-pydantic models]
C --> D[Assemble + validate<br/>Trestle + NIST schema]
D --> E[OSCAL SAR JSON]
E --> F[FedRAMP 20x<br/>CA-7 continuous monitoring]
E --> G[CJIS v6.1<br/>AU-6 weekly review]
Five portfolio audit tools emit native JSON findings. The pipeline ingests and adapts that JSON, transforms it into typed OSCAL models (oscal-pydantic), then assembles and validates an Assessment Results (SAR) document with IBM Compliance Trestle against the published NIST OSCAL JSON Schema. The SAR feeds FedRAMP 20x continuous-monitoring consumers and CJIS v6.1 weekly review / retention. See ARCHITECTURE.md for stage detail.
Audit evidence is moving from Word/PDF to machine-readable formats. FedRAMP 20x makes OSCAL the canonical one. Once my audit tools emit OSCAL Assessment Results instead of plaintext logs, an assessor or a continuous-monitoring pipeline can consume them without manual transcription.
This repo is the transformation layer between those operational audit tools and the OSCAL ecosystem. Without it, every tool produces a different JSON shape that I have to hand-map into an SAR entry. With it:
audit tool runs → emits structured JSON → pipeline transforms → OSCAL SAR JSON → assessor / dashboard / Trestle assemble
Without this layer, every finding from five audit tools gets hand-mapped into an SAR entry: read the tool's JSON, decide which OSCAL observation fields it fills, transcribe, repeat per finding, per tool, per assessment cycle. Each transcription is a chance to drop a control ID or mistype a timestamp, and the resulting document is only as consistent as the person who assembled it that day.
With the pipeline, transcription is zero for an adapted tool. v1.0 ships the secret-scanner adapter; the other four land as each upstream adds structured JSON output, so each new tool is one adapter, not a new manual process. Findings become schema-validated OSCAL observations mechanically, every SAR passes the same three validation layers before it exists, and deterministic UUIDs make two SARs from different runs diffable. The program-level number is FedRAMP's own: machine-readable packages get a 30-day review SLA against 90 days for traditional ones. This pipeline produces the input format that 30-day track expects.
This pipeline is a meta-tool. It does not satisfy access controls directly. It satisfies the assessment, monitoring, and audit-record-generation controls that govern how compliance evidence is produced and preserved.
| NIST 800-53 Rev 5 | FedRAMP High | CJIS v6.1 | How This Pipeline Validates |
|---|---|---|---|
| CA-2 Control Assessments | Yes | - | Produces the OSCAL SAR artifact that documents each assessment cycle |
| CA-7 Continuous Monitoring | Yes | - | Produces the per-cycle SAR artifact a monitoring program consumes; scheduling the cycles lives outside this pipeline |
| AU-3 Content of Audit Records | Yes | - | Preserves timestamp, source tool, finding type, mapped control IDs in every SAR observation |
| AU-12 Audit Record Generation | Yes | - | Wraps audit-tool outputs into a generated record. Trestle-structural-validated and gated against the published NIST OSCAL JSON Schema at the emit boundary (and in CI) |
| CM-3 Configuration Change Control | Yes | - | Deterministic observation UUIDs (uuid5 over stable identity) and sorted ingest order make two SARs from different runs meaningfully diffable: the same input reproduces the same observation, finding, and subject UUIDs, so only assessment timestamps and the two timestamp-seeded document UUIDs differ between runs |
| CA-2, CA-7, AU-12 | Yes | 1-year retention, weekly review | SAR JSON is the artifact retained for the CJIS AU-6 weekly review |
An assessor reviewing a FedRAMP High or CJIS v6.1 authorization package can consume the SAR JSON directly. Each emitted SAR passes Trestle's structural model validation and the published NIST OSCAL JSON Schema gate before write. The canonical sample SAR is also checked in CI on every pull request and on pushes to main.
Each SAR observation maps one-to-one to an NIST 800-53A assessment objective. Example: an s3-audit finding of "BucketX failed encryption check" becomes an OSCAL observation with relevant-evidence pointing to the source tool, subjects referencing the bucket, and props carrying the mapped control IDs (sc-28, sc-28.1). The assessor's adequacy determination (satisfied / other-than-satisfied) is the OSCAL finding object.
Combined with evidence-logger for retention and aws-config-compliance-monitor for continuous detection, this completes the FedRAMP 20x evidence loop: detect → transform → retain → review.
FedRAMP 20x (Pilot launched March 2025, High pilot FY26 Q4) restructures the program around five pillars: compliance-as-code, machine-readable evidence, continuous monitoring, API-driven evidence, and automated scanning. This pipeline targets the machine-readable evidence pillar:
- OSCAL output, not Word/PDF: Every SAR is a JSON document. At runtime each emitted SAR passes Trestle's structural model validation (Layer 2) and the published NIST OSCAL JSON Schema gate (Layer 3) before write. CI also validates the canonical sample SAR on every pull request and on pushes to
main. A schema-nonconformant SAR fails the run (exit 1, no artifact written) or blocks merge in CI. The Layer-3 gate enforces JSON structure, required properties, enums, types, and an approximate token regex (not fulldate-time/uriformat checking). No manual transcription. No version drift between a spreadsheet and the system.- See ARCHITECTURE.md §11 (Validation Layers & Schema-Pinning Policy) for the three-layer model (oscal-pydantic typed import → Trestle structural → published-schema gate) and the
OSCAL_VERSIONschema-pinning policy.
- See ARCHITECTURE.md §11 (Validation Layers & Schema-Pinning Policy) for the three-layer model (oscal-pydantic typed import → Trestle structural → published-schema gate) and the
- Continuous evidence generation: Each pipeline run emits a timestamped SAR. A FedRAMP 20x reviewer comparing two SARs from different dates can read the delta directly. That is a KSI metric in flight.
- API-driven: The pipeline is a library + CLI. I can invoke it from CI/CD, from a scheduled job, or from an evidence orchestrator (for example, on every CloudTrail event that indicates an audit-tool re-run).
- 30-day vs 90-day review window: FedRAMP 20x machine-readable packages get a 30-day review SLA versus 90 days for traditional packages. The SAR output is the unit of input to that 30-day review.
CJIS Security Policy v6.1 (released June 25, 2026) is the current policy, aligned with NIST 800-53 Rev 5. v6.x has been the default audit baseline since April 1, 2026 (v5.9.5 sunset March 31, 2026); modernized Priority 2-4 controls are fully enforceable Oct 1, 2027 (timing varies by state CSA).
The material delta this pipeline supports is AU-6: agencies handling CJI must retain audit records for 1 year and conduct weekly review of those records. The SAR JSON is the artifact I retain for that 1-year window and the input to that weekly review. A reviewer can read it without going back to the raw CloudTrail / IAM policy / S3 audit output.
For public-safety SaaS (FedRAMP High + CJIS), the same SAR feeds both review tracks. Producing two separate evidence sets is unnecessary when both frameworks reference the same control catalog.
OSCAL (Open Security Controls Assessment Language) is a data format, not a framework. NIST defines seven OSCAL models that cover the compliance lifecycle:
| OSCAL Model | What It Represents | Produced By |
|---|---|---|
| Catalog | The control inventory itself (e.g., NIST 800-53 Rev 5) | NIST publishes; you consume |
| Profile | A selection / tailoring of a catalog (e.g., FedRAMP High baseline) | FedRAMP PMO publishes; you consume |
| Component Definition | What a specific tool, service, or component implements | You author per tool / per AWS service |
| System Security Plan (SSP) | Full system documentation | You author |
| Assessment Plan (SAP) | What the assessor will check, and how | Assessor or you (for self-assessment) |
| Assessment Results (SAR) | What was found during the assessment | This pipeline |
| Plan of Action and Milestones (POA&M) | Open findings and remediation plan | This pipeline (v1.1) |
Phase 1 produces SAR. Phase 2 adds POA&M derivation from FAIL findings. Phase 3 adds Component Definition generation from each portfolio audit tool's capability set.
See ARCHITECTURE.md for the full pipeline design, library rationale (Trestle + oscal-pydantic), and integration map for each upstream audit tool.
- Python 3.11+
oscal-pydantic: typed OSCAL modelscompliance-trestle: OSCAL workflow CLI + assemble/split- Source audit tools (any subset):
s3-audit,sg-audit,cloudtrail-audit,secret-scanner,evidence-logger
The pinned dependency set lives in requirements.txt and requirements.lock. See Development Setup for how to install them and how the two differ.
The package scaffold (oscal_pipeline/, tests/, examples/, requirements.txt, requirements.lock, pyproject.toml) is in place. To work on the pipeline locally:
# Clone and enter the repo
git clone https://github.com/0xBahalaNa/oscal-evidence-pipeline.git
cd oscal-evidence-pipeline
# Create an isolated virtual environment (Python 3.11+)
python3 -m venv .venv
source .venv/bin/activate
# Install exact pinned deps (--no-deps skips the resolver; avoids trestle/pydantic conflict)
pip install --no-deps -r requirements.lock
pip install -e . --no-deps
# Smoke-test that the package imports and exposes a version
python -c "import oscal_pipeline; print(oscal_pipeline.__version__)"
# Run the test suite
pytestThe complete pinned dependency tree lives in requirements.lock. It is the CM-3 artifact for this repo: the exact versions used to produce any given OSCAL SAR, recorded once and version-controlled. requirements.txt documents direct-dependency intent. The looser compatible-release pins in pyproject.toml define the contract for downstream installers. The three files separate "what the package needs" from "what we shipped against."
main's protection is codified in scripts/setup-branch-protection.sh and applied as a one-time ops step. Once applied, merging requires both CI checks to pass:
test: pytest + the 80% coverage gate + the OSCAL JSON-Schema validation (Layer 3).lint:mypy(strict) +ruff.
The script also sets merges to squash-only with the source branch auto-deleted on merge. Apply (or re-assert) the full protected state with:
bash scripts/setup-branch-protection.shThe script is idempotent. Re-running re-asserts the same configured state. That is what makes the test/validation suite a merge precondition (NIST 800-53 CM-3) rather than an advisory run: it configures enforce_admins: true, so once applied the gate is non-bypassable. A red check cannot be merged past, including by repository admins.
Phase 1 MVP. CLI surface subject to change before v1.0 tag.
oscal-pipeline run \
--input-dir ./audit-outputs/ \
--output ./evidence/ \
--profile fedramp-highThe pipeline reads each *.json file in --input-dir, identifies the source tool by schema fingerprint, transforms each finding into an OSCAL observation + finding, assembles the full SAR via Trestle, validates it against the published NIST OSCAL JSON Schema (Layer 3), and emits an assessment-results.json that has passed both Trestle structural validation (Layer 2) and the schema gate. See ARCHITECTURE.md §11.
A complete worked example lives in examples/:
- Input:
examples/sample-secret-scanner-input.json. Asecret-scannerrun that flagged four files. - Output:
examples/sample-assessment-results.json. The OSCAL SAR the pipeline produced from it. Trestle-structural-validated on generation and gated against the published NIST OSCAL JSON Schema in CI (the canonical sample the Layer-3 test validates).
Regenerate the output from the input at any time:
oscal-pipeline run --input-dir examples/ --output ./evidence/ --profile fedramp-highNo real credentials appear in the sample. The
secret-scannerschema records only the detection regex (pattern_matched), a file path, and a severity. Never the matched secret. So the evidence artifact cannot leak a credential by structure.
The scanner found a hardcoded AWS access key. The input finding:
{
"file_path": "src/config/deploy.tf",
"line_number": 12,
"finding_type": "AWS Access Key ID",
"pattern_matched": "AKIA[0-9A-Z]{16}",
"severity": "CRITICAL",
"control_ids": ["IA-5(7)", "SC-12", "SC-28"]
}The pipeline splits it into two OSCAL objects: an observation (the raw fact that was seen) and a finding (the assessor's conclusion). Control mappings live on the finding, because mapping a fact to a control is an assessment decision. The observation stays reusable raw evidence.
| OSCAL field | Comes from | What it means |
|---|---|---|
observation.description |
finding_type |
What the scanner detected |
observation.methods |
constant ["EXAMINE"] |
The NIST 800-53A assessment method a file scan maps to. (methods is a free-form OSCAL string; the pipeline populates it with the 800-53A examine / interview / test vocabulary. A scan inspects artifacts, so examine.) |
observation.collected |
scan_metadata.timestamp |
When the evidence was gathered. Provenance from the source tool, not pipeline run time |
observation.props[*] |
finding fields | AU-3 audit-record content: source tool, severity, file, line, detection pattern |
observation.subjects[0] |
file_path |
The assessed subject (type: software, title: the path) |
finding.target.target-id |
control_ids[0] → slug |
A machine-oriented reference to the 800-53A assessment objective the failure implicates (IA-5(7) → ia-5.7_obj) |
finding.props[control-id] |
all control_ids |
The catalog control IDs the finding maps to (raw form, e.g. IA-5(7)). The human/catalog-facing side of the same decision |
finding.target.status.state |
severity | not-satisfied for any FAIL/WARN. OSCAL offers only satisfied / not-satisfied, so a WARN collapses into not-satisfied by design |
results[0].reviewed-controls |
union of all findings' controls | The controls under assessment in this result, in slug form, sorted |
An INFO finding ("no secrets detected") produces an observation but no finding. There is nothing to conclude, and OSCAL does not require a finding for every observation.
Re-run the pipeline on the same input and every observation, finding, and subject UUID reproduces exactly. Each is a uuid5 hash of the object's stable identity (an observation or finding keys on file_path + line + detection pattern; a subject keys on file_path alone), not random. Only the assessment timestamps (start / end / last-modified) and the two timestamp-seeded document UUIDs change between runs. That stability is what makes two SARs from different dates diffable. Diffable SARs are the unit of input to a FedRAMP 20x continuous-monitoring review.
- POA&M generation from FAIL findings (v1.1)
- Component Definition generation per source audit tool (v1.2)
- AI evidence module: emit AI-specific evidence (model lineage, training data audit logs, bias testing results) as OSCAL observations; connects this pipeline to the AI portfolio layer (Project 10 AI Risk Assessment, Project 12 AI Controls Mappings) (v1.3)
- SSP skeleton generation from a Profile + Component Definition set (v2.0)
- KSI metric extraction from cross-run SAR diffs (v2.0)
- S3 archival of SAR JSON with Object Lock for CJIS AU-6 1-year retention
- Extend the published-schema CI test coverage beyond the canonical secret-scanner sample to a fixture per source audit tool. (Operator-runtime output is already schema-validated at the CLI emit boundary, and the canonical sample in CI, as of #42. What remains is broadening the CI fixture set.)
Control family mappings and AWS implementation details are documented in nist-800-53-rev-5-to-aws-mapping.
OSCAL specifications: pages.nist.gov/OSCAL
FedRAMP 20x program documentation: fedramp.gov
MIT