The Report Certification feature answers a narrow question: is there current, reproducible evidence that a governed data product moves and represents data as its approved contract requires?
It evaluates every required handoff from source to consumer. It does not certify an organization, a source application, an employee, or this repository. It also does not create or approve business definitions. Those remain owned by the applicable subject-matter experts (SMEs) and policy authorities.
The bundled data and decisions are synthetic. A production decision is valid only when an authorized host supplies current evidence under an identified policy version.
The portable contracts separate facts, evidence, and decisions:
| Record | Purpose |
|---|---|
| Product | Identifies the consumer asset, owner, source, criticality, required gates, and metric-contract binding. |
| Gate | Represents one required pipeline edge or the consumer gate. |
| Check result | Records a policy-defined test, status, tolerance, observed value, owner, and evidence references. |
| Artifact | References immutable evidence by URI, content hash, producer, and observation time. |
| Metric contract | Versions the approved grain, measures, and calculation contract. |
| SME approval | Binds an authorized approval to one exact metric-contract version and validity window. |
| Exception | Records a policy-approved, owned, time-bounded deviation from a specific failed check. |
| Run | Pins policy, agent, code, pipeline-manifest, semantic-model, and time versions. |
| Decision | Contains the gate decisions, findings, achieved tier, verdict, validity, and reproducible evidence bundle. |
| Recertification trigger | Records a change or operational event that invalidates or reopens a prior decision. |
Raw evidence must be stored separately from the verdict. A verdict is a derived record and must reference immutable artifact hashes and the exact versions used to derive it.
The default policy evaluates four pipeline gates; the final mart-to-consumer handoff is also the consumer-semantic gate:
- source to staging;
- staging to intermediate;
- intermediate to mart; and
- mart to consumer, including consumer-semantic behavior.
Each gate is independently addressable and owned. Lineage must fail closed when a required edge is absent, ambiguous, or cannot be tied to the declared product. An excellent downstream result cannot compensate for an unproven upstream edge.
Confirm the product's purpose, accountable owner, steward, criticality, source, consumer, row grain, stable identity, and complete lineage.
Verify schemas, required fields, compatible types, nullability, accepted values, keys, uniqueness, referential integrity, and declared grain.
Verify watermarks, row and distinct-key movement, duplicates, omissions, late arrivals, deletion/tombstone behavior, and change-ordering semantics.
Verify join cardinality, explained filter loss, dimension coverage, aggregation reconciliation, and rounding or tolerance behavior across representative time windows. A row-count match alone is insufficient when joins can inflate a measure.
Verify freshness, successful-run rate, runtime distribution, retries, idempotency, backfill behavior, partial-failure recovery, and accountable owner routing.
Verify mart fields against semantic fields, measures against the approved metric contract, labels and units, refresh state, relationships, and applicable row- or column-level access rules.
Verify ownership, classification, least-privilege access, retention, documentation, exception governance, and the current SME approval.
Verify active monitoring, change detection, recertification triggers, SLO alerts, accountable routing, and evidence that the incident path has been exercised.
Exact comparison is preferred. When exact equality is inappropriate, the tolerance must be defined by the applicable policy or approved contract. An agent or runtime evaluator must not invent or loosen it.
The achieved tier is the minimum tier supported by every required gate and the consumer gate. It is never an average and cannot be raised by unrelated passing checks.
T0 is an implementation sentinel, not a certification award. It means the minimum registration evidence is missing or the evaluator cannot establish a required gate.
Purpose, owner, grain, source, consumer, stable identity, and complete lineage are known.
T1 plus required structural, movement, transformation-fidelity, and operational checks passing for every required gate.
T2 plus all of the following:
- the active metric contract is bound to the product by exact ID and version;
- a current SME approval references that exact contract ID and version;
- the approval was issued by an allowed human authority and has not been revoked or expired;
- consumer-semantic and governance controls pass; and
- no failed required check is covered only by an unapproved, invalid, or expired exception.
The agent cannot act as the SME or policy authority.
The approval artifact binds the approver, contract hash, approval time, and validity deadline. Extending the deadline requires a new signed artifact; an adapter cannot make an expired approval current by editing metadata.
T3 plus active monitoring, relevant change-triggered recertification, SLO alerting, owner routing, and proven incident response. A declaration that monitoring exists is not enough; the run must reference evidence of its configuration and operation. Run history, monitor/test results, routing policy, and an exercised incident record are separate required evidence duties; one artifact kind cannot stand in for all four.
Tier and verdict answer different questions. The tier states the assurance level supported by the gates. The verdict describes the current evidence or lifecycle outcome.
| Verdict | Meaning |
|---|---|
PASS |
All requirements for the reported tier are currently proven. |
CONDITIONAL |
A specific failure has an active, policy-approved exception with owner, reason, scope, and expiry. |
BLOCKED |
A required control failed without an acceptable active exception, or policy explicitly blocks the product. |
EXPIRED |
The run, approval, or governing exception is outside its validity window. |
NOT_PROVEN |
Required evidence is missing, unreadable, ambiguous, unbound, or otherwise insufficient. |
NOT_PROVEN must never be converted to PASS. A lifecycle label such as
CONDITIONAL does not silently grant a higher tier than the evidence supports.
The policy evaluator is pure, deterministic code. Given the same normalized input and evaluation time, it must produce the same decision and evidence bundle.
- Validate identifiers, timestamps, versions, references, and the required gate set against a trusted host-registry topology hash.
- Resolve every check to its gate and every evidence reference to an artifact.
Missing references become
NOT_PROVENfindings. - Evaluate required checks by tier using policy-owned tolerances.
- Validate exceptions against the exact check, policy version, approving authority, owner, reason, and expiry.
- Validate the product's metric-contract binding and the exact, current SME approval before allowing T3.
- Validate continuous-assurance evidence before allowing T4.
- Calculate each gate's achieved tier and verdict.
- Set the product tier to the minimum achieved tier across all required gates.
- Derive the product verdict from blocking, not-proven, conditional, and expiry findings without concealing any lower-tier result.
- Emit a reproducible evidence bundle containing the run and artifact hashes, policy and agent versions, code revision, pipeline-manifest hash, semantic-model version, metric-contract binding, and approval reference.
Presentation scores may help a reviewer navigate evidence. They are not inputs to the minimum-gate result unless a named policy rule explicitly defines them.
An exception is a governed record, not a UI dismissal. A conditional exception requires:
- the exact failed check;
- an accountable owner;
- a reason and bounded scope;
- approval by an allowed policy authority;
- the governing policy version;
- approval and expiry timestamps; and
- a remediation or review path.
An agent cannot approve an exception. A pending, revoked, mismatched, or expired exception provides no relief. At its deadline, the host must automatically re-evaluate the product and block or downgrade it as policy requires. The approval artifact binds the exception's reason, scope, remediation plan, policy version, approval time, and expiry. Changing any of those fields requires a new governed artifact.
Every decision should preserve:
- run ID and timestamps;
- product and gate IDs;
- policy ID/version and evaluator/agent ID/version;
- code revision, transformation-manifest hash, and semantic-model version;
- check name, parameters, query or test hash, sample window, observed result, tolerance, status, reason code, and owner;
- artifact URI, content hash, producer, and observation time;
- metric-contract ID/version/hash and SME approval ID;
- exceptions and their validity; and
- achieved gate tiers, final tier, verdict, findings, downstream impact, and recertification triggers.
The bundle should be sufficient for an authorized reviewer to reproduce the decision without trusting an LLM narrative. Raw controlled data does not need to be copied into the bundle when a governed, immutable reference is sufficient.
Before a run begins, the host should resolve:
- the consumer asset ID, location, workspace, accountable owner, and steward;
- its semantic model/dataset and declared mart dependencies;
- complete catalog lineage from the primary source through every required gate;
- transformation manifests, catalogs, tests, run results, and freshness data;
- authorized read-only warehouse/query access and relevant job history;
- orchestrator run, retry, late-data, partial-failure, and backfill history;
- extraction watermarks or change-data-capture metadata;
- the metric contract's grain, dimensions, measures, filters, tolerances, and exact version;
- a signed or otherwise verifiable SME approval for that exact version;
- classification, access policy, and row/column security requirements; and
- the active certification policy and exception register.
If a required input cannot be resolved unambiguously, record it as
NOT_PROVEN and route the gap to an owner. Do not guess an identity, lineage
path, approval, or policy value.
Resolve the consumer back through every required upstream asset. Confirm owner, steward, purpose, grain, domain, criticality, and an unambiguous lineage path.
Pin schemas, keys, grain, partition/watermark behavior, freshness and completeness SLOs, join/loss tolerances, null and referential-integrity rules, metric-contract version, and SME approval.
Execute the required check families independently for each pipeline edge and
the consumer gate. Read-only queries and metadata inspection are the default.
Unavailable evidence is recorded as NOT_PROVEN.
Apply the deterministic minimum-gate evaluator. The agent may orchestrate the collection and explain findings, but it does not decide outside the policy code.
Publish the tier and verdict, edge-by-edge chain, passed/failed/not-proven controls, downstream impact, remediation owners, exact contract and approval, validity window, and recertification triggers.
Reopen certification whenever a relevant revision or operational signal may invalidate the stored evidence.
Relevant changes include:
- pipeline code, configuration, tests, packages, or materialization;
- source/target schema, lineage, grain, join, or filter behavior;
- semantic measures, relationships, labels, units, or security;
- metric-contract or SME-approval version/status;
- owner, exception, approval, or policy validity;
- SLO breach, anomaly, failed backfill, or unresolved incident; and
- any source revision explicitly named by the applicable policy.
A trigger creates or schedules a new run; it does not edit the prior evidence bundle. Historical decisions remain immutable and retain their original validity context.
The optional certification agent is a read-only orchestrator and explainer. It may:
- inspect authorized metadata;
- invoke approved read-only checks;
- preserve evidence references;
- identify ambiguity and downstream impact;
- draft remediation guidance; and
- route findings to accountable owners.
It must not:
- mutate production data, schemas, code, dashboards, contracts, or policy;
- grant or infer SME approval;
- approve business logic or its own exception;
- waive a required gate;
- choose a more permissive tolerance;
- report unavailable evidence as passing; or
- conceal a weaker upstream result behind a summary score.
Recommendations follow the host's normal review and deployment controls.
The agent's narrative must remain subordinate to the deterministic decision. For each completed run it should present:
- the run ID, policy/evaluator versions, achieved tier, verdict, and validity;
- an edge-by-edge evidence chain;
- passed, failed, conditional, expired, and not-proven controls;
- evidence references and hashes without exposing unauthorized raw values;
- downstream impact and affected consumer assets;
- remediation recommendations and accountable owners;
- the exact metric-contract and SME-approval versions; and
- the event or date that will trigger recertification.
An appropriate reference instruction is:
Determine whether the named data product is trustworthy by collecting reproducible, authorized evidence for every required source-to-consumer handoff. Operate read-only. Treat version-specific SME approval and policy-owned tolerances as inputs; never invent, change, or approve them. Preserve evidence and invoke the deterministic minimum-gate evaluator. Missing evidence is NOT_PROVEN. Return the evaluator's verdict and tier, edge-by-edge findings, downstream impact, remediation owners, validity, and recertification triggers. Never waive a gate, mutate production, or approve your own exception.
A host may obtain evidence from a catalog/lineage service, transformation artifacts, a warehouse, an orchestrator, a semantic/reporting platform, an approval store, an issue/exception system, and immutable object or evidence storage. Named commercial or open-source products are implementation examples only; this repository configures none of them.
Adapters must remain server-side, use least-privilege read access, and normalize vendor payloads into the exported serializable contracts. Credentials, raw controlled records, privileged endpoints, and unredacted identities must not be sent to the browser.
A production implementation should prove at least these cases:
- A healthy mart cannot pass when source-to-staging loses required records.
- Passing data tests cannot produce T3 without the exact current SME approval.
- A many-to-many join that preserves row count but inflates a measure fails.
- Unavailable evidence produces
NOT_PROVEN, neverPASS. - A consumer measure or security change reopens certification.
- A historical verdict can be reproduced from its stored evidence bundle.
- The weakest required gate determines the overall tier.
- The agent cannot mutate production or approve an exception.
- Each failure identifies its owner and downstream impact.
- An expired conditional exception automatically blocks or downgrades the product under the active policy.
- T4 is denied without evidence of monitoring, alerts, routing, and incident response.
Before publishing fixtures or evidence, remove real tenant/workspace/project IDs, internal endpoints, artifact URIs, query text, schemas, sample records, employee or SME identities, source revision hashes, metric definitions, and organization-specific classification or tolerance details. Synthetic fixtures should be unmistakably labeled and should not imitate active internal names.
An endpoint that starts a certification run is an operational capability. It must enforce server-side authentication and authorization, CSRF protection, idempotency, rate limits, queueing, audit logging, and least-privilege connector credentials. The bundled demonstration does not provide such an endpoint.