You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Defines apcore's three conformance levels (Level 0 Core, Level 1 Standard, Level 2 Full) with per-level MUST/SHOULD/MAY components, test suite requirements, and conformance declaration rules.
apcore — Conformance Definitions
This document defines conformance levels for apcore framework implementations, test suite requirements, and conformance declaration specifications.
1. Overview
1.1 Purpose
As a cross-language AI-Perceivable module standard, apcore needs to ensure behavioral consistency among SDK implementations in various languages. This conformance specification defines three progressive conformance levels, with each level clearly listing MUST, SHOULD, and may implement components, as well as corresponding test requirements.
Implementers can choose their target conformance level based on their needs and verify conformance through the corresponding test suite.
Level 0 defines the minimal viable implementation of apcore. SDKs reaching this level can complete module definition, registration, discovery, and basic execution, but do not include advanced features such as permission control, middleware, and observability.
2.2 Must Implement (MUST)
Component
Responsibility
Reference Section
Module interface
Module base class/interface, includes execute(), input_schema, output_schema, description
PROTOCOL_SPEC §5.6
Schema validation
Input/output validation based on JSON Schema Draft 2020-12, supports type, properties, required, enum, $ref (local references)
Level 1 adds permission control, middleware, basic observability, and structured logging on top of Level 0. SDKs reaching this level meet the needs of most production environments.
Configurable symlink following and cycle detection
PROTOCOL_SPEC §3.4
3.5 Level 1 Required Test Categories
All tests from Level 0, plus:
ACL tests (15+ cases)
Middleware tests (10+ cases)
Observability tests (10+ cases)
Context propagation tests (5+ cases)
Error guidance tests (3+ cases): Verify framework errors include ai_guidance field with actionable recovery hints
4. Level 2 — Full Conformance
4.1 Overview
Level 2 adds all extension points, async modules, hot loading, and advanced observability on top of Level 1. Conformance at this level confirms feature coverage; deployment suitability still depends on workload testing, operational controls, and the implementation's support policy.
4.2 Must Implement (MUST)
All MUST components from Level 1, plus:
Component
Responsibility
Reference Section
Extension point framework
All five extension points (discoverer, middleware, acl, span_exporter, module_validator) via ExtensionManager with register(), get(), get_all(), unregister(), apply(), list_points(). Note: these names map to actual runtime extension needs rather than the original theoretical design names (SchemaLoader, ModuleLoader, IDConverter, ACLChecker, Executor).
PROTOCOL_SPEC §11.3, §11.6
Extension point chain
first_success, all, fallback strategies
PROTOCOL_SPEC §11.3
Extension loading order
load_extensions() algorithm
PROTOCOL_SPEC §11.7
Async modules
submit(), get_status(), cancel(), list_tasks() via AsyncTaskManager
PROTOCOL_SPEC §5.8
Async state machine
State transition rules (PENDING → RUNNING → COMPLETED/FAILED/CANCELLED) via TaskStatus enum
PROTOCOL_SPEC §5.8
Middleware state machine
Complete state transitions (init → before → execute → after → done, with error branches)
The following features are specified in PROTOCOL_SPEC but not yet fully implemented in current SDK releases (apcore-python, apcore-typescript). Implementers SHOULD document these deviations in their conformance declarations.
Feature
Spec Reference
Current Status
Config class (YAML loading, env override, schema validation)
PROTOCOL_SPEC §9.1, §9.2, §9.3
Stub implementation only. YAML loading, environment variable override, and validate_config() schema validation are not implemented.
Not on Registry. A standalone SchemaExporter class is available for schema export.
Error codes GENERAL_NOT_IMPLEMENTED and DEPENDENCY_NOT_FOUND
PROTOCOL_SPEC §8.2, §8.7
Implemented in both SDKs as FeatureNotImplementedError and DependencyNotFoundError.
Version negotiation
PROTOCOL_SPEC §13.3
negotiate_version() algorithm not yet implemented.
Schema migration
PROTOCOL_SPEC §13.4
migrate_schema() algorithm not yet implemented.
Module isolation
PROTOCOL_SPEC §5.5
Process-level or container-level isolation not yet implemented.
Multi-version coexistence
PROTOCOL_SPEC §5.4
Multiple versions of the same module running concurrently not yet implemented.
AsyncTaskManager.submit() / cancel() sync vs async
PROTOCOL_SPEC §5.8
Python AsyncTaskManager.submit() and cancel() are async methods; TypeScript equivalents are synchronous.
Implementations declaring conformance MUST list any of these deviations that apply in their known_deviations section.
8. Conformance Test Fixtures
The repository ships 72 cross-language fixture files under conformance/fixtures/ covering 843 test cases. These two numbers are checked against the fixtures themselves by conformance-integrity, together with §8.1's per-fixture counts and its Total row — a count nobody verifies reads as coverage in every review and every inventory built from it. Each fixture is a JSON document of shape { "description": "...", "test_cases": [...] } consumed by all three SDK test runners (apcore-python, apcore-typescript, apcore-rust). A SDK declaring a conformance level MUST pass every fixture whose tested feature is required at that level (see §2 Level 0, §3 Level 1, §4 Level 2 for the per-feature breakdown).
Authorization and approval requirement are two orthogonal results; the built-in arguments condition scopes a rule to this call; an unevaluable rule's requirement is pending, not discarded (spec §6.1.1/§6.1.6/§6.1.7/§6.1.8/§6.9)
A rule's effect value is a closed set at every entry point — file loading, direct construction and runtime insertion; default_effect on the same terms (spec §6.1.5)
A callers / targets pattern array's shape is a closed set at every entry point, plus a validator-only tier for well-formed arrays that match nothing (spec §6.2.1)
ApprovalRequest carries caller_id (read straight off Context.caller_id — null on a top-level call, never the @external ACL sentinel) and action (= module_id), populated by the approval gate at Executor Step 4.5 (spec §7.3.1, decision D-03)
Binding-directory resolution: a loader invoked without an explicit directory resolves bindings.dir / bindings.pattern under §9.2 precedence, an explicit argument wins, and no scan happens at client initialisation (spec §5.12.6)
The project root that path-typed values will resolve against from v2.0 — config-file directory for §9.14 tiers 1-5, CWD for the user-level tiers 6-7 and when no file is found; one case per tier, plus the narrow deprecation-warning condition (spec §9.2.2)
```python title="apcore-python — pytest example"
import json
import pathlib
FIXTURES = pathlib.Path("conformance/fixtures")
def load_fixture(name: str) -> dict:
return json.loads((FIXTURES / f"{name}.json").read_text())
def test_acl_evaluation():
fixture = load_fixture("acl_evaluation")
for case in fixture["test_cases"]:
# ... evaluate `case` against your ACL implementation,
# asserting the case's `expected` outcome
...
```
=== "TypeScript"
```typescript title="apcore-js — vitest example"
import * as fs from 'node:fs';
import * as path from 'node:path';
import { test } from 'vitest';
const FIXTURES = 'conformance/fixtures';
interface Fixture {
description: string;
test_cases: Array<Record<string, unknown>>;
}
function loadFixture(name: string): Fixture {
return JSON.parse(
fs.readFileSync(path.join(FIXTURES, `${name}.json`), 'utf-8'),
);
}
test('acl_evaluation', () => {
const fixture = loadFixture('acl_evaluation');
for (const c of fixture.test_cases) {
// ... evaluate `c` against your ACL implementation,
// asserting the case's `expected` outcome
}
});
```
=== "Rust"
```rust title="apcore-rust — cargo test example"
use std::fs;
use std::path::PathBuf;
use serde_json::Value;
fn load_fixture(name: &str) -> Value {
let path: PathBuf = ["conformance", "fixtures", &format!("{}.json", name)]
.iter()
.collect();
serde_json::from_str(&fs::read_to_string(path).unwrap()).unwrap()
}
#[test]
fn acl_evaluation() {
let fixture = load_fixture("acl_evaluation");
for case in fixture["test_cases"].as_array().unwrap() {
// ... evaluate `case` against your ACL implementation,
// asserting the case's `expected` outcome
let _ = case;
}
}
```
8.2.1 Locating the fixtures (normative)
The examples above hardcode a relative path for brevity. No SDK can: the fixtures live in the spec repo, the SDK lives in its own repo, and CI checks the spec repo out somewhere the SDK cannot guess. Every SDK therefore resolves the directory at run time, and because all three do it, the resolution order is a cross-SDK contract rather than three private conventions.
An SDK's conformance runner MUST resolve conformance/fixtures/ in this order, taking the first that exists:
#
Source
Names
Meaning
1
Environment
CONFORMANCE_FIXTURES
A conformance/fixturesdirectory, used directly
2
Environment
CONFORMANCE_SPEC_REPO
The spec repo root; conformance/fixtures is appended
3
Filesystem
—
../apcore/conformance/fixtures beside the SDK repo
CONFORMANCE_FIXTURESMUST take precedence over CONFORMANCE_SPEC_REPO. It names a directory of fixture files with no repository around it, which is what makes it useful: a driver can be run against a synthesised fixture set — an older shape, a single edited case, a mutation — without producing a whole spec repo to hold it. An SDK that supports only the repo-root form forces that verification to fabricate a repo, and in practice means it is not done.
A variable that is set but does not resolve MUST fail loudly, naming the variable that was actually set. Falling through to the next source would silently test against different fixtures than the operator named, which is worse than not running.
APCORE_FIXTURES and APCORE_SPEC_REPO are transitional fallbacks for 1 and 2 (apcore#86 / apcore#88). They are not the canonical names and MUST NOT be documented to users: PROTOCOL_SPEC §9.2 makes every APCORE_* variable a configuration override, so APCORE_SPEC_REPO=/path injected spec.repo into the config document §9.1's required-field check runs against. A test locator is infrastructure, not configuration.
Resolution for other spec-repo subdirectories — schemas/, most importantly — MUST NOT consult CONFORMANCE_FIXTURES. It names one directory, not a repo, so there is nothing to append to.
!!! warning "Drivers land before fixtures, so a driver MUST tolerate the older fixture"
A new fixture turns CI red in all three SDK repositories until every driver exists, so the landing order is **drivers first, fixture last** (§8.3). A driver therefore runs against a fixture that predates the keys it reads, and **MUST** degrade rather than fail: an absent expectation key means "this case does not pin that property", never "compare against nothing".
This cannot be verified from a working tree, which already holds the newer fixture — it is exactly what `CONFORMANCE_FIXTURES` is for. Point it at a copy with the new keys removed and the suite **MUST** still pass. Measured: a strict `!== null` check on a key absent from the older fixture failed 19 of 20 cases in apcore-typescript, green locally and red in CI.
A fixture that pins a **shared constant** rather than new behaviour inverts the order: it **MUST** land *before* the SDKs. `acl_rule_key_closure.json` carries the closed ACL rule-key set, so an SDK that adds a key before the fixture lists it goes red against the set it is meant to agree with.
8.3 Adding a New Fixture
Create conformance/fixtures/<name>.json with { "description": "...", "test_cases": [...] }.
Each case MUST carry a stable id, the input fields, and the expected outcome ({ "expected": ... } or { "expected_error": "<CODE>" }).
Use canonical terminology: caller_id / target_id (never bare caller / target).
Add a row to the table in §8.1 above; bump the total.
Reference the fixture from the relevant spec section so the bidirectional traceability is maintained.