Skip to content

CR-014.md

Latest commit

 

History

History
249 lines (187 loc) · 16.3 KB

CR-014.md

File metadata and controls

249 lines (187 loc) · 16.3 KB
Raw

CR-014 — Assessment Metamodel v1 + Maturity Scoring v2

Field Value
CR CR-014
Title Assessment Metamodel v1 (sub-tree) + Maturity Scoring v2 (renames, non-linear bands, effort multipliers)
Status Proposed
Date 2026-08-20
Author Coder (for eaojnr)
Version additive (no version bump on the canonical VERSION; assessment sub-tree is a v1.0.0 of its own scope)
Depends on CR-005 (Assessment & Measurement), CR-008 (Semantic Architecture & Conformance Specification)
Scope Adds a new assessment-models/ sub-tree to the canonical repository, containing the assessment sub-metamodel (PlantUML, 12 JSON Schemas, 4 vocabularies, 5 examples, 3 governance docs) plus a maturity-scoring-v2 layer that renames the CMMI-era bands and re-scales them non-linearly with explicit effort multipliers
Out of scope Pilot migration of any specific assessment instrument; consumer-side updates to dea-cli / dea-web-viewer; capability / scenario / benchmark catalog creation; cross-boundary resolution with archived Assessment-Models repos (those are historical)

1. Context

Two related pieces of work converge in this CR.

1.1 The assessment sub-metamodel

The OpenDEA Assessment Metamodel defines the conceptual model for the assessment domain: AssessmentModel, AssessmentInstrument, AssessmentExecution, AssessmentResult, Capability, Scenario, Measure, Evidence, ScoringModel, BenchmarkModel (and MaturityModel as an optional interpretation layer).

Until now, the assessment domain has been served from:

  • technehub-labs/dea-metamodel/metamodel/profiles/assessment/ — assessment profile (constraint definitions for assessment-related entities inside the core metamodel)
  • technehub-labs/dea-metamodel/pydantic/assessment*.py — generated Pydantic models for the same
  • technehub-labs/dea-metamodel/models/scenarios/ — concrete scenario YAMLs

The conceptual model itself — schemas for AssessmentModel / AssessmentResult / Capability / Scenario / Measure / Evidence / ScoringModel / BenchmarkModel, and the governance policy for versioning, compatibility, and lifecycle — has not previously been defined as a coherent unit.

A supplementary CR (CR-AM-01-supplement-metamodel-v1.md, originally filed under Assessment-Models/dea-metamodel/change-requests/, byte-identical to the attachment md5 c0f086be67791444995237ceb0c20765) provides the PlantUML, JSON Schemas, vocabulary, examples, and governance policy. That material is being re-landed here as part of this CR.

1.2 The maturity-scoring proposal

A second proposal (docs/maturity-scoring-v2-proposal, originally filed as PR #1 on Assessment-Models/dea-catalog-maturity-models, an archived repository) addresses two structural criticisms of the current maturity scoring scheme:

  1. Archaic naming. The CMMI-derived names (Ad Hoc / Defined / Managed / Quantitatively Managed / Optimising) describe 1990s software-process culture. The proposal replaces them with Emergent / Structured / Systematic / Adaptive / Self-Optimising.
  2. Linear presentation hides diminishing returns. Capability maturity exhibits two crossing curves:
    • Effort is superlinear (roughly exponential): each level costs disproportionately more than the previous.
    • Value is sublinear (roughly logarithmic): the largest outcome gains come early; L4→L5 gains are real but marginal.

The narrowing top bands in the current model already encode this implicitly. The proposal makes it explicit and computable via:

  • Non-linear bands (20 / 25 / 25 / 18 / 12 points) instead of the current linear (25 / 25 / 25 / 15 / 10).
  • Per-level effort_multiplier (1.0× / 1.5× / 2.5× / 4.0× / 6.0×) normalising the relative cost of one point within each band.

The multiplier enables a second computed metric — effort-adjusted value — that exposes the diminishing-returns curve in numbers rather than prose.

1.3 The repos that are no longer live

The Assessment-Models GitHub organisation is now archived. The following repos are read-only historical references:

  • Assessment-Models/dea-catalog-assessment-tools — landed CR-AM-01 reference (merged PR #1)
  • Assessment-Models/dea-catalog-maturity-models — landed CR-AM-01 xref (merged PR #2) + open PR #1 (this proposal)
  • Assessment-Models/dea-assessment-{modernization,technology,operations,services-delivery} — frozen at v1.0.0-alpha
  • Assessment-Models/dea-metamodel — deleted (the wrong-location repo is gone; its content is preserved verbatim in this CR's assessment-models/ sub-tree)

The work in those repos is not being migrated — it remains as historical reference at the moment of archive. The new canonical home for the assessment sub-metamodel is this repository.

2. Proposed change

2.1 Add assessment-models/ sub-tree to the canonical repository

New top-level sub-tree at assessment-models/ containing:

assessment-models/
├── README.md                                   # entry point, with master cross-link
├── CHANGELOG.md                                # version history of the sub-tree
├── LICENSE                                     # MIT
├── .gitignore
├── .github/workflows/ci.yml                   # CI: schemas parse, examples validate, v2 band arithmetic
│
├── change-requests/
│   ├── README.md                               # sub-tree CR index
│   └── CR-AM-01-supplement-metamodel-v1.md     # supplementary CR (md5 c0f086be...) — the assessment sub-metamodel spec
│
├── model/
│   └── assessment-metamodel.puml              # canonical PlantUML class diagram
│
├── schemas/                                    # 12 JSON Schemas, Draft 2020-12
│   ├── common.schema.json
│   ├── assessment-model.schema.json
│   ├── assessment-instrument.schema.json
│   ├── assessment-execution.schema.json
│   ├── assessment-result.schema.json
│   ├── capability.schema.json
│   ├── scenario.schema.json
│   ├── measure.schema.json
│   ├── evidence.schema.json
│   ├── scoring-model.schema.json
│   ├── compatibility.schema.json
│   └── relationship.schema.json
│
├── vocabulary/                                 # 4 controlled vocabularies
│   ├── assessment-types.yaml
│   ├── relationship-types.yaml
│   ├── lifecycle-status.yaml
│   └── evidence-types.yaml
│
├── examples/                                   # 5 canonical YAML examples
│   ├── legacy-technology-instrument.yaml
│   ├── canonical-technology-assessment.yaml
│   ├── zero-touch-operations-assessment.yaml
│   ├── zero-touch-operations-result.yaml
│   └── benchmark-eligibility.yaml
│
├── governance/                                 # 3 policy docs
│   ├── versioning.md
│   ├── compatibility.md
│   └── lifecycle.md
│
└── maturity/                                   # NEW — maturity scoring v2
    ├── README.md                               # v2 scoring scheme + migration notes
    ├── maturity-bands-v2.yaml                  # canonical band definitions (renames + non-linear widths + effort_multiplier)
    ├── v2-to-v1-legacy-name-map.yaml           # explicit legacy_name aliases (Emergent→Ad Hoc, etc.)
    ├── examples/
    │   └── effort-adjusted-value.yaml          # worked example (score 80 → 49.2 effort-adjusted value units)
    └── governance/
        └── migration.md                        # 4-phase migration plan (advisory → beta → consumer flags → promotion)

2.2 Master cross-link from the canonical repository

  • Add a section to the canonical README.md introducing the assessment-models/ sub-tree (single canonical home; no separate dea-metamodel repo).
  • Add assessment-models/README.md cross-linking back to the canonical repo's metamodel/profiles/assessment/, pydantic/assessment*.py, and models/scenarios/.
  • Update metamodel/profiles/assessment/profile.yaml and metamodel/profiles/dmm/scoring.yaml if they exist in a form that needs a forward-pointer to the new sub-tree (surgical, only if needed; otherwise deferred).

2.3 Acceptance of the maturity-scoring proposal

  • Accept the proposal previously filed as Assessment-Models/dea-catalog-maturity-models PR #1 (docs/maturity-scoring-v2-proposal).
  • The archived PR remains visible as historical reference but is not the canonical implementation. The canonical implementation lands here under assessment-models/maturity/.
  • No change to v1-alpha maturity model YAMLs (maturity-models/v1-alpha/*.yaml) in this PR — those still live in the archived repo and remain the canonical v1 schema. The v2 scheme lands as the canonical v2 schema inside assessment-models/maturity/.

3. Why this lands now

Three forces converged:

  1. CR-AM-01 (Assessment Metamodel Evolution) established the 8-phase roadmap that requires a coherent assessment sub-metamodel.
  2. CR-AM-01 supplement (PlantUML + JSON Schemas + governance) provided the implementation-ready material.
  3. The maturity-scoring-v2 proposal established that the band renames + effort-multiplier scheme is technically justified.

CR-014 is the place where all three meet in the canonical repo.

4. Design rules (binding)

These rules govern every file under assessment-models/. They are derived from CR-AM-01 §5 (the architectural principle) and CR-AM-01 §46 (the migration strategy):

  1. Single source of authority. No dea-metamodel repo elsewhere. No dual-authority arrangement. Cross-boundary references inside the canonical repo just work — no kind: external declarations needed.
  2. No backward-incompatible changes to v1. The v1 maturity models (CMMI names, linear bands) remain valid. v2 lands alongside v1, not as a replacement.
  3. Backwards-compatible migration path. Bands v2 are advisory first; v1 stays canonical until promotion (Phase D).
  4. Effort multipliers are declarative, not enforced. They enable a computed metric; they do not change which level an organisation is "in". The decision-maker reads value_realised(score); the model publishes the multipliers.
  5. Sub-tree is additive. No change to metamodel.yaml, metamodel/profiles/*, pydantic/*, models/*, or any other canonical sub-tree, except for an optional cross-link from metamodel/profiles/assessment/profile.yaml.
  6. DMM-01 names-line stays parked. The Discrete / Converged / Composable / Cognitive / Autonomous progression (from earlier discussion) is not part of this CR. It will resurface as input to a Capability Model under CR-AM-01 §8 in a later CR.

5. Migration strategy

Per CR-AM-01 §46: additive migration only.

5.1 For the assessment sub-metamodel

No migration needed. The sub-tree is new. Existing instruments, scoring rubrics, and consumer tooling are unaffected.

5.2 For the maturity-scoring-v2 scheme

Per the proposal's §5, phased rollout:

Phase What ships Status in this CR
A — registry maturity/maturity-bands-v2.yaml published alongside v1 ✅ in this PR
B — beta files v2-to-v1-legacy-name-map.yaml + worked example ✅ in this PR
C — consumers dea-cli gains --scoring v2; dea-web-viewer renders both band sets behind a flag out of scope (separate CRs in technehub-labs/dea-cli and technehub-labs/dea-web-viewer)
D — promotion After one full assessment cycle on v2, v2 becomes canonical; v1 deprecated future CR

6. Acceptance criteria

  • All 12 JSON Schemas under assessment-models/schemas/ parse as JSON Schema Draft 2020-12.
  • The 5 example YAMLs under assessment-models/examples/ validate against their target schemas.
  • The PlantUML source assessment-models/model/assessment-metamodel.puml is syntactically valid PlantUML.
  • assessment-models/maturity/maturity-bands-v2.yaml defines 5 contiguous, non-overlapping bands covering 0–100.
  • The v2 band boundaries (20 / 25 / 25 / 18 / 12) sum to 100.
  • The worked example in assessment-models/maturity/examples/effort-adjusted-value.yaml reproduces: score 80 → effort-adjusted value = 49.2 (±0.1).
  • Every v2 level has a legacy_name mapping to its v1 counterpart (Emergent→Ad Hoc, Structured→Defined, Systematic→Managed, Adaptive→Quantitatively Managed, Self-Optimising→Optimising).
  • The canonical README.md contains a pointer to assessment-models/.
  • metamodel/profiles/assessment/profile.yaml (if modified) carries only an additive forward-pointer.
  • No existing file outside assessment-models/ has been modified except the canonical README.md and (optionally) metamodel/profiles/assessment/profile.yaml.
  • CI workflow under .github/workflows/ validates everything above on every push and PR.

7. Risks and mitigations

Risk Mitigation
PlantUML render tooling unavailable in CI Source-only delivery; render is a downstream concern (render locally, host on dea-web-viewer later)
Schemas refer to common.schema.json#/$defs/... — consumers need to resolve the $ref A consumer-side $ref resolver pattern is documented in assessment-models/README.md
Maturity v2 and v1 use different band schemes simultaneously v2 is advisory until Phase C (consumer support) lands; v1 stays canonical throughout Phases A–C
PR #1 on the archived maturity-models repo stays open forever A note is added to that PR explaining it has been superseded by this CR
Consumers of the archived Assessment-Models org hit dead ends The archived repos' READMEs already point at the CR-AM-01 xref; the canonical repo's README.md will absorb those pointers

8. Out of scope (recorded for future CRs)

  • Pilot migration of dea-assessment-technology to the canonical assessment sub-metamodel (CR-AM-01 Phase 3 — separate CR against dea-metamodel).
  • dea-catalog-maturity-models consumer update — the maturity YAML files in maturity-models/v1-alpha/ stay in the archived repo at v1; v2 lives in assessment-models/maturity/ (this CR). Migration to the canonical repo is a separate cleanup.
  • Capability / Scenario / Benchmark catalog repos (dea-catalog-capability-models, -scenarios, -benchmarks) — CR-AM-01 §18 says "introduce catalogs only when justified". Not yet justified; deferred.
  • dea-cli extension (dea assess run / validate / result / compare / benchmark / maturity / history) — separate CR in technehub-labs/dea-cli.
  • dea-web-viewer radar chart updates to render the new v2 bands behind a flag — separate CR.
  • DMM-01 names-line — separate CR, deferred until a Capability Model is justified.

9. Definition of done

This CR is implemented when:

  1. The assessment-models/ sub-tree exists on main of technehub-labs/dea-metamodel with the file structure in §2.1.
  2. All 12 schemas pass JSON Schema Draft 2020-12 validation.
  3. All 5 example YAMLs validate against their target schemas.
  4. The maturity-bands-v2.yaml + legacy-name-map.yaml + worked example reproduce the §6 acceptance criteria.
  5. The canonical README.md points at assessment-models/.
  6. CI runs all of the above on every push and PR to main.
  7. The PR has been merged via the canonical-repo CR process.
  8. The archived Assessment-Models org's PR #1 (maturity-scoring-v2-proposal) has been closed with a pointer to this CR.

10. References