Skip to content

About

Modular data platform building blocks wired through explicit contracts, not hidden dependencies

Topics

Resources

Code of conduct

Contributing

Security policy

Accessibility

Stars

8 stars

Watchers

1 watching

Forks

Latest commit

Β 

History

1,621 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

πŸš€ Composable Data Stack (CDS)

The modern data stack has a composition problem. CDS is an open framework for building data platforms that can evolve without being rebuilt.

Composable Data Stack logo

Project:

CI PyPI License

SonarCloud β€” Quality Gate & Ratings:

Quality Gate Status Reliability Rating Security Rating Maintainability Rating

SonarCloud β€” Issues:

Bugs Vulnerabilities Code Smells

SonarCloud β€” Metrics:

Coverage Duplicated Lines (%) Technical Debt


You start with a straightforward goal: orchestrate some pipelines, store the data, and make it useful.

Then the tool decisions begin.

Airflow, Dagster, or Prefect? Iceberg, Delta, or Hudi? Superset or Metabase? Every choice opens another comparison, another integration, and another architecture diagram. Every diagram looks different. Every tool has its own configuration model, assumptions, and lifecycle. Before long, the platform is less a designed system than a collection of decisions held together by custom code and institutional memory.

The hardest question is no longer "Which tool is best?" It is "How do these tools fit together, and can we change our minds later?"

Why Data Platforms Have a Composition Problem

The modern data ecosystem is excellent at producing capable tools and terrible at making them work as interchangeable parts.

Teams are expected to assemble orchestration, storage, transformation, cataloging, observability, security, and BI into one dependable platform. But the boundaries between those tools are usually implicit. Connection details leak into environment variables. Service names become dependencies. Deployment logic assumes a specific vendor. Compatibility lives in a staff engineer's headβ€”or is discovered during an incident.

That fragmentation creates a familiar cycle:

  • evaluate an overwhelming number of tools
  • debate architecture instead of delivering data products
  • hand-build integrations that only one team understands
  • duplicate stacks across development, CI, and production
  • postpone upgrades because nobody knows what they will break
  • accept lock-in because replacing one component feels like replacing the platform

This is not primarily a tooling problem. It is a composition problem. We have many good components, but no shared way to declare what they provide, what they need, and whether a particular assembly is valid.

The Cost of Choosing Wrong

A technology choice that looks local rarely stays local.

Choose an orchestrator and its concepts spread into deployment, secrets, logging, and team workflows. Choose a table format and it constrains engines, catalogs, and migration paths. Choose a BI layer and dashboards become coupled to authentication, database behavior, and operational assumptions.

The bill arrives later:

  • migrations become multi-quarter programs
  • upgrades become risky, manual compatibility exercises
  • platform teams maintain glue code instead of platform capabilities
  • product teams wait while engineers renegotiate architecture
  • vendors gain leverage because the cost of leaving keeps rising

There is no universally correct stack, and pretending otherwise is expensive. The right choice today may be the wrong choice at a different scale, under a new regulatory constraint, or after the team learns more. A healthy platform must make change routineβ€”not heroic.

What If Data Platforms Were Composable?

What if a data platform described capabilities and interfaces before naming products?

What if an orchestrator could request a database contract without hardcoding a specific database? What if a profile could declare a supported combination, validate it before deployment, and produce the runtime configuration? What if replacing a component meant satisfying the same contract instead of rebuilding every integration around it?

That is the goal of Composable Data Stack (CDS).

CDS is an open-source framework for defining, validating, securing, and assembling data platforms from reusable components. Think of it as Infrastructure as Code for the shape of a data platform: the components, the interfaces between them, and the rules that make the whole stack coherent.

CDS introduces three building blocks:

  • πŸ”§ Modules package reusable capabilities such as orchestration, storage, BI, and secret management.
  • πŸ”— Contracts define explicit, testable interfaces between those capabilities.
  • 🧩 Profiles compose modules into supported, reproducible stacks.

Instead of burying platform architecture in scripts and tribal knowledge, CDS makes it declarative. It answers:

  • How do these tools fit together? Through explicit contracts and declared bindings.
  • What happens if I replace a component later? CDS validates the new composition against the same requirements before runtime.
  • How do I avoid vendor and technology lock-in? Consumers depend on capabilities, not hardcoded product identities.

The result is composability without chaos, flexibility with guarantees, and a platform that can evolve without starting over.

⚑ Why CDS

Most teams are forced into one of two bad options:

Approach What You Gain What It Costs
Monolithic stack A consistent starting point Rigid boundaries and an expensive exit
Custom integrations Freedom to choose every component Fragile glue, inconsistent stacks, and hidden coupling
CDS Explicit composition and replaceable components A contract and profile model the community maintains together

CDS does not claim that every tool is interchangeable or that integration complexity disappears. It makes compatibility explicit, validates known constraints early, and gives the ecosystem a shared place to improve how tools compose.


🎯 When To Use CDS

Use CDS if you:

  • expect your architecture to evolve as requirements change
  • want to swap tools (Airflow ↔ Dagster, Superset ↔ Metabase)
  • need reproducible environments across dev, CI, and prod
  • are building a platform for multiple teams
  • want contract-driven integration instead of implicit coupling
  • want to contribute to an open interoperability model for data tooling

CDS may be overkill if:

  • you only run a single-tool stack
  • you do not need interchangeable components

πŸ—οΈ See Composition in Practice

The local-dagster-postgres-superset profile defines:

  • Dagster -> orchestration
  • Postgres -> storage
  • Superset -> BI

CDS then:

  1. Validates module definitions
  2. Resolves contract bindings
  3. Checks compatibility and security constraints
  4. Produces a fully wired stack definition

cds plan resolves the full dependency graph before runtime configuration is generated, ensuring all module interactions are valid and predictable.

The composition model creates a path to replace components while preserving the contracts the rest of the platform depends on:

Dagster -> Airflow
Superset -> Metabase
Postgres -> MariaDB

πŸ—ΊοΈ Architecture Overview

CDS wires modules through contracts, not direct dependencies. This section has two levels: a high-level picture of what gets wired together (Overview), and a detailed look at what happens when you run a CDS command (Internal Flow).

Overview

Below, local-dagster-postgres-superset wires Dagster to Postgres to Superset through contracts:

Dagster, Postgres, and Superset connected through contracts

Internal Flow

CDS splits into two phases: compile-time, where cds itself validates, resolves, and renders a plain docker-compose.yaml; and runtime, where the real docker compose binary builds and starts containers from that file. CDS never runs containers itself.

cds test runs the full compile-time pipeline in order β€” validate β†’ security β†’ plan β†’ render. A validate (or plan) failure skips all downstream stages, but a security failure does not skip plan/render: those stages still run so cds test reports every stage's status in one pass, while the overall command still exits non-zero if any stage failed. cds up runs the same pipeline minus security (validate β†’ plan β†’ render), then hands off to docker compose build/docker compose up. See the CLI table below for exactly what each command runs.

  • Validate checks profile shape, module configs, dependencies, secret refs, contract bindings, and outputs.
  • Security (cds test only) runs rule-based checks against modules and resolved secrets; a failing check is reported but does not prevent plan/render from also running.
  • Plan resolves contract bindings and substitutes secrets and defaults.
  • Render generates the final docker-compose.yaml, with secret values as ${CDS_VAR} placeholders; never the raw value.
  • Runtime (cds up only): docker compose build (skippable with --no-build), then docker compose up --detach, then a live cds state view that polls until the stack settles (skippable with --detach). Build/up/log output is persisted to a log file (.cds/logs/ by default). Docker Compose, not CDS, resolves ${CDS_VAR} placeholders from a .env file (see cds init) and starts the containers.

Passing validate β†’ security β†’ plan β†’ render proves the profile compiles correctly β€” it is not proof that the resulting stack runs correctly. For the broader success criteria used to certify a profile (booting the rendered stack, service health, an end-to-end pipeline run, and downstream data consumption in Superset), see docs/profile-testing/test-plan.md and docs/profile-testing/failure-path-and-ci.md.

CDS compile-time and runtime flow

This mirrors the cds command table below: validate, plan, and render are each callable on their own; security only runs as part of cds test, not cds up. Module and contract definitions follow the Contract-First design principle, so most of what "Validate" and "Plan" check comes directly from module.yaml and profile.yaml.

See also: Security for what the security stage checks, Troubleshooting for what each error code means and how to fix it, and docs/architecture.md Β§ Secrets and contract resolution for a deeper look at how secrets.* and contractRef values actually resolve.

πŸ” Security

CDS includes built-in security validation to prevent unsafe configurations before a stack is deployed.

The cds security checks analyze profiles and modules for common risks such as:

  • weak or default passwords
  • missing secret configurations
  • insecure service exposure
  • unsafe defaults in module configuration
  • incomplete contract bindings that may leak data

Security checks run as part of validation and can be extended with custom rules.

Example

cds security local-dagster-postgres-superset

πŸ“¦ What You Get

When a profile compiles, CDS produces:

  • validated module graph
  • resolved contract bindings
  • dependency-aware execution plan
  • generated Docker Compose configuration
  • generated Helm chart for Kubernetes
  • reproducible stack definition

This turns a declarative platform design into a reproducible, runnable local data stackβ€”and turns architectural assumptions into things the project can inspect, validate, and improve.


βœ… Prerequisites

CDS has separate compile-time and runtime requirements.

Validate, Plan, And Render

The commands cds validate, cds security, cds plan, and cds render do not require Docker. Install:

  • Python 3.14 or newer, including pip and venv
  • Git when cloning and installing CDS from source
  • Internet access for the initial source and Python dependency downloads

Build And Run A Stack

The cds up command and generated local profiles additionally require:

  • Docker Engine (Linux) or Docker Desktop (macOS and Windows)
  • Docker Compose v2, invoked as docker compose
  • A running Docker daemon accessible to the current user
  • Internet access for the initial container image pulls and builds, unless all required artifacts are already cached
  • Free host ports required by the selected profile
  • Write access to the checkout for .env, docker-compose.yml, and workdir data

For Kubernetes, install Helm and kubectl and provide an explicit kube context. The local k3s workflow uses k3d and is documented in docs/kubernetes.md.

Docker Desktop on Windows must use the WSL 2 backend. See the support policy for supported operating systems and runtime versions. For platform-specific setup from an empty machine, follow the complete installation guide.

For the complete example stack, allocate at least 8 GB of memory to Docker and keep at least 10 GB of disk space free for images, build layers, volumes, and logs. Larger workloads require additional resources.

Preflight Check

Linux/macOS:

python3 --version
git --version
docker --version
docker compose version
docker info >/dev/null

Windows PowerShell:

py --version
git --version
docker --version
docker compose version
docker info | Out-Null

Python must report version 3.14 or newer, Docker Compose must report v2, and docker info must complete without a daemon or permission error. Git is optional when CDS is installed from a package rather than source.

Tools such as make, Node.js, and pre-commit are contributor tooling and are not required to run CDS profiles.

πŸš€ Quickstart

CLI package preview

The Python distribution contains the cds command and its built-in security rules. After production PyPI publishing is enabled, install the isolated CLI with:

pipx install composable-data-stack

The package does not yet download profiles, modules, image build contexts, or runtime workdirs. Continue with a repository checkout for the reference stack, or configure external project assets with CDS_PROFILE_PATH and CDS_MODULE_PATH. See Packaging for the current TestPyPI flow.

1. Clone

git clone https://github.com/RonaldHensbergen/composable-data-stack.git
cd composable-data-stack

2. Setup Environment

Linux/macOS:

python3 -m venv .venv
source .venv/bin/activate
pip install -e .

Windows PowerShell:

py -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -e .

Windows CMD:

py -m venv .venv
.venv\Scripts\activate.bat
python -m pip install -e .

If PowerShell blocks the activation script, run Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass in the same terminal session and activate the environment again.

3. Configure Environment

cds init local-dagster-postgres-superset

Set:

CDS_ANALYTICS_DB_NAME
CDS_ANALYTICS_DB_USER
CDS_DAGSTER_DB_NAME
CDS_DAGSTER_DB_USER
CDS_SUPERSET_DB_NAME
CDS_SUPERSET_DB_USER
CDS_POSTGRES_SUPERUSER_PASSWORD
CDS_ANALYTICS_DB_PASSWORD
CDS_DAGSTER_DB_PASSWORD
CDS_SUPERSET_DB_PASSWORD
CDS_SUPERSET_SECRET_KEY
CDS_SUPERSET_ADMIN_PASSWORD

4. Validate A Stack

cds validate local-dagster-postgres-superset

Expected output:

Profile is valid.

5. Check Runtime Prerequisites

cds preflight local-dagster-postgres-superset

This checks the runtime CLI and daemon, Compose support, required environment values, and declared host ports without starting services. cds init only creates configuration, while cds validate checks profile structure.

6. Run Security Checks

cds security local-dagster-postgres-superset

7. Generate A Plan

cds plan local-dagster-postgres-superset

This resolves:

  • module dependencies
  • contract bindings
  • execution order

8. Render The Stack

cds render local-dagster-postgres-superset

By default, this writes docker-compose.yml to the project root.

Render the same plan as a Helm chart with:

cds render local-dagster-postgres-superset --target helm

Use a custom location when needed:

cds render local-dagster-postgres-superset --output build/docker-compose.yml

This generates:

  • docker-compose.yml
  • service definitions
  • fully wired module configuration

9. Run The Stack

cds up local-dagster-postgres-superset

This runs validate β†’ plan β†’ render β†’ docker compose build β†’ docker compose up in one step.

docker compose build output is streamed to the terminal (grouped by image) and also persisted to a log file (default: .cds/logs/up-<profile>-<timestamp>.log; override with --log-file). docker compose up always runs detached: in the default mode its output goes only to the log file while the terminal shows the live state view; with --detach it is streamed to the terminal like the build output. Once the stack starts, cds up polls docker compose ps and redraws the same grouped/colored view as cds state until every service is healthy/running/exited or a timeout is hit (default 180s; override with --timeout; disable colored labels with --no-color). Container logs from that point on go only to the log file, not the terminal, since the terminal is showing the live state view. cds up exits 0 once the stack settles, or 1 on timeout or an unhealthy service.

The rendered docker-compose.yml is a standard Compose file, so the stack also runs without cds β€” docker compose -f docker-compose.yml up -d works on its own from the project root. See docs/underlying-commands.md for the full mapping of every cds lifecycle command (render / up / state / down, Compose and Helm) to its underlying docker compose / helm / kubectl commands.

Add --detach (or -d) to skip the live state view and return as soon as the stack starts, e.g. for scripting or CI:

cds up local-dagster-postgres-superset --detach

Use --no-build to skip the build step when images are already available:

cds up local-dagster-postgres-superset --no-build

9. Persistent Incoming Data Folder For Dagster

The Dagster module mounts a host directory into the containers so incoming files survive reboots.

  • Host path: workdirs/shared-data/incoming
  • Container path: /app/data/cds/incoming

Dagster includes a sensor that detects new files in /app/data/cds/incoming and runs a pickup job. Picked files are moved to:

  • Host path: workdirs/shared-data/processed
  • Container path: /app/data/cds/processed

Create the directories once if they do not exist:

mkdir -p workdirs/shared-data/incoming workdirs/shared-data/processed

🧩 Core Concepts

Modules

Reusable building blocks:

  • orchestration (Dagster, Airflow)
  • warehouse (Postgres, MariaDB)
  • BI (Superset, Metabase)
  • secrets (env, vault)

Structure:

modules/<category>/<name>/
β”œβ”€β”€ module.yaml
β”œβ”€β”€ defaults.yaml
β”œβ”€β”€ compose.yaml
β”œβ”€β”€ scripts/
└── tests/

Contracts

Contracts define how modules interact.

Examples:

Contract Purpose
sql-database database interface
http-service service exposure
secrets-provider secret resolution

Example binding:

dagster.database -> postgres.sql-database
superset.database -> postgres.sql-database

No implicit dependencies. Everything is explicit.

Profiles

Profiles define supported stacks:

local-dagster-postgres-superset
local-airflow-postgres-superset
integration-airflow-postgres-dbt

Structure:

profiles/[profile]/
β”œβ”€β”€ profile.yaml
β”œβ”€β”€ values.yaml
└── README.md

Environment Overlays

A profile can be promoted from local development through staging to production without duplicating its configuration. Add an environments/ directory next to profile.yaml containing one YAML file per environment; each file is merged over the base profile when --environment <name> is passed to a profile-consuming command:

profiles/[profile]/
β”œβ”€β”€ profile.yaml
└── environments/
    β”œβ”€β”€ dev.yaml
    └── prod.yaml

An overlay only needs to declare the values that differ from the base profile. Module entries are merged by their stable id (not array position), so an overlay can override just one field of one module:

# profiles/[profile]/environments/prod.yaml
metadata:
  environment: production   # promotes the profile's security classification

spec:
  modules:
    - id: postgres
      config:
        storage:
          size: 20Gi
cds validate my-profile --environment prod
cds plan my-profile --environment prod
cds test my-profile --environment prod
cds up my-profile --environment prod

local-dagster-postgres-superset ships a working dev/prod example under profiles/local-dagster-postgres-superset/environments/ you can inspect or copy.

Setting metadata.environment in an overlay also changes which security policy cds security/cds test applies (see Security) β€” promoting to production enables stricter checks, so a profile that passes locally may report new findings once resolved with --environment prod. That's expected: it surfaces settings that are fine for local development but unsafe to carry into production.

Compare what an overlay actually changes with cds diff, without ever printing a secret value (profiles only ever hold secret references, never resolved values):

cds diff my-profile --from dev --to prod
Differences from 'dev' to 'prod':

  ~ metadata.environment: "development" -> "production"
  ~ spec.modules[postgres].config.storage.size: "2Gi" -> "20Gi"

Persist an environment default for a project with cds config set environment prod; commands without --environment then use that overlay. An explicit --environment always takes precedence.


Profile Composition (extends)

A profile can also factor out shared configuration into one or more parent profiles instead of duplicating it, using a top-level extends field:

# profiles/analytics-prod/profile.yaml
apiVersion: cds/v1alpha1
kind: Profile
metadata:
  name: analytics-prod
  environment: production
extends:
  - analytics-base   # a bare name resolves to profiles/analytics-base/profile.yaml
spec:
  modules:
    - id: postgres
      config:
        storage:
          size: 20Gi

extends accepts a non-empty list of parent references, each either a bare profile name (resolved under the profiles root) or a path relative to the child profile's directory (e.g. ../shared/profile.yaml). Parents are resolved and merged left-to-right β€” later parents win over earlier ones β€” and then the child profile's own document is merged on top of all parents. Composition uses the exact same deep-merge/module-merge-by-id engine as environment overlays: mappings merge recursively, spec.modules entries merge by stable id, and any other array is replaced wholesale rather than concatenated. Parent profiles may themselves use extends (chains are resolved transitively).

extends is not a CLI flag β€” it's read directly from profile.yaml, so every profile-consuming command resolves it automatically, with no new syntax to learn:

cds validate analytics-prod
cds plan analytics-prod
cds up analytics-prod

A profile can extend more than one parent, which is merged in the order listed (later entries win over earlier ones):

extends:
  - networking-base
  - observability-base

extends and --environment compose together: parents are merged first, then the child, then the selected environment overlay is applied on top of that fully-composed result β€” so a shared base profile and environment promotion can both be used without duplicating configuration in either dimension.

A malformed extends chain fails validation/planning before anything else runs, with a dedicated diagnostic code:

Code Meaning
E110 extends is missing, not a list, empty, or contains a non-string/empty entry
E111 A parent reference resolves outside the profiles root
E112 A referenced parent profile does not exist
E113 A cycle was detected in the extends chain

βš™οΈ CLI

See docs/cli-reference.md for the full command table, flag reference, and [profile] resolution rules. The commands used in the Quickstart above are cds init, cds validate, cds preflight, cds security, cds plan, cds render, and cds up; run cds --help or cds <command> --help for inline usage at any time.


πŸͺŸ Windows Task Runner

Windows contributors without make can use Makefile.ps1, a PowerShell equivalent covering the core developer tasks:

# Install in editable mode
.\Makefile.ps1 install

# Validate the default profile
.\Makefile.ps1 validate

# Validate a specific profile
.\Makefile.ps1 validate-profile -P profiles/local-dagster-postgres-superset/profile.yaml

# Build distribution packages
.\Makefile.ps1 package

# List available targets
.\Makefile.ps1 help

This does not replace the Linux/macOS Makefile, both exist side by side. Windows users can still install make via WSL or Chocolatey if they prefer the original workflow. lint and docker-build are not ported here. Run yamllint . and npx markdownlint-cli directly, or use pre-commit if it is set up in this repo. Docker Desktop's docker build works the same on Windows as it does elsewhere.


πŸ› οΈ Troubleshooting

Common errors from cds validate, cds plan, and cds render, and how to fix them.

Error Cause Fix
[E020] ... YAML file not found: <path> The profile identifier or file path passed to cds validate, cds plan, or cds render <profile> doesn't resolve to an existing YAML file. Run cds list profiles to see valid identifiers. Set CDS_PROFILE_PATH to a profile name, a profile.yaml file path, or a profiles root directory.
[E024] ... Module "<id>" requires cds >= <version>, but <installed> is installed A module referenced by the profile declares spec.requiresCliVersion (the minimum cds CLI version its templates need, e.g. for ${service.dns} or defaultByEnvironment) and the installed CLI is older, so the module would otherwise fail later at render time with E071. Upgrade the CLI (pip install -U composable-data-stack), or fetch assets pinned to a matching ref (cds get --ref v<installed-version> ...). See docs/modules.md Β§ CLI version requirement.
[E081] ... Required secret "CDS_X_PASSWORD" not found in environment A secret marked required: true in the profile's spec.secrets.values is missing from the shell environment or the .env file in the current working directory. Run cds init <profile> to generate .env in the project root, set the missing CDS_* variable, or export it directly before running the command.
[E041] ... Contract ref "x.y" points to unknown module "x" A consumes binding's contractRef refers to a module ID that isn't defined in the profile. Check spec.modules for the correct module id, and confirm the contract ref follows <module-id>.<contract-name>.
[E041] ... but it does not provide "<contract-name>" The referenced module exists, but its spec.provides list doesn't expose that contract name. Check the producing module's module.yaml for the contracts it actually provides, and fix the consumer's contractRef to match.
[E129] ... Certificate ref ... is not defined in spec.certificates.values (or malformed) A module config value references certificates.<alias>.cert/.key, but the alias isn't declared under spec.certificates.values -- or the reference is bare/has an unknown trailing part instead of .cert or .key. Declare the alias under spec.certificates.values with its certEnv/keyEnv names, or fix the reference to name one half of a declared pair. CDS only maps names; the certificate/key material itself is supplied at runtime by the external secret/cert provider.
[E042] ... Contract kind mismatch The consumer expects one contract kind (e.g. sql-database) but the producer exposes a different kind. Point the binding at a module that provides the expected contract kind, or update the consumer's expected kind if the mismatch is intentional.
[E043] ... recorded as unsupported in the compatibility registry The binding's kind matches, but this exact provider/consumer pairing is explicitly recorded as unsupported in cli/resources/compatibility-registry.json -- it's known NOT to work together despite matching shape. Use a different provider/consumer pairing for this contract, or if the pairing has since been fixed/verified, update its status to tested in the registry (see docs/architecture.md Β§ Secrets and contract resolution).
[E128] ... requires a TLS-protected endpoint (tls: required), but ... does not advertise one A module's consumes entry declares tls: required (issue #577), but the bound provider's contract spec does not set tls: true (absent, false, or a ${config.<path>} reference that does not resolve to true against the producer's profile config). Point the binding at a provider that advertises TLS, or have the provider set tls: true in its provide entry's contract spec. Drop tls: required only if the binding is genuinely allowed to run unencrypted.
[E154] ... expects database engine ... but contract ref ... advertises engine ... / ... advertises no "engine" in its spec A module's consumes entry declares engineFrom: spec.config.<path> (issue #682), but the bound provider's sql-database contract advertises a different engine, or none at all -- the consumer would speak the wrong wire protocol to the bound database (e.g. Keycloak's KC_DB vs the bound provider). The expected engine is read from the profile config at that path, falling back to the configSchema default when unset. Set the consumer's config field (e.g. keycloak metadataDatabase.driver) to the provider's advertised engine, bind a provider advertising the expected engine, or have the provider set engine in its provide entry's contract spec. Drop engineFrom only if the consumer genuinely accepts any engine.
[E130] ... Duplicate backup policy for workload class "<class>" spec.backup.policies declares more than one policy for the same workload class. Keep one policy per workload class.
[E131] ... Backup encryption key ref ... is not defined in spec.secrets.values spec.backup.encryption.keyFrom references a secrets.<alias> that the profile does not declare. Declare the alias under spec.secrets.values with its env name, or fix the reference.
[E132] ... Backup target contractRef ... The pinned spec.backup.target.contractRef is malformed, points to an unknown module or contract, is not a backup-target provider, or (in production) the provider does not advertise encrypted: true / supportsRestoreVerification: true. Point at a module providing a backup-target contract that advertises the required capabilities, or drop the pin to leave the storage provider open.
[E133] ... Production profiles must declare spec.backup ... / ... missing workload classes: ... A metadata.environment: production profile has no spec.backup block, or lacks a policy for a workload class it runs (warehouse, orchestration/BI/identity/secrets metadata, or configuration). Declare encrypted, restore-verified backup policies for every workload class the profile runs; see docs/backup-restore.md.
[E134] ... Production backups must be encrypted ... / ... must be declared required: true A production profile has no spec.backup.encryption.keyFrom, or the referenced secret alias is declared required: false so the key can be missing at runtime. Declare the encryption key alias with required: true so a missing key fails closed at startup.
[E135] ... Production backup policy "<class>" must declare restoreVerification A production backup policy has no restoreVerification block. Add restoreVerification.frequencyDays to every production backup policy.
[E136] ... Reverse-proxy route ... (malformed or unknown contractRef / no resolvable http-service endpoint / backend address not a rendered compose service / entrypoint not declared by the proxy / undeclared certificate alias / route with neither host nor pathPrefix / duplicate route rule) A config.routes entry on a reverse-proxy provider has a contractRef that is not <module-id>.<contract-name> or names a module that is missing, or that module does not provide the named http-service contract with host/port (or a port outside 1-65535), advertises a backend address that matches no rendered compose service on the Compose target -- or no rendered Kubernetes Service on the Helm target, where the same check reports E150 and the generated file-provider ConfigMap dials the backend by its release-scoped Service DNS name (multi-service modules must advertise serviceName), names an entrypoint the proxy's config.entryPoints does not declare, references an undeclared certificate alias, names neither host nor pathPrefix, or duplicates an earlier route's rule on the same proxy (#581, #883). Fix the route's contractRef/certificate/host/pathPrefix/entryPoints (entrypoints must match the proxy's declared config.entryPoints keys), or advertise the serving compose service via serviceName in the target's http-service provide entry.
[E150] ... Provided contract ... advertises network host ... A module's provided http-service, reverse-proxy, sql-database, cache-service, object-storage, lineage-sink, log-sink, metrics-sink, or trace-sink contract advertises a network host (the host field, the endpoint's or connectionUri's hostname part, or a serviceName mapped through the renderer's service-naming rule; a scheme-less host:port value contributes its host part, #1036) that matches no service the render produced for that module -- no compose service on the compose target, no rendered Kubernetes Service on the helm target -- so consumers would resolve a hostname nothing answers (#1010, #1029, #1035). Compose: instantiate the module with an id matching its compose service name, advertise the serving service via a serviceName field in the provide entry, or build the endpoint from ${service.dns} or ${service.services.<name>}. Helm: render the serving workload with a service block (or as a StatefulSet, which gets a governing Service), advertise the serving workload via a serviceName field, or build the endpoint from ${service.dns}.
[E151] ... Raw manifest ... must be an object with apiVersion, kind, and metadata.name / ... must declare non-empty string apiVersion, kind, and metadata.name A raw manifest under a module's implementation.kubernetes.manifests is malformed -- a plan loaded from a file bypasses the module schema, so the Helm renderer fails closed instead of shipping a broken manifest (issue #767). Fix the manifest shape in the module (or the saved plan): every entry needs string apiVersion/kind and a metadata.name.
[E103] ... config.image.tag is required ... when config.image.source is "registry" A module instance sets config.image.source: registry without also setting config.image.tag. Set config.image.tag to a version, or switch back to config.image.source: build.
[E139] ... without an explicit host authentication advertisement / ... host authentication method "trust" A metadata.environment: production profile runs a sql-database provider whose contract does not advertise a host authentication method, or advertises trust (issue #206). Configure the provider's authentication explicitly (e.g. postgres config.authentication.hostAuth: scram-sha-256); trust authentication is prohibited in production profiles.
[E140] ... configures webServer.mode "development" A production profile sets an HTTP-serving module's config.webServer.mode to development, selecting its built-in development web server (issue #216). Set config.webServer.mode to production (or remove the override) so the service runs behind its production process manager; see docs/serving-modes.md.
[E137] ... accepts unauthenticated clients A production profile runs a cache-service provider that does not advertise authentication: required (no password credential configured). Supply a cache password through a secrets.<alias> reference (e.g. keydb config.passwordFrom), declared required: true in spec.secrets.values.
[E138] ... Credential field ... must reference a secret alias / ... is empty / ... uses a ${CDS_*} profile input / ... contains a plaintext value A credential-named config field of a production database/cache provider is not a secrets.<alias> reference, or its alias is not declared required: true (issue #206). Supply the credential through a secret reference declared required: true so a missing value fails closed at startup.
[W105] ... legacy md5 host authentication A production sql-database provider advertises md5, which validates but is a deprecated password method. Prefer scram-sha-256 for production profiles.
[E148] ... RBAC rule ... must be a mapping with non-empty "apiGroups", "resources", and "verbs" lists / ... The "serviceAccount" block ... must be a mapping ... A module's implementation.kubernetes.serviceAccount block (or one of its rbac rules) is malformed -- a plan loaded from a file bypasses the module schema, so the Helm renderer fails closed instead of shipping a broken Role (#75). Fix the serviceAccount/rbac shape in the module (or the saved plan).
[E149] ... The "networkPolicy"/... block ... must be a mapping with optional "ingress"/"egress" lists ... A module's custom NetworkPolicy allow rules are malformed (plan-file bypass of the module schema) (#75). Fix the networkPolicy shape; each rule must be a mapping.
[E152] ... waitFor gate of workload "<module>/<workload>" names host "<host>", ... On the Helm target, a waitFor gate names a host that is neither a Service of the same module emitted by this render (a workload disabled via enabledFrom gets no Service), a Service of a provider module the module consumes (the binding-derived egress edge is provider-module-wide, #1125), nor the target of one of the module's reverse-proxy routes, and no custom networkPolicy.egress ipBlock rule or declared implementation.kubernetes.egress entry covers the host:port, so the default-deny NetworkPolicy blocks the init container's connection and it would only hang until its timeout (#1031, #1033, #1023, #1125, #1126). A gate naming another module's in-cluster Service always fails closed, even with a passthrough rule declared. Bind the host's module through a consumes contract, route to it via config.routes, gate on a same-module ${k8s.service.*} Service, declare the destination under implementation.kubernetes.egress (an IP/CIDR with the gate's port), or add a custom networkPolicy.egress ipBlock rule covering the host and the gate's port (a DNS-name host cannot be matched against ipBlock CIDRs at render time: with such a passthrough rule declared that could cover it, the gate renders with a W133 warning instead).
[W133] ... waitFor gate of workload "<module>/<workload>" names host "<host>", which is neither a Service rendered in this chart nor an IP literal ... On the Helm target, a waitFor gate names a DNS-name host that is not a Service in this chart, so the renderer cannot match it against the module's custom networkPolicy.egress ipBlock rules at render time (no DNS resolution); the gate renders because a declared passthrough rule has an ipBlock peer and opens the gate's port, so the resolved address may be covered at runtime (issue #1126). If no ipBlock rule actually covers the resolved address (honoring except), the init container will only hang until its timeout. Confirm the passthrough rule covers the host's resolved IP/CIDR on the gate's port; to make the render check exact, gate on the resolved IP instead of a DNS name.
[E153] ... External egress entry of module \"<module>\" at ... is invalid ... A declared external egress entry under implementation.kubernetes.egress (or workloads.<name>.egress) is malformed -- plan-file bypass of the module schema (#1023): exactly one of host (IP literal; DNS names are rejected) / cidr (network form, no host bits), except CIDRs inside the cidr, port/ports within 1-65535, protocol TCP/UDP/SCTP, and all ${config.*} expressions must resolve. Fix the entry shape in the module (or the saved plan); see docs/kubernetes.md ("Declaring external egress").
[E162] Malformed waitFor gate at "module:<module>.implementation.kubernetes.workloads.<name>.waitFor[<i>]" ... A waitFor gate entry of a Kubernetes workload is malformed -- a plan loaded from a file bypasses the module schema (#1130): the block must be a list of mappings, each with a non-empty string host and a port (integer or string), and timeoutSeconds (if present) an integer. Without the re-check the Helm renderer would crash with a raw KeyError/TypeError instead of a diagnostic. Fix the entry shape in the module (or the saved plan): every waitFor entry needs a non-empty string host and a port, and timeoutSeconds must be an integer when present.
[W128] ... carries no numeric port and provider module ... advertises no service ports ... A contract binding carries no port evidence, so its generated NetworkPolicy egress rule allows all ports to the provider (#75). Declare the port in the contract so the rule stays port-scoped.
[W129] ... Reverse-proxy route ... cannot be derived as a NetworkPolicy edge ... On the Helm target, a config.routes entry of a reverse-proxy module has a contractRef naming an unknown module, a module without the named http-service provide, or a target whose http-service contract carries no usable port evidence (absent, or outside 1-65535, and the target advertises no service ports), so no port-scoped NetworkPolicy edge can be derived and the proxy's default-deny policy keeps the target unreachable (#1033, #883). Fix the route's contractRef (<module-id>.<contract-name> naming an existing http-service provide) and declare a valid port in that contract, or add a custom networkPolicy.egress rule for the target.
[W130] ... renders resources for operator "<name>" (CRD <crd>), which CDS does not install itself ... On the Helm target, a module declares an out-of-band operator under implementation.kubernetes.operators (e.g. the CloudNativePG operator of modules-experimental/warehouse/postgres-operator): CDS renders the CRs but never installs the operator, so the chart only reconciles once the operator is on the cluster (#768). The warning names the install command; the same note is in the chart's NOTES.txt. Run the install command from the warning (e.g. helm repo add cnpg https://cloudnative-pg.github.io/charts && helm upgrade --install cnpg --namespace cnpg-system --create-namespace cnpg/cloudnative-pg), verify with kubectl get crd clusters.postgresql.cnpg.io, then re-run; see docs/kubernetes.md ("Operator-managed modules").
[W131] ... provides open-data-provider contract ... with classification ... which is outside the recommended DCAT-AP-NL data-theme vocabulary ... A module provides an open-data-provider contract whose classification is free text outside the recommended standard vocabulary (the DCAT-AP-NL data themes: AGRI, ECON, EDUC, ENER, ENVI, GEOJ, GOVE, HEAH, INTR, JUST, REGI, SOCI, TRAN, or the full EU data-theme authority URI), so the dataset is harder to compare and find across domains (issue #845). The value still renders; this is a warning only. Set classification to a theme from the DCAT-AP-NL data-theme vocabulary (e.g. in the postgres module's config.dataProvider.classification); see shared/contracts/open-data-provider.yaml and docs/fds-alignment.md.
[E155] ... needs operator "<name>", but its CRD "<crd>" is not present on the cluster ... cds up --target helm checked the cluster (kubectl get crd <crd>) and the operator a module depends on is confirmed absent, so helm upgrade --install would fail with "no matches for kind" (#768). Install the operator with the command named in the error (e.g. helm upgrade --install cnpg --namespace cnpg-system --create-namespace cnpg/cloudnative-pg after helm repo add cnpg https://cloudnative-pg.github.io/charts), then re-run cds up --target helm.
[E156] ... forEach marker of manifest ... resolved ... to nothing / to a <type>, but fan-out needs a list / produced the duplicate metadata.name ... A raw manifest under implementation.kubernetes.manifests declares forEach: <expr>, but the expression does not resolve to a list in the plan, or two fanned-out copies resolve to the same metadata.name (they would collide in the release namespace and in the chart's template file names) (issue #1170). Point forEach at a list-valued config path (e.g. config.services), and reference ${item...} in the manifest's metadata.name so every fanned-out copy is uniquely named.
[E157] ... Duplicate datasetId ... spec.dataSharing.datasets declares the same datasetId more than once; each disclosed dataset must be declared once (issue #844). Remove the duplicate entry, or give each entry a distinct datasetId.
[E158] ... Dataset ... names module ... which is not an enabled module instance ... A spec.dataSharing.datasets entry's module names a module id that is not an enabled module instance in the profile (issue #844). Point module at an existing, enabled module instance id from spec.modules, or drop the field.
[E159] ... enrolls service "<name>" in its service-mesh config (config.services[<i>].name), but no module instance with that id exists in the profile ... At plan time, a module providing the service-mesh contract has a config.services[].name that matches no module instance id of the profile (a typo such as warehose for warehouse, or a disabled module) (issue #1179). The rendered per-service AuthorizationPolicy would select zero pods via the cds.dev/module label -- a silent no-op allow policy. Fix the enrollment to name an existing module instance id (the error lists the planned ids), add the module with that id, or remove the entry from config.services; enable a disabled module you want to enroll.
[E161] ... Unknown module id '<id>' ... / ... addresses/traverses a non-scalar value ... / ... matches a resolved secret value ... A persisted cds config set module.<id>.config.<path> <value> override in .cds/config.json or the shared cds.shared.json (issue #1220) names a module id that is not in the resolved profile, points at a dict/list instead of a scalar leaf, or tries to store a resolved secret value. The same code is emitted at load time when a hand-edited config.json breaks, so validate/plan/render fail loudly instead of silently ignoring the override. Fix the id or path (only module.<id>.config.<scalar leaf> keys are accepted), remove the override with cds config unset [--shared] module.<id>.config.<path>, or store a secrets.<alias> reference instead of a secret value; see docs/cli-reference.md.
[E163] ... waitFor gate of workload \"<module>/<workload>\" names host \"<host>\" on port <port>, but no binding to that host opens that port ... On the Helm target, a waitFor gate names the endpoint of a consumed provider (so the host itself passes E152), but its port is one no binding to that provider opens: the binding-derived NetworkPolicy egress rule is port-scoped (the contract's Service-facing port mapped through the provider Service's targetPort), so the init container's connection to host:port would be denied and only hang until its timeout (#1124). Gate on one of the binding's ports (e.g. ${bindings.<name>.port}), open the port on the provider's Service and contract (keeping the port/targetPort mapping consistent), or cover the host and port with a custom networkPolicy.egress ipBlock rule or a declared implementation.kubernetes.egress entry.
[E164] ... Provided contract ... advertises port <port> on <compose service / Kubernetes Service> \"<host>\", but that service declares no such container port ... A module's provided http-service, sql-database, cache-service, object-storage, lineage-sink, log-sink, metrics-sink, or trace-sink contract advertises a host that is a service the render produced (so it passes E150), but its numeric port β€” or, for the sink kinds, the port embedded in connectionUri (and for object-storage, in endpoint) β€” is not one of that service's declared container ports (runtime.service.ports[].containerPort, the compose service's container port, or on Helm the Service's port/targetPort). That is usually the host-published loopback port advertised instead of the in-network port (#1067): it renders green but consumers get connection refused at runtime (#1245). Advertise one of the serving service's declared container ports on the contract (never the 127.0.0.1:${config.port}:<containerPort> host publish), or declare the port on the service (runtime.service.ports / compose ports / the workload Service's port/targetPort).
[E165] Malformed port declaration at "module:<module>.implementation.kubernetes.workloads.<name>..." ... On the Helm target, a workload port declaration -- service.ports[] port/targetPort/nodePort or containerOverrides.<container>.ports[] containerPort -- does not resolve to a port number (a plan loaded from a file bypasses the module schema; listen-address values like :80 are accepted, issue #1132). Without the re-check the renderer would crash or emit a null port instead of a diagnostic. Fix the port value in the module (or the saved plan); entrypoint addresses such as :8080 parse to their port.
[E166] Certificate mount ... of module "<module>" ... A certificateMounts entry of a module's implementation.kubernetes block is malformed -- a plan loaded from a file bypasses the module schema -- or declares neither fromBind nor mountPath (issue #1132). Fix the certificateMounts shape in the module (or the saved plan): each entry is a mapping with a fromBind (the compose bind it replaces) and/or a mountPath.
[E167] Consume entry "<name>" ... binds governed contract "<ref>" ... A module's consumes entry binds a contract whose provider advertises governed: true in its contract spec (a literal boolean, or a ${config.<path>} reference resolving against the producer's profile config), but the consume entry declares no policyFrom, or its policyFrom path resolves to no binding -- the consumer would reach governed data without a data-access policy (issue #1284). Add policyFrom: spec.config.<path> to the consume entry and bind a data-access-policy contract at that path (e.g. {contractRef: policy.data-access-policy}), or bind an ungoverned provider. See docs/data-access-policy.md.
[W132] ... enrolls service "<name>" in its service-mesh config, but the module renders no Kubernetes workload pods ... On the Helm target, a known enrollment names a module with no rendered workloads, such as a manifest-only module or one whose workloads are all disabled by enabledFrom (issue #1198). If enabled, the per-service AuthorizationPolicy would select zero pods via the cds.dev/module label. Enroll a module that renders at least one Kubernetes workload, or enable one of its gated workloads; remove the enrollment if no pod workload is intended.
[W134] ... declares \"<egress|networkPolicy.ingress|networkPolicy.egress|workloads.<name>.egress>\", but the module renders no Kubernetes workload pods, so it gets no NetworkPolicy and the declaration has no effect ... On the Helm target, a module that renders no pods of its own -- a manifests-only module (e.g. a GitOps bootstrap) or one whose workloads are all disabled by enabledFrom/scaled to zero -- declares implementation.kubernetes.egress, a workload-scoped workloads.<name>.egress, or networkPolicy rules: the entries are validated (E153/E149) but no NetworkPolicy is rendered for the module, so the declaration would otherwise be silently dropped (issue #1131). Move the declaration to a module that renders pods, or remove it; see docs/kubernetes.md ("Declaring external egress").
[W097] ... config.image.tag is "latest" with config.image.source "registry" config.image.tag: latest under source: registry still validates, but drifts silently between deploys instead of pinning a reproducible version. Pin an explicit tag from publish-images.yml's output (or tests/fixtures/signed-images.json) instead of latest.
[E127] ... Unknown confinement key ... / ... must be 'default', 'unconfined' or 'file:<path>' ... / ... Invalid SELinux label(s) ... A security.runtime.confinement block (or a module instance's confinement) is malformed: an unsupported key, an invalid seccomp/apparmor/selinux value, or a file: seccomp profile path that is missing or resolves outside the profile directory. Use seccomp: default/unconfined/file:<relative path inside the profile directory>, apparmor: runtime/default/unconfined/<host profile name>, and selinux: disable or a label: value list; see docs/runtime-confinement.md.
[W071] ... has kind "docker-compose" but no "compose" definition A module implementation declares the docker-compose kind without an implementation.compose block, so the renderer skips the module. Add the implementation.compose block (services, volumes) to the module's module.yaml, or set a supported implementation kind.
[W072] ... Container "<name>" in Kubernetes workload "<module>/<workload>" has no resource requests or limits Rendering the Helm target: a container or init container listed in a Kubernetes workload has no matching entry under the workload's resources. Add a resources.<container> entry with requests and limits to the workload in the module's Kubernetes implementation (see docs/kubernetes.md).
[W090] ... Invalid .env line format: "<line>" (expected KEY=VALUE) A line in the .env file is neither blank, a # comment, nor in KEY=VALUE form, so it is ignored. Fix the line to KEY=VALUE form, comment it out with #, or remove it.
[W098] ... Applied plaintext endpoint exposure waiver for production profile (modules: ...) A production profile's spec.security.waivers.plaintextEndpointExposure waived the CDS-SEC-074 plaintext-endpoint findings; CDS reports the waiver so it stays visible in security output. Confirm the waiver is still justified and its reason is accurate; see docs/security-rules.md (Warnings).
[W099] ... plaintextEndpointExposure is set but has no effect for profile class ... spec.security.waivers.plaintextEndpointExposure is declared on a non-prod profile, where CDS-SEC-074 and its waiver only apply to prod-class profiles. Remove the waiver from the non-prod profile, or move it to the prod profile (overlay) it is meant for.
[W103] ... Applied runtime policy waiver for production profile (checks: ...) A production profile's spec.security.waivers.runtimePolicy suppressed CDS-SEC-090 violations; CDS reports the waiver so it stays visible in security output. Confirm the waiver is still justified and its reason is accurate; see docs/security-rules.md (Warnings).
[W104] ... runtimePolicy is set but has no effect for profile class ... spec.security.waivers.runtimePolicy is declared on a non-prod profile, where CDS-SEC-090 and its waiver only apply to prod-class profiles. Remove the waiver from the non-prod profile, or move it to the prod profile (overlay) it is meant for.

All diagnostics print with their error code and YAML path (e.g. spec.modules[1].config), so search the profile file for that path to find the exact line to fix.


πŸ”„ Workflow

1. cds validate -> check module definitions
2. cds security -> detect unsafe configurations
3. cds plan -> resolve dependencies and bindings
4. cds render -> generate Docker Compose stack
5. cds up -> start services
6. cds test -> one-shot validate + security + plan + render smoke check

πŸ“‚ Repository Structure

.
β”œβ”€β”€ cli/
β”œβ”€β”€ modules/
β”‚   β”œβ”€β”€ bi/
β”‚   β”œβ”€β”€ orchestration/
β”‚   β”œβ”€β”€ secrets/
β”‚   └── warehouse/
β”œβ”€β”€ profiles/
β”œβ”€β”€ docs/
β”œβ”€β”€ pyproject.toml
└── Makefile

🧱 Design Principles

Contract-First

Modules declare:

  • what they provide
  • what they require
  • configuration inputs
  • health checks
  • lifecycle hooks

Profile-Driven

Profiles define supported stacks. The profile is the unit of support, not individual modules.

Zero Hidden Coupling

  • no implicit environment variables
  • no cross-module assumptions
  • no shared mutable state

All interactions happen through explicit contracts.

Security By Default

CDS validates configurations before runtime, ensuring that:

  • weak credentials are detected early
  • secrets are properly configured
  • services are not unintentionally exposed

Security is part of platform composition, not an afterthought.

One Model, Multiple Environments

The same composition model applies across:

  • local development
  • CI environments
  • production

Only runtime packaging differs.


πŸ“Š Comparison

Capability Monolith Custom pipelines CDS
Swap components ❌ ⚠️ βœ…
Reuse modules ❌ ❌ βœ…
Explicit contracts ❌ ❌ βœ…
Reproducibility ⚠️ ⚠️ βœ…
Security validation ❌ ❌ βœ…
Vendor lock-in βœ… ⚠️ ❌

πŸ“Œ Status

MVP ready:

  • module validation
  • contract resolution
  • security checks
  • profile composition
  • Docker Compose rendering
  • Kubernetes Helm rendering and local k3s lifecycle
  • runtime startup, health state, and shutdown

Next:

  • advanced secret providers
  • stack bootstrap and health checks

See docs/roadmap.md for milestones and detailed status. See docs/support-policy.md for OS support policy and platform-specific limitations.


🀝 Contributing

CDS will only become a useful interoperability layer if the people building and operating data platforms shape it together.

  • Run a profile and report what works, what breaks, and what is missing.
  • Document compatibility between providers and consumers with evidence.
  • Add or improve modules without coupling them to a specific stack.
  • Strengthen the model through contracts, validation, security rules, and representative profiles.
  • Star and follow the repository if you want to track the project.

Real-world feedback is especially valuable. CDS should reflect how platforms actually fail and evolve, not how architecture diagrams pretend they do.

Please read these first:

Development helper tools are located in the gitignored tools/ directory. See tools/pr-cli/README.md for PR creation scripts.


πŸ“– Documentation

  • Quickstart β€” get running in 5 minutes
  • CLI Reference β€” full command/flag table and [profile] resolution rules
  • Underlying Commands β€” the exact docker compose / helm / kubectl commands behind cds render / up / state / down, for running the stack without the cds CLI
  • How CDS Works β€” C4 + sequence diagrams of the validate β†’ plan β†’ render engine
  • Kubernetes Target: Helm rendering, Docker Compose migration guide, and isolated local k3s workflow β€” with ready-made example profiles (profiles/local-kubernetes-example, profiles/production-kubernetes-example with its prod overlay)
  • From Docker Compose to CDS Profile β€” complete transformation guide
  • Architecture β€” design and core concepts
  • Modules β€” how to structure reusable components
  • Repository Layout β€” best practice for keeping platform assets (modules, profiles) separate from project code (Dagster pipelines, dbt/dlt, database scripts)
  • Workload Runtime API β€” cds-runtime: named database connections for Python workloads
  • Production Hardening Checklist β€” control-to-owner mapping and the operator verification procedure for a production profile
  • Roadmap β€” planned features and milestones

πŸ“œ License

See LICENSE. For security, support, and regulatory-compliance documentation, see LEGAL.md.

About

Modular data platform building blocks wired through explicit contracts, not hidden dependencies

Topics

Resources

Code of conduct

Contributing

Security policy

Accessibility

Stars

8 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages