Server to/fro implementation with composable models, routes, mocks/fakes

Author: SamuelMarks

.github/workflows/ci.yml

@@ -14,13 +14,13 @@ jobs:
   build:
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v7
     - name: Set up Python
       uses: actions/setup-python@v6
       with:
         python-version: '3.11'
     - name: Checkout Test Harness
-      uses: actions/checkout@v6
+      uses: actions/checkout@v7
       with:
         repository: SamuelMarks/cdd-openapi-test-harness
         path: cdd-openapi-test-harness
@@ -60,7 +60,7 @@ jobs:
     if: startsWith(github.ref, 'refs/tags/v')
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v6
+    - uses: actions/checkout@v7
     - uses: actions/download-artifact@v8
       with:
         name: python-build

.gitignore

@@ -185,3 +185,4 @@ update_tests.py
 update_tests2.py
 update_tests3.py
 update_tests4.py
+.tmp_*

.pre-commit-config.yaml

@@ -38,6 +38,18 @@ repos:
     language: system
     pass_filenames: false
     always_run: true
+  - id: swagger-2-0-generated-server-test
+    name: Swagger 2.0 Generated Server Test
+    entry: uv run python scripts/run_generated_server_test.py petstore.json .tmp_gen_swagger
+    language: system
+    pass_filenames: false
+    always_run: true
+  - id: openapi-3-2-0-generated-server-test
+    name: OpenAPI 3.2.0 Generated Server Test
+    entry: uv run python scripts/run_generated_server_test.py petstore_oas3.json .tmp_gen_oas3
+    language: system
+    pass_filenames: false
+    always_run: true
   - id: build-wasm
     name: Build WASM
     entry: uv run python scripts/build_wasm.py

README.md

@@ -1,87 +1,174 @@
 cdd-python-all
-==============
+============
+
 [![License](https://img.shields.io/badge/license-Apache--2.0%20OR%20MIT-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 [![interactive WASM web demo](https://img.shields.io/badge/interactive-WASM_web_demo-blue.svg)](https://offscale.io/wasm_web_demo)
 [![CI](https://github.com/offscale/cdd-python-all/actions/workflows/ci.yml/badge.svg)](https://github.com/offscale/cdd-python-all/actions)
-[![Test Coverage](https://img.shields.io/badge/test_coverage-99%25-brightgreen.svg)](#)
+[![Test Coverage](https://img.shields.io/badge/test_coverage-100%25-brightgreen.svg)](#)
 [![Doc Coverage](https://img.shields.io/badge/doc_coverage-100%25-brightgreen.svg)](#)
 
-----
+**Compiler Driven Development (CDD)** is a development approach designed to eradicate the disconnect between: API specifications; server implementations; client SDKs; and command-line tooling.
 
-OpenAPI ↔ Python. This is one compiler in a suite, all focussed on the same task: Compiler Driven Development (CDD).
+Unlike traditional code generators—that treat outputs as disposable or read-only—CDD provides a **complete, standalone compiler** for each supported language. These compilers are fully CST-aware (Concreate Syntax Tree is a whitespace+comment aware Abstract Syntax Tree), allowing true bidirectional synchronization between existing hand-edited source code and OpenAPI specifications.
 
-Each compiler is written in its target language, is whitespace and comment sensitive, and has both an SDK and CLI.
+---
 
-The core philosophy of Compiler Driven Development (CDD) is synchronization without compromise. Where traditional generators silo your API boundaries into read-only files, this compiler natively merges changes into your codebase via a robust, [whitespace and comment aware] Abstract Syntax Tree (AST) driven parser & emitter. It bridges the gap between design and implementation, allowing you to seamlessly generate SDKs from a spec or extract a spec from existing code. By keeping your APIs, SDKs, and tests in continuous, automated alignment, it drastically improves both delivery speed and software reliability.
+## 🏗️ The Standalone Compiler Architecture
 
-The CLI—at a minimum—has:
+Traditional tools use naïve templating—if you regenerate, your custom code is overwritten.
 
-- `cdd-python-all --help`
-- `cdd-python-all --version`
-- `cdd-python-all from_openapi to_sdk_cli --mcp -i spec.json`
-- `cdd-python-all from_openapi to_sdk -i spec.json`
-- `cdd-python-all from_openapi to_server -i spec.json`
-- `cdd-python-all to_openapi -f path/to/code`
-- `cdd-python-all to_docs_json --no-imports --no-wrapping -i spec.json`
-- `cdd-python-all serve_json_rpc --port 8080 --listen 0.0.0.0`
+The CDD ecosystem is fundamentally different. It utilizes language-specific, standalone compilers capable of full AST parsing, semantic diffing, and surgical patching.
 
-## SDK Example
+**The Core Guarantee:** Every part of the generated codebase is fully editable.
+You are encouraged to open the generated routing files, model definitions, and CLI structures, and directly inject your business logic.
 
-```py
-from cdd import generate_sdk, Config
+- **When your specification changes**, the CDD compiler reads your code, builds an AST, diffs it against the new spec, and safely patches in new endpoints or fields without touching your custom logic.
+- **When your codebase changes**, the compiler reverse-engineers your structural updates back into a 100% accurate, authoritative OpenAPI specification.
 
-if __name__ == '__main__':
-    config = Config(input_path='spec.json', output_dir='src/models')
-    generate_sdk(config)
-    print("SDK generation complete.")
-```
+---
 
-## Installation
+## 🔄 The Bidirectional Synchronization Loop
 
-```bash
-python3 -m pip install -e .
-```
+```mermaid
+flowchart TD
+    OAS["📄 OpenAPI v3 Spec"] <--> CDD{"⚙️ CDD Compiler"}
 
-## Development
-
-You can use standard tooling commands or the included cross-platform Makefiles to fetch dependencies, build, and test:
-
-```bash
-python3 -m pip install -e .
-python3 -m pytest
-# or
-make deps
-make build
-make test
-# or on Windows
-.\make.bat deps
-.\make.bat build
-.\make.bat test
-```
+    CDD <--> Codebase
 
-See [PUBLISH.md](PUBLISH.md) for packaging and releasing.
+    subgraph Codebase ["💻 Application Codebase"]
+        direction TB
 
-## Features
+        subgraph Outputs ["📦 Primary Outputs"]
+            direction TB
+            CLI["⌨️ CLI Tooling"]
+            SDK["📦 Client SDK"]
+            Server["🖥️ Server"]
 
-The `cdd-python-all` compiler leverages a unified architecture to support various facets of API and code lifecycle management. For a deep dive into the compiler's design, see [ARCHITECTURE.md](ARCHITECTURE.md).
+            %% Force vertical stacking inside the subgraph
+            CLI ~~~ SDK ~~~ Server
+        end
 
-- **Compilation**:
-    - **OpenAPI → `Python`**: Generate idiomatic native models, network routes, client SDKs, and boilerplate directly from OpenAPI (`.json` / `.yaml`) specifications.
-    - **`Python` → OpenAPI**: Statically parse existing `Python` source code and emit compliant OpenAPI specifications.
-    - **Model Context Protocol (MCP)**: Generate and run MCP clients and servers, exposing your APIs seamlessly to AI models.
-- **AST-Driven & Safe**: Employs static analysis instead of unsafe dynamic execution or reflection, allowing it to safely parse and emit code even for incomplete or un-compilable project states.
-- **Seamless Sync**: Keep your docs, tests, database, clients, and routing in perfect harmony. Update your code, and generate the docs; or update the docs, and generate the code.
+        subgraph Core ["🔗 Core Architecture"]
+            direction TB
+            Models["🔗 Data Models"]
+            Routes["🔀 API Routes"]
+            Tests["🧪 Tests"]
 
-**Uncommon Features:**
+            %% Force vertical stacking inside the subgraph
+            Models ~~~ Routes ~~~ Tests
+        end
 
-`cdd-python-all` supports standard CDD features.
+        Mocks["🎭 API Mocks / Fakes"]
 
-## CLI Options
+        %% Simple dependency flow down the page
+        Outputs --> Core
+        Tests --> Mocks
+    end
 
-```text
-Usage: cdd-python-all [OPTIONS] <COMMAND>
+    style OAS fill:#e3f2fd,stroke:#1e88e5,stroke-width:2px
+    style CDD fill:#f3e5f5,stroke:#8e24aa,stroke-width:2px
+    style Codebase fill:#fafafa,stroke:#9e9e9e,stroke-width:2px,stroke-dasharray: 5 5
+    style Outputs fill:#e8f5e9,stroke:#43a047,stroke-width:2px
+    style Core fill:#fff3e0,stroke:#f57c00,stroke-width:2px
 ```
 
+The CDD lifecycle supports continuous evolution from any starting point:
+1. **Generate**: Scaffold servers, SDKs, or CLIs from a central specification.
+2. **Edit**: Developers write real, unconstrained code directly in the generated files.
+3. **Extract**: Reverse-compile the edited code to produce an updated OpenAPI spec.
+4. **Sync**: Apply new specification changes seamlessly into the existing, hand-edited codebase.
+
+---
+
+## 🌐 The Global Language Ecosystem
+
+Every supported language operates on the same core CDD philosophies but is powered by a dedicated, native compiler tailored to that language's specific AST, idioms, and package management.
+
+All implementations share a standardized CLI interface (`cdd [subcommand]`), acting as a universal toolchain.
+
+| Repository | Language | Client; Client CLI; Server | Extra features | Standards | CI Status |
+|---|---|---|---|---|---|
+| [`cdd-c`](https://github.com/SamuelMarks/cdd-c) | C (C89) | Client; Client CLI; Server | FFI | Swagger 2.0 & OpenAPI 3.2.0 | [![CI](https://github.com/SamuelMarks/cdd-c/actions/workflows/ci.yml/badge.svg)](https://github.com/SamuelMarks/cdd-c/actions/workflows/ci.yml) |
+| [`cdd-cpp`](https://github.com/SamuelMarks/cdd-cpp) | C++ | Client; Client CLI; Server | Upgrades Swagger & Google Discovery to OpenAPI 3.2.0 | Google Discovery; Swagger 2.0 & OpenAPI 3.2.0 | [![CI](https://github.com/SamuelMarks/cdd-cpp/actions/workflows/ci.yml/badge.svg)](https://github.com/SamuelMarks/cdd-cpp/actions/workflows/ci.yml) |
+| [`cdd-csharp`](https://github.com/SamuelMarks/cdd-csharp) | C# | Client; Client CLI; Server | CLR | Swagger 2.0 & OpenAPI 3.2.0 | [![CI](https://github.com/SamuelMarks/cdd-csharp/actions/workflows/ci.yml/badge.svg)](https://github.com/SamuelMarks/cdd-csharp/actions/workflows/ci.yml) |
+| [`cdd-go`](https://github.com/SamuelMarks/cdd-go) | Go | Client; Client CLI; Server | | Swagger 2.0 & OpenAPI 3.2.0 | [![CI](https://github.com/SamuelMarks/cdd-go/actions/workflows/ci.yml/badge.svg)](https://github.com/SamuelMarks/cdd-go/actions/workflows/ci.yml) |
+| [`cdd-java`](https://github.com/SamuelMarks/cdd-java) | Java | Client; Client CLI; Server | | Swagger 2.0 & OpenAPI 3.2.0 | [![CI](https://github.com/SamuelMarks/cdd-java/actions/workflows/ci.yml/badge.svg)](https://github.com/SamuelMarks/cdd-java/actions/workflows/ci.yml) |
+| [`cdd-kotlin`](https://github.com/offscale/cdd-kotlin) | Kotlin (ktor for Multiplatform) | Client; Client CLI; Server | Auto-Admin UI | Swagger 2.0 & OpenAPI 3.2.0 | [![CI](https://github.com/offscale/cdd-kotlin/actions/workflows/ci.yml/badge.svg)](https://github.com/offscale/cdd-kotlin/actions/workflows/ci.yml) |
+| [`cdd-php`](https://github.com/SamuelMarks/cdd-php) | PHP | Client; Client CLI; Server | | Swagger 2.0 & OpenAPI 3.2.0 | [![CI](https://github.com/SamuelMarks/cdd-php/actions/workflows/ci.yml/badge.svg)](https://github.com/SamuelMarks/cdd-php/actions/workflows/ci.yml) |
+| [`cdd-python`](https://github.com/offscale/cdd-python) | Python | N/A (server building blocks) | CLI ↔ SQL ↔ Pydantic ↔ docs ↔ JSON-schema | N/A | [![Linting, testing, coverage, and release](https://github.com/offscale/cdd-python/workflows/Linting,%20testing,%20coverage,%20and%20release/badge.svg)](https://github.com/offscale/cdd-python/actions) |
+| [`cdd-python-all`](https://github.com/offscale/cdd-python-all) | Python | Client; Client CLI; Server |  | Swagger 2.0 & OpenAPI 3.2.0 | [![CI](https://github.com/offscale/cdd-python-client/actions/workflows/ci.yml/badge.svg)](https://github.com/offscale/cdd-python-all/actions/workflows/ci.yml) |
+| [`cdd-ruby`](https://github.com/SamuelMarks/cdd-ruby) | Ruby | Client; Client CLI; Server |  | Swagger 2.0 & OpenAPI 3.2.0 | [![CI](https://github.com/SamuelMarks/cdd-ruby/actions/workflows/ci.yml/badge.svg)](https://github.com/SamuelMarks/cdd-ruby/actions/workflows/ci.yml) |
+| [`cdd-rust`](https://github.com/SamuelMarks/cdd-rust) | Rust | Client; Client CLI; Server |  | Swagger 2.0 & OpenAPI 3.2.0 | [![CI](https://github.com/offscale/cdd-rust/actions/workflows/ci.yml/badge.svg)](https://github.com/offscale/cdd-rust/actions/workflows/ci.yml) |
+| [`cdd-sh`](https://github.com/SamuelMarks/cdd-sh) | Shell (/bin/sh) | Client; Client CLI; Server |  | Swagger 2.0 & OpenAPI 3.2.0 | [![CI](https://github.com/SamuelMarks/cdd-sh/actions/workflows/ci.yml/badge.svg)](https://github.com/SamuelMarks/cdd-sh/actions/workflows/ci.yml) |
+| [`cdd-swift`](https://github.com/SamuelMarks/cdd-swift) | Swift | Client; Client CLI; Server |  | Swagger 2.0 & OpenAPI 3.2.0 | [![CI](https://github.com/SamuelMarks/cdd-swift/actions/workflows/ci.yml/badge.svg)](https://github.com/SamuelMarks/cdd-swift/actions/workflows/ci.yml) |
+| [`cdd-ts`](https://github.com/offscale/cdd-ts) | TypeScript | Client; Client CLI; Server | Auto-Admin UI; Angular; React; Vue; fetch; Axios; Node.js | Swagger 2.0 & OpenAPI 3.2.0 | [![Tests and coverage](https://github.com/offscale/cdd-ts/actions/workflows/ci.yml/badge.svg)](https://github.com/offscale/cdd-ts/actions/workflows/ci.yml) |
+
+---
+
+## 🛠️ Universal CLI Toolchain
+
+A true ecosystem requires standardized tooling. Once a developer learns the CDD toolchain, they can synchronize architecture across the entire polyglot stack.
+
+### Global Arguments
+
+- `--help`: Print help information.
+- `--version`: Print version information.
+- `--input, -i` (or `-f`): Target file, directory, or OpenAPI spec.
+- `--output, -o`: Destination path for generation or sync.
+
+### Core Subcommands
+
+#### `from_openapi to_sdk_cli`
+Generate a client SDK and a corresponding command-line interface (CLI) from an OpenAPI specification.
+- `--input, -i <spec>`: Path to the OpenAPI specification file.
+
+#### `from_openapi to_sdk`
+Generate a client SDK from an OpenAPI specification.
+- `--input, -i <spec>`: Path to the OpenAPI specification file.
+
+#### `from_openapi to_server`
+Generate server boilerplate, models, and routing logic from an OpenAPI specification.
+- `--input, -i <spec>`: Path to the OpenAPI specification file.
+
+#### `to_openapi`
+Parse the existing codebase and extract an authoritative OpenAPI specification.
+- `--input, -i <path>` (or `-f <path>`): Path to the source code directory or file to parse.
+
+#### `to_docs_json`
+Convert an OpenAPI specification into a localized, documentation-optimized JSON format.
+- `--input, -i <spec>`: Path to the OpenAPI specification file.
+- `--no-imports`: Disable import statements in the generated documentation.
+- `--no-wrapping`: Disable line wrapping in the generated documentation.
+
+#### `serve_json_rpc`
+Launch a JSON-RPC server for editor and tool integrations.
+- `--port <port>` (or `-p`): Port to listen on (e.g., `8080`).
+- `--listen <address>` (or `-l`): Address to bind to (e.g., `0.0.0.0`).
+
+#### `mcp`
+Run the Model Context Protocol server via stdio.
+
+#### `sync`
+Sync a directory containing client.py, mock_server.py, test_client.py, cli_main.py
+- `--dir <dir>`: Path to directory containing Python files to sync.
+- `--truth {class,sqlalchemy,function,openapi}`: Designate a single source of truth for synchronization.
+
+### Detail Features Beyond Common Subset
+
+- **Model Context Protocol (MCP)**: Native support to generate an MCP server and adapter (`--mcp`), and run the MCP server over stdio.
+- **Directory Synchronization**: Uniquely features a `sync` command to harmonize `client.py`, `mock_server.py`, `test_client.py`, and `cli_main.py` with configurable sources of truth (`class`, `sqlalchemy`, `function`, `openapi`).
+- **Comprehensive Scaffolding**: Optionally generates GitHub Actions pipelines and installable Python package scaffolding automatically.
+- **Integration Testing & Mocks**: Emits full integration tests and mock servers during generation (`--tests`).
+- **Batch Processing**: Capable of reading from an entire directory of OpenAPI specifications (`--input-dir`) simultaneously.
+
+---
+
+## 🚀 The End of "Spec Drift"
+
+With Compiler Driven Development, specifications and code are no longer loosely coupled artifacts. They are strict, isomorphic reflections of one another, maintained by dedicated standalone compilers.
+
+Choose your language ecosystem above and start treating your architecture as a seamlessly compiled, endlessly editable whole.
+
 ---
 
 ## License