codegen: generate the protocol wire names, and drift-guard them

The names were hand-written on both sides of the C++ boundary, and a rename
proved why they should not be. A version bump that misses the C++ tree is a red
build, because the versions are generated and drift-guarded. A rename that
missed it was a silent misroute instead: every request the DuckDB extension
sent named a protocol the server no longer hosted, and nothing failed until
integration. Same wire contract, two spellings, two very different costs.

`vgi.codegen.cpp_protocol_name` emits both names into one header,
`vgi_protocol_names.hpp`. One artifact rather than one per protocol, unlike the
versions: these are not two independent facts but the wire-name contract,
decided together and — as dc6f82d shows — changed together. One artifact means
one regen entry, one drift test, and no way to update one name while leaving the
other stale.

The name is resolved through vgi-rpc's own `_protocol_wire_name` rather than
reimplemented. Its rule has a subtlety worth not duplicating: it reads
`vars(protocol)`, not `getattr`, so a Protocol that subclasses another and
forgets to redeclare gets its own class name rather than silently inheriting its
parent's routing key. A generator that reimplemented that with `getattr` would
disagree with the dispatcher it exists to serve — precisely the drift it is for.
The name is also run through vgi-rpc's `validate_protocol_name` (without
`allow_reserved`, since nothing here may claim the framework's `vgi_rpc.`
prefix), so an unroutable name fails at generation rather than becoming a
literal every client sends and every server rejects.

The drift test asserts against `_protocol_wire_name`, not against
`Protocol.protocol_name` — a test written against `getattr` would pass while the
client addressed a protocol the server does not host. All three guards were
mutation-checked rather than assumed: a doctored header fails the checked-in
comparison, an upstream rename with a stale header fails it too, and a
reserved-prefix name aborts generation.

Two fixes found while in here:

`scripts/regen_generated.py` never listed the secret protocol's three artifacts,
so `--check` answered "no drift" over a stale secret header — false confidence,
which is worse than no script. Listed now; all three were in fact current.

`cpp_secret_protocol_version.emit()` parsed `--namespace` and then dropped it on
the floor, never forwarding it to `emit_version_header`. Harmless for the
default namespace and silently wrong for the standalone-SDK case the flag exists
to serve.

Suite: 2724 passed, 103 skipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: rustyconover

scripts/regen_generated.py

@@ -34,6 +34,22 @@
     ("vgi.codegen.cpp_request_builders", "src/generated/vgi_request_builders.hpp", "vgi"),
     ("vgi.codegen.cpp_constants", "src/generated/vgi_protocol_constants.hpp", "vgi"),
     ("vgi.codegen.cpp_protocol_version", "src/generated/vgi_protocol_version.hpp", "vgi"),
+    ("vgi.codegen.cpp_protocol_name", "src/generated/vgi_protocol_names.hpp", "vgi"),
+    # The secret protocol's three artifacts were checked in but never listed here,
+    # so `--check` reported "no drift" over a stale secret header. Listed now: a
+    # regen script that silently skips an artifact is worse than no script, because
+    # it answers the drift question with false confidence.
+    (
+        "vgi.codegen.cpp_secret_protocol_version",
+        "src/generated/vgi_secret_protocol_version.hpp",
+        "vgi",
+    ),
+    ("vgi.codegen.cpp_secret_schemas", "src/generated/vgi_secret_protocol_schemas.hpp", "vgi"),
+    (
+        "vgi.codegen.cpp_secret_request_builders",
+        "src/generated/vgi_secret_request_builders.hpp",
+        "vgi",
+    ),
     ("vgi.codegen.go_schemas", "vgi/generated/protocol_schemas.go", "vgi-go"),
     ("vgi.codegen.rust_schemas", "vgi-protocol/src/generated/protocol_schemas.rs", "vgi-rust"),
     ("vgi.codegen.rust_request_builders", "vgi-protocol/src/generated/request_params.rs", "vgi-rust"),

tests/test_generated_cpp_protocol_name.py

@@ -0,0 +1,135 @@
+# Copyright 2025, 2026 Query Farm LLC - https://query.farm
+
+"""Drift test for ``vgi.codegen.cpp_protocol_name``.
+
+This test exists because its absence already cost a wire break. The protocol
+*versions* are generated and drift-guarded, so a version bump that misses the
+C++ tree is a red build. The protocol *names* were hand-written on both sides,
+so a rename was a silent misroute instead: every request the DuckDB extension
+sent named a protocol the server no longer hosted, and nothing failed until
+integration.
+
+So the assertions below are not "the file looks plausible". They are, in order:
+the generator agrees with the dispatcher about what the name IS, the name is
+routable, and the checked-in header says the same thing.
+
+Skipped when the ``vgi`` repo isn't checked out next to ``vgi-python``; the
+other tests still run.
+"""
+
+from __future__ import annotations
+
+import io
+import os
+import re
+from pathlib import Path
+
+import pytest
+from vgi_rpc.rpc._types import _protocol_wire_name, validate_protocol_name
+
+from vgi.codegen.cpp_protocol_name import (
+    current_protocol_name,
+    current_secret_protocol_name,
+    emit,
+)
+from vgi.protocol import VgiProtocol
+from vgi.secret_protocol import VgiSecretProtocol
+
+_CONSTANTS = {
+    "VGI_PROTOCOL_NAME": (VgiProtocol, current_protocol_name),
+    "VGI_SECRET_PROTOCOL_NAME": (VgiSecretProtocol, current_secret_protocol_name),
+}
+
+
+def _vgi_generated_path() -> Path:
+    """Locate ``vgi/src/generated/vgi_protocol_names.hpp``."""
+    override = os.environ.get("VGI_GENERATED_PROTOCOL_NAMES_HPP")
+    if override:
+        return Path(override)
+    return Path(__file__).resolve().parents[2] / "vgi" / "src" / "generated" / "vgi_protocol_names.hpp"
+
+
+_REGEN_HINT = (
+    "To regenerate, run:\n"
+    "  uv run --project ~/Development/vgi-python python scripts/regen_generated.py\n"
+    "\n"
+    "Do not redirect a generator into its destination with `>`: the shell\n"
+    "truncates the file before the generator runs, so any failure destroys\n"
+    "the checked-in artifact. The script renders to memory first."
+)
+
+
+def _emitted() -> str:
+    out = io.StringIO()
+    emit(out)
+    return out.getvalue()
+
+
+def _emitted_literal(constant: str) -> str:
+    match = re.search(rf'inline constexpr std::string_view {constant} = "([^"]*)";', _emitted())
+    assert match is not None, f"generator did not emit {constant}"
+    return match.group(1)
+
+
+def test_generator_is_deterministic() -> None:
+    """Running the generator twice produces byte-identical output."""
+    assert _emitted() == _emitted()
+
+
+@pytest.mark.parametrize("constant", sorted(_CONSTANTS))
+def test_emitted_value_matches_the_dispatcher(constant: str) -> None:
+    """The emitted literal is what vgi-rpc will actually route on.
+
+    Compared against ``_protocol_wire_name`` — the function the server itself
+    uses — rather than against ``Protocol.protocol_name``. Those differ in a way
+    that matters: the wire name is read from ``vars(protocol)``, so a Protocol
+    that subclasses another and forgets to redeclare gets its own class name,
+    NOT its parent's routing key. A test written against ``getattr`` would pass
+    while the client addressed a protocol the server does not host.
+    """
+    protocol, _accessor = _CONSTANTS[constant]
+    assert _emitted_literal(constant) == _protocol_wire_name(protocol)
+
+
+@pytest.mark.parametrize("constant", sorted(_CONSTANTS))
+def test_emitted_value_is_routable(constant: str) -> None:
+    """A name the server would refuse to route must never become a client constant."""
+    validate_protocol_name(_emitted_literal(constant))
+
+
+def test_the_two_protocols_do_not_share_a_routing_key() -> None:
+    """Distinct protocols must be distinctly addressable.
+
+    They are co-hostable by construction, and dispatch resolves (protocol,
+    method). Were both to answer to one key, a co-hosting server would route
+    ``secret_lookup`` by whichever binding registered last.
+    """
+    assert current_protocol_name() != current_secret_protocol_name()
+
+
+def test_checked_in_generated_hpp_matches_generator() -> None:
+    """The .hpp checked into the vgi repo must match the current generator output."""
+    path = _vgi_generated_path()
+    if not path.exists():
+        pytest.skip(
+            f"{path} not found; set VGI_GENERATED_PROTOCOL_NAMES_HPP or check out the vgi repo next to vgi-python"
+        )
+
+    checked_in = path.read_text()
+    expected = _emitted()
+
+    if checked_in != expected:
+        for i, (actual_ch, expected_ch) in enumerate(zip(checked_in, expected, strict=False)):
+            if actual_ch != expected_ch:
+                window_start = max(0, i - 40)
+                window_end = i + 40
+                raise AssertionError(
+                    f"checked-in {path} differs from generator output at offset {i}.\n"
+                    f"  checked-in: {checked_in[window_start:window_end]!r}\n"
+                    f"  expected:   {expected[window_start:window_end]!r}\n"
+                    f"{_REGEN_HINT}"
+                )
+        raise AssertionError(
+            f"checked-in {path} is shorter/longer than generator output "
+            f"({len(checked_in)} vs {len(expected)} chars)\n{_REGEN_HINT}"
+        )

vgi/codegen/cpp_protocol_name.py

@@ -0,0 +1,197 @@
+# Copyright 2025, 2026 Query Farm LLC - https://query.farm
+
+r"""Emit ``vgi_protocol_names.hpp`` — the C++ side's view of the wire routing keys.
+
+Third sibling of :mod:`vgi.codegen.cpp_constants` (byte keys from
+``vgi_rpc.metadata``) and :mod:`vgi.codegen.cpp_protocol_version` (the surface
+versions). This one emits the ``vgi_rpc.protocol`` *values*: the names a client
+must send to reach each protocol this repo declares.
+
+Why this generator exists
+-------------------------
+
+It was added after a real break, and the break is the specification. Neither
+Protocol declared a ``protocol_name``, so every implementation's wire name
+defaulted to whatever its local class happened to be called — six
+implementations, four different answers. That was invisible until the
+transports made ``vgi_rpc.protocol`` a required routing key, at which point
+there was no single string the DuckDB extension could send.
+
+The names were then decided (``vgi.v2``, ``vgi.secret.v1``) and hand-written on
+both sides. That is the same shape of hazard one step removed: the version is
+generated *and* drift-guarded, so a version bump that misses the C++ tree is a
+red build, while a rename that missed it was a silent misroute. A wire contract
+that two repos spell independently will eventually be spelled differently.
+
+Both names live in ONE header, unlike the versions, which have one file each.
+They are not two independent facts: they are the wire-name contract, decided
+together and — as the commit that introduced them shows — changed together. One
+artifact means one regen entry and one drift test, and makes it impossible to
+update one name while leaving the other stale.
+
+The name is read through vgi-rpc's own ``_protocol_wire_name`` rather than
+reimplementing its rule here. The rule has a subtlety worth not duplicating:
+it reads ``vars(protocol)`` rather than ``getattr``, so a Protocol that
+subclasses another and forgets to redeclare does NOT silently inherit its
+parent's routing key. Reimplementing that with ``getattr`` would make this
+generator disagree with the dispatcher it exists to serve — which is precisely
+the class of drift the generator is for.
+
+Workflow:
+
+    uv run --project ~/Development/vgi-python python scripts/regen_generated.py
+
+``tests/test_generated_cpp_protocol_name.py`` enforces drift detection at PR time.
+"""
+
+from __future__ import annotations
+
+import argparse
+import io
+import sys
+from typing import TYPE_CHECKING
+
+from vgi_rpc.rpc._types import (
+    _protocol_wire_name,
+    validate_protocol_name,
+)
+
+from vgi.codegen._common import (
+    DEFAULT_CPP_NAMESPACE,
+    GeneratorError,
+    close_namespace,
+    open_namespace,
+    parse_cpp_namespace,
+    provenance_comment,
+)
+from vgi.protocol import VgiProtocol
+from vgi.secret_protocol import VgiSecretProtocol
+
+if TYPE_CHECKING:
+    from typing import TextIO
+
+
+GENERATOR_VERSION = "1"
+
+
+def current_protocol_name() -> str:
+    """Return the wire routing key for :class:`vgi.protocol.VgiProtocol`."""
+    return _wire_name(VgiProtocol)
+
+
+def current_secret_protocol_name() -> str:
+    """Return the wire routing key for :class:`vgi.secret_protocol.VgiSecretProtocol`."""
+    return _wire_name(VgiSecretProtocol)
+
+
+def _wire_name(protocol: type) -> str:
+    """Resolve and validate one Protocol's wire name.
+
+    Resolution is delegated to vgi-rpc so this never disagrees with the
+    dispatcher. Validation is vgi-rpc's too, and runs here so a name that the
+    server would refuse to route fails at generation rather than becoming a
+    string literal that every client dutifully sends and every server rejects.
+
+    ``allow_reserved`` is deliberately NOT set: the ``vgi_rpc.`` prefix belongs
+    to protocols the framework itself defines, and nothing this repo declares
+    may claim it.
+    """
+    name = _protocol_wire_name(protocol)
+    try:
+        validate_protocol_name(name)
+    except ValueError as exc:
+        raise GeneratorError(f"{protocol.__name__} has an unroutable wire name: {exc}") from exc
+
+    # The name is emitted into a C++ string literal AND into an HTTP URL path
+    # segment. validate_protocol_name's charset already excludes everything that
+    # would need escaping in either; this asserts that rather than assuming it,
+    # so a future loosening upstream surfaces here instead of as a malformed
+    # header or a percent-encoding mismatch at some proxy.
+    if not all(0x20 <= ord(c) < 0x7F for c in name):
+        raise GeneratorError(f"non-printable byte in protocol_name {name!r}; this is a bug")
+    return name
+
+
+def emit(out: TextIO, namespace: list[str] | None = None) -> None:
+    """Emit ``vgi_protocol_names.hpp`` to *out*."""
+    if namespace is None:
+        namespace = parse_cpp_namespace(DEFAULT_CPP_NAMESPACE)
+
+    entries = [
+        (
+            "VGI_PROTOCOL_NAME",
+            current_protocol_name(),
+            "VgiProtocol",
+            "the worker/catalog protocol: bind, init, catalog_*, aggregates, table_buffering_*",
+        ),
+        (
+            "VGI_SECRET_PROTOCOL_NAME",
+            current_secret_protocol_name(),
+            "VgiSecretProtocol",
+            "the standalone secret service: the single unary secret_lookup",
+        ),
+    ]
+
+    body = io.StringIO()
+    body.write("#pragma once\n\n")
+    body.write("#include <string_view>\n\n")
+    body.write(open_namespace(namespace))
+    body.write("\n")
+    body.write("// Wire routing keys. A vgi-rpc server dispatches on the pair\n")
+    body.write("// (protocol, method): one server may co-host several protocols and their\n")
+    body.write("// method names may collide, so every request must name the protocol it\n")
+    body.write("// addresses under `vgi_rpc.protocol`. On the raw transports that metadata\n")
+    body.write("// key is the only carrier; over HTTP the same value is also the protocol\n")
+    body.write("// path segment, and the server rejects a request whose two disagree.\n")
+    body.write("//\n")
+    body.write("// The major version is part of the name, so an incompatible major is a\n")
+    body.write("// DIFFERENT protocol and an unroutable request 404s -- an answer any proxy\n")
+    body.write("// understands without an Arrow parser.\n")
+    for constant_name, value, source, purpose in entries:
+        body.write("\n")
+        body.write(f"// {source} -- {purpose}.\n")
+        body.write(f"// Sourced from {source}.protocol_name (vgi-python).\n")
+        body.write(f'inline constexpr std::string_view {constant_name} = "{value}";\n')
+    body.write("\n")
+    body.write(close_namespace(namespace))
+
+    out.write("// ============================================================================\n")
+    out.write(
+        provenance_comment(
+            generator_module="vgi.codegen.cpp_protocol_name",
+            generator_command="python -m vgi.codegen.cpp_protocol_name",
+            generator_version=GENERATOR_VERSION,
+            regen_command_lines=[
+                "uv run --project ~/Development/vgi-python python scripts/regen_generated.py",
+            ],
+            body=body.getvalue(),
+        )
+    )
+    out.write("// ============================================================================\n")
+    out.write("\n")
+    out.write(body.getvalue())
+
+
+def main() -> None:
+    """Console-script entry point."""
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--namespace",
+        default=DEFAULT_CPP_NAMESPACE,
+        help=(
+            "C++ namespace to emit into, `::`-separated "
+            f"(default: {DEFAULT_CPP_NAMESPACE}). VGI is not DuckDB-only: a "
+            "standalone worker SDK wants something like `vgi::generated`."
+        ),
+    )
+    args = parser.parse_args()
+    try:
+        namespace = parse_cpp_namespace(args.namespace)
+    except GeneratorError as e:
+        print(f"\nerror: {e}\n", file=sys.stderr)
+        sys.exit(2)
+    emit(sys.stdout, namespace)
+
+
+if __name__ == "__main__":
+    main()

vgi/codegen/cpp_secret_protocol_version.py

@@ -58,6 +58,10 @@ def emit(out: TextIO, namespace: list[str] | None = None) -> None:
             "uv run --project ~/Development/vgi-python python -m vgi.codegen.cpp_secret_protocol_version \\",
             "  > ~/Development/vgi/src/generated/vgi_secret_protocol_version.hpp",
         ],
+        # Was dropped on the floor: `main` parsed --namespace, passed it to
+        # emit(), and emit() never forwarded it. Harmless for the default, and
+        # silently wrong for the standalone-SDK case the flag exists to serve.
+        namespace=namespace,
     )