Canonical constraint records for declarative catalogs (vgi.v3)

A declarative Table(constraints=[...]) could not attach: nothing advertised
the constraint read profile, there was no ergonomic way to build a record,
and records DuckDB rejects at ATTACH were served without complaint.

- constraint_record() / ReferencedKey: build a ConstraintInfo from schema
  path, table, id, kind and column paths. enforcement and validation are
  required keywords; the worker attests its own facts.
- ReadOnlyCatalogInterface.catalog_capabilities() advertises
  vgi.constraint.records.read.v1 + vgi.expression.duckdb.scalar.v1 when any
  declared table carries records. The worker already selects against the
  client's offer after catalog_attach, so attach() overrides cannot drop it.
- validate_constraint_records(): the extension's ATTACH rules. Every record
  must be ENFORCED + ALL_ROWS and owned by its table. Arrow nullability must
  equal NOT_NULL/PRIMARY_KEY membership, recursing into ordinary structs, and
  this applies to tables with no records too. Table.__post_init__ runs it, so
  authors' unit tests catch it, and the worker runs it again when serving.
  Messages name the table, constraint id and column path.
- The legacy not_null/unique/check/primary_key/foreign_key fields are now
  rejected at Table construction, with an error naming constraint_record().
  Their dead validation and serialization helpers are removed.
- vgi.catalog re-exports RequiredFilterPaths.
- docs/catalog-interface.md: examples use records, and a Constraints section
  covers the three ATTACH rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Author: rustyconover

docs/catalog-interface.md

@@ -122,9 +122,8 @@ Information about a table:
 | `name` | `str` | Table name |
 | `schema_path` | `SchemaPath` | Parent schema path |
 | `columns` | `SerializedSchema` | Column definitions as Arrow schema bytes |
-| `not_null_constraints` | `list[int]` | Column indices with NOT NULL |
-| `unique_constraints` | `list[list[int]]` | Column index groups for UNIQUE |
-| `check_constraints` | `list[str]` | SQL check expressions |
+| `constraints` | `list[bytes]` | Canonical `ConstraintInfo` records (see [Constraints](#constraints)) |
+| `definition_revision` | `bytes \| None` | Opaque table-definition revision |
 | `comment` | `str \| None` | Optional description |
 | `tags` | `dict[str, str]` | Key-value metadata |
 
@@ -509,7 +508,7 @@ For most use cases, the declarative catalog API provides a simpler way to define
 
 ```python
 from vgi import Worker, TableFunctionGenerator
-from vgi.catalog import Catalog, Schema, Table, View
+from vgi.catalog import Catalog, Schema, Table, View, constraint_record
 ```
 
 ### Function-Backed Tables (Recommended)
@@ -527,7 +526,7 @@ class UsersFunction(TableFunctionGenerator):
     """Generate user data."""
 
     FIXED_SCHEMA: ClassVar[pa.Schema] = pa.schema([
-        ("id", pa.int64()),
+        pa.field("id", pa.int64(), nullable=False),  # NOT NULL: see constraints below
         ("name", pa.string()),
         ("active", pa.bool_()),
     ])
@@ -545,8 +544,16 @@ class UsersFunction(TableFunctionGenerator):
 users_table = Table(
     name="users",
     function=UsersFunction,  # Schema derived from FIXED_SCHEMA
-    not_null=["id"],         # Constraint column names validated
-    unique=[["id"]],
+    constraints=(            # Paths and nullability validated at definition time
+        constraint_record(
+            ["main"], "users", "users.id.not_null", "NOT_NULL", [["id"]],
+            enforcement="ENFORCED", validation="ALL_ROWS",
+        ),
+        constraint_record(
+            ["main"], "users", "users.id.unique", "UNIQUE", [["id"]],
+            enforcement="ENFORCED", validation="ALL_ROWS",
+        ),
+    ),
     comment="User accounts",
 )
 ```
@@ -593,7 +600,7 @@ if __name__ == "__main__":
 | Feature | Description |
 |---------|-------------|
 | **No schema duplication** | Function-backed tables derive schema automatically |
-| **Constraint validation** | `not_null`, `unique` column names validated at definition time |
+| **Constraint validation** | Constraint paths, trust and Arrow nullability validated at definition time |
 | **Automatic scan handling** | Function-backed tables don't need `table_scan_function_get()` |
 | **Type safety** | Frozen dataclasses with runtime validation |
 
@@ -606,16 +613,60 @@ For tables not backed by functions, provide the schema explicitly:
 config_table = Table(
     name="config",
     columns=pa.schema([
-        ("key", pa.string()),
+        pa.field("key", pa.string(), nullable=False),
         ("value", pa.string()),
     ]),
-    not_null=["key"],
-    unique=[["key"]],
+    constraints=(
+        constraint_record(
+            ["main"], "config", "config.key.pk", "PRIMARY_KEY", [["key"]],
+            enforcement="ENFORCED", validation="ALL_ROWS",
+        ),
+    ),
 )
 ```
 
 **Note:** Tables with explicit columns require the worker to implement `table_scan_function_get()` to tell DuckDB how to scan the data.
 
+### Constraints
+
+A table states its constraints as canonical records built with
+`constraint_record()`. Each record is the worker's own attestation — the SDK
+never derives one — so `enforcement` and `validation` are always explicit:
+
+```python
+from vgi.catalog import ReferencedKey, constraint_record
+
+constraint_record(
+    ["main"], "orders", "orders.customer.fk", "FOREIGN_KEY", [["customer_id"]],
+    enforcement="ENFORCED", validation="ALL_ROWS",
+    references=ReferencedKey(
+        schema_path=["main"], table="customers", columns=[["id"]], constraint_id="customers.pk",
+    ),
+)
+constraint_record(
+    ["main"], "orders", "orders.amount.check", "CHECK", [],
+    enforcement="ENFORCED", validation="ALL_ROWS", check_sql="amount >= 0",
+)
+```
+
+DuckDB checks three things when it attaches the catalog, and `Table` checks the
+same at definition time (the worker checks again when serving each table):
+
+- It reads only `ENFORCED` + `ALL_ROWS` records; state one only when every row
+  the table can ever return satisfies it.
+- Every Arrow field — descending through ordinary structs, but not into
+  extension or logical-type storage — is non-nullable exactly when a
+  `NOT_NULL` or `PRIMARY_KEY` record names its path. This applies to tables
+  with no constraints too: a non-nullable field there is an error.
+- The records are readable only under the negotiated
+  `vgi.constraint.records.read.v1` profile (plus `vgi.expression.duckdb.scalar.v1`
+  for CHECK). `ReadOnlyCatalogInterface` advertises both automatically when any
+  declared table carries records; a catalog that serves tables some other way
+  overrides `catalog_capabilities()`.
+
+The legacy `not_null`, `unique`, `check`, `primary_key` and `foreign_key`
+fields are rejected; replace them with records.
+
 ### Validation
 
 Declarative catalogs include comprehensive validation:
@@ -627,10 +678,21 @@ Table(name="bad")  # ValueError: must specify either 'columns' or 'function'
 # Error: invalid constraint column
 Table(
     name="users",
-    columns=pa.schema([("id", pa.int64())]),
-    not_null=["nonexistent"],  # ValueError: column 'nonexistent' not found
+    columns=pa.schema([pa.field("id", pa.int64(), nullable=False)]),
+    constraints=(
+        constraint_record(
+            ["main"], "users", "nn", "NOT_NULL", [["nonexistent"]],
+            enforcement="ENFORCED", validation="ALL_ROWS",
+        ),  # ValueError: column path ['nonexistent'] is missing
+    ),
 )
 
+# Error: Arrow nullability must match NOT_NULL/PRIMARY_KEY records exactly
+Table(
+    name="users",
+    columns=pa.schema([pa.field("id", pa.int64(), nullable=False)]),
+)  # ValueError: ... the field is non-nullable but no ... record names it
+
 # Error: default_schema not in schemas
 Catalog(
     name="myapp",

tests/catalog/test_constraint_public_contract.py

@@ -170,16 +170,29 @@ def test_discovery_never_hides_constraints_and_mutation_requires_revision() -> N
         Worker._table_for_capabilities(table_info(definition_revision=None), PROFILES)
 
 
-@pytest.mark.parametrize("trusted", [False, True])
-def test_discovery_rejects_nullability_that_does_not_match_trust(trusted: bool) -> None:
-    info = INFO if trusted else replace(INFO, enforcement=ConstraintEnforcement.METADATA_ONLY)
-    wrong = pa.schema([pa.field("a.b", pa.int64(), nullable=trusted)])
+def test_discovery_rejects_nullability_that_does_not_match_records() -> None:
+    wrong = pa.schema([pa.field("a.b", pa.int64(), nullable=True)])
     with pytest.raises(ValueError, match="nullability"):
         Worker._table_for_capabilities(
-            table_info(columns=wrong.serialize().to_pybytes(), constraints=[info.serialize_to_bytes()]), PROFILES
+            table_info(columns=wrong.serialize().to_pybytes(), constraints=[INFO.serialize_to_bytes()]), PROFILES
         )
 
 
+@pytest.mark.parametrize(
+    "state",
+    [
+        {"enforcement": ConstraintEnforcement.METADATA_ONLY},
+        {"validation": ConstraintValidation.FUTURE_WRITES},
+        {"validation": ConstraintValidation.UNVALIDATED},
+    ],
+)
+def test_discovery_rejects_records_duckdb_will_not_read(state: dict[str, Any]) -> None:
+    """The extension fails ATTACH unless every record is ENFORCED + ALL_ROWS."""
+    info = replace(INFO, **state)
+    with pytest.raises(ValueError, match="accepts only ENFORCED \\+ ALL_ROWS"):
+        Worker._table_for_capabilities(table_info(constraints=[info.serialize_to_bytes()]), PROFILES)
+
+
 def test_create_checks_negotiation_and_current_support_before_provider(monkeypatch: pytest.MonkeyPatch) -> None:
     for selected, available in [(CatalogCapabilities(), PROFILES), (PROFILES, CatalogCapabilities())]:
         instance, _, calls = worker(monkeypatch, selected, available)