fix(ci): close core runtime security gates

Author: JuliaEdom

Dockerfile.api

@@ -49,6 +49,7 @@ RUN pip install --no-cache-dir --upgrade pip \
     && wheel="$(find /tmp/dist -name '*.whl' -print -quit)" \
     && pip install --no-cache-dir --no-deps "${wheel}" \
     && pip check \
+    && python -m pip uninstall --yes pip setuptools wheel \
     && rm -rf /tmp/dist /tmp/requirements-docker.lock
 
 # Non-root runtime (audit P1-4): this image ran as root by default. Helm

docs/PROJECT_CLOSURE.md

@@ -57,14 +57,29 @@ rehearsal and third-party penetration test.
 
 ## Обязательные внешние closure gates
 
-- local closing commits published to `main`;
+- closing commits `59fc19c` (Operator/Kafka reproducibility) and `72b9609`
+  (Python compatibility/core lock remediation) published to `main`; the final
+  core-only/Ruff/runtime-image hardening commit remains to be published;
 - all required GitHub checks green on the exact closing SHA;
 - two open Dependabot PRs receive a final one-by-one disposition;
 - the substantial post-v2 changes receive a final release/version decision and
   published supply-chain evidence if released.
 
 Push, PR mutations and release actions require explicit owner authorization.
 
+## Текущий closing handoff — 2026-07-30
+
+- Python compatibility 3.11/3.12/3.13 is green on pushed SHA `72b9609`.
+- The remaining main-CI failures on that SHA (Ruff 0.16 SDK Markdown and
+  core-only API import without `pyiceberg`) are fixed and locally verified.
+- The Security Scan findings were traced to packages vendored by runtime
+  `pip`; the final image now removes installer tooling after `pip check`.
+- Independent Mac image validation reports zero HIGH/CRITICAL findings with
+  Trivy 0.70.0; see
+  [security-runtime-image-trivy-2026-07-30.md](security-runtime-image-trivy-2026-07-30.md).
+- Production status remains `candidate`; none of this closes the live Iceberg,
+  restore/replay, fresh soak/rollback, external pen-test, or npm approval gates.
+
 ## Сохранённые локальные артефакты
 
 Root `AGENTS.md` and `plan_sol_23_07_26` remain untracked and unchanged. They do

docs/STATUS.md

@@ -2,6 +2,7 @@
 
 > Updated: **2026-07-30** (clean-checkout PyFlink OCI build + submission smoke
 > **PASS**; Flink Kubernetes Operator + Helm golden-topology deploy **PASS**;
+> core-only API and hardened runtime-image remediation locally **PASS**;
 > golden topology remains a production candidate, not production accepted) ·
 > release line **`v2.0.0`**. Numbers below come only from measured, in-repo
 > evidence — see the linked reports for methodology and reproduction commands.
@@ -21,6 +22,17 @@ and external security acceptance remain open. The earlier 4 h @ 100 eps result
 remains valid evidence for its measured pre-materializer path, not for the
 still-unaccepted full golden topology.
 
+Closing CI hardening on 2026-07-30 also verified the Python 3.11/3.12/3.13
+compatibility lanes, made `pyiceberg` optional for the core-only API path,
+aligned SDK examples with Ruff 0.16 Markdown formatting, and removed
+`pip`/`setuptools`/`wheel` from the final API image after `pip check`.
+A clean core-only HTTP smoke returned 200 for health, entity, and NL query.
+An independent Mac rebuild scanned with Trivy 0.70.0 at zero HIGH/CRITICAL
+findings; see
+[security-runtime-image-trivy-2026-07-30.md](security-runtime-image-trivy-2026-07-30.md).
+The exact post-remediation GitHub SHA must still complete all required checks
+before the external closure gate is claimed.
+
 **Project lifecycle:** closure candidate. Engineering scope is frozen; the
 production-candidate boundary, future acceptance program, and release gates are
 recorded in [PROJECT_CLOSURE.md](PROJECT_CLOSURE.md).
@@ -40,6 +52,7 @@ recorded in [PROJECT_CLOSURE.md](PROJECT_CLOSURE.md).
 | Multi-tenant ClickHouse write key | adversarial two-tenant suite green on live CH 25.3 (CI `test-integration` + audit stand) | [security-audit.md](security-audit.md), `tests/integration/test_clickhouse_tenant_isolation_live.py` |
 | Clean-checkout PyFlink OCI build + submission smoke | PASS clean-checkout OCI build + submission smoke on 2026-07-30 — image built, JobID `RUNNING`, not Operator/E2E | [perf/golden-flink-submission-2026-07-30.md](perf/golden-flink-submission-2026-07-30.md) |
 | Flink Kubernetes Operator + Helm golden deploy | PASS clean kind Operator/Helm deploy of verified image on exact HEAD `36ed1ec` — CR/job stable, checkpoints `2→23`, leader flaps `0`, not lake E2E | [perf/golden-operator-acceptance-2026-07-30.md](perf/golden-operator-acceptance-2026-07-30.md) |
+| Hardened API runtime image (local acceptance) | core-only API import/HTTP smoke PASS; Trivy 0.70.0 reports 0 HIGH/CRITICAL after runtime installer removal | [security-runtime-image-trivy-2026-07-30.md](security-runtime-image-trivy-2026-07-30.md) |
 
 ## 2026-07-23 audit closure
 

docs/release-readiness.md

@@ -83,6 +83,15 @@ on the 3.11/3.12/3.13 `python-compat` matrix, so a failed compatibility lane
 cannot be bypassed merely because its job name is not a separate protected
 context.
 
+The 2026-07-30 closing remediation additionally covers two clean-install
+boundaries: the core-only API imports and serves health/entity/query without
+the optional `pyiceberg` package, and the final API image removes
+`pip`/`setuptools`/`wheel` after its hash-locked install and `pip check`.
+Independent Mac validation with Trivy 0.70.0 reported zero HIGH/CRITICAL
+findings; details and evidence limits are in
+[security-runtime-image-trivy-2026-07-30.md](security-runtime-image-trivy-2026-07-30.md).
+All required GitHub checks must still be green on the exact final pushed SHA.
+
 ## Scope
 
 Component, contract, Helm, and replay tests validate the checked-in streaming,

docs/security-audit.md

@@ -154,11 +154,21 @@ The repository includes a dedicated security workflow in GitHub Actions:
 - Safety for dependency vulnerability scanning
 - Trivy for container image scanning and CycloneDX SBOM generation
 
+On 2026-07-30, a Trivy scan of the API image identified vulnerable packages
+vendored by runtime `pip`, not dependencies from the application lock. The
+final stage now removes `pip`, `setuptools`, and `wheel` after the hash-locked
+install and `pip check`. An independent Mac rebuild and Trivy `0.70.0` scan
+reported zero HIGH/CRITICAL findings while the API import remained healthy.
+See
+[security-runtime-image-trivy-2026-07-30.md](security-runtime-image-trivy-2026-07-30.md).
+
 The Bandit baseline currently records a historical `B310` finding in `src/serving/backends/clickhouse_backend.py`. SQL construction findings are not globally suppressed; reviewed identifier construction is handled through narrow suppressions and tests.
 
 Helm defaults no longer embed production-shaped API-key verifier hashes. Operators can render a chart-managed Secret for local use or mount an existing Kubernetes Secret, which is friendlier to External Secrets Operator, Sealed Secrets, or equivalent workflows.
 
-Evidence: `.github/workflows/security.yml`, `.bandit`, `.bandit-baseline.json`, `docs/helm-deployment.md`
+Evidence: `.github/workflows/security.yml`, `.bandit`, `.bandit-baseline.json`,
+`docs/helm-deployment.md`,
+`docs/security-runtime-image-trivy-2026-07-30.md`
 
 ## 9. Operational Security and Auditability
 

docs/security-runtime-image-trivy-2026-07-30.md

@@ -0,0 +1,74 @@
+# Runtime image Trivy remediation — 2026-07-30
+
+## Scope
+
+This record covers the API runtime image only. It does not claim a signed
+registry release, an external penetration test, or production acceptance.
+
+## Trigger
+
+GitHub Actions Security Scan run
+[`30570556301`](https://github.com/brownjuly2003-code/agentflow/actions/runs/30570556301)
+scanned commit `72b960930302ecbced46c89479a5b09082f0a406` with Trivy `0.70.0`.
+The image job reported:
+
+- `GHSA-6v7p-g79w-8964`: vendored `msgpack 1.1.2`;
+- `CVE-2025-47273`: stale `setuptools 70.3.0`.
+
+The uploaded CycloneDX SBOM associated both findings with the final installer
+layer and no application package path. The same layer listed other packages
+vendored by `pip`, while the actual installed setuptools distribution was the
+already-fixed `82.0.1`. Neither vulnerable version came from
+`requirements-docker.lock`.
+
+## Remediation
+
+`Dockerfile.api` still performs the hash-locked dependency install, installs
+the AgentFlow wheel with `--no-deps`, and runs `pip check`. It then removes
+runtime-only installer tooling:
+
+```dockerfile
+python -m pip uninstall --yes pip setuptools wheel
+```
+
+The builder stage is unchanged. A unit contract requires the uninstall to
+remain in the final stage and after `pip check`.
+
+## Independent Mac validation
+
+Validation used an isolated clone at
+`/tmp/agentflow-security-72b9609-codex-01`, based on exact commit
+`72b960930302ecbced46c89479a5b09082f0a406`, plus the candidate
+image-affecting files:
+
+- `Dockerfile.api` blob `9cc33d81a88802991f632225a45f9422ab8e4358`;
+- `src/quality/monitors/metrics_collector.py` blob
+  `475c656b0c7de9418e2447bd6b5cca36b9026146`.
+
+The image was built from the pinned `python:3.11-slim` digest already declared
+in `Dockerfile.api`:
+
+```text
+agentflow-api:security-codex-72b9609
+manifest list sha256:934357fe14e8f6ed3a712eb7ae8bec1a369aad06f54691eadd2b8aa7e2cd1591
+```
+
+Build output confirmed that `pip 26.2`, `setuptools 82.0.1`, and `wheel 0.47.0`
+were uninstalled only after `pip check` returned
+`No broken requirements found`.
+
+Runtime checks:
+
+- `python -m pip --version` failed with `No module named pip` as intended;
+- `from src.serving.api.main import app` succeeded (`AgentFlow Query API`);
+- Trivy `0.70.0`, with `HIGH,CRITICAL`, `--ignore-unfixed`, and
+  `--exit-code 1`, exited `0`;
+- Trivy summary: Debian `13.5` vulnerabilities `0`; every detected Python
+  package vulnerabilities `0`.
+
+## Evidence boundary
+
+This is content-level local acceptance of the candidate runtime image. The
+next pushed SHA must still pass the repository's GitHub Security Scan and all
+other required checks. Published-image signing and external penetration-test
+evidence remain separate gates.

sdk/README.md

@@ -22,6 +22,7 @@ python -m pip install -e "./sdk"
 
 ```python
 from agentflow import AgentFlowClient
+
 client = AgentFlowClient("http://localhost:8000", api_key="dev-key")
 order = client.get_order("ORD-20260404-1001")
 print(order.status, client.get_metric("revenue", "24h").value)
@@ -38,6 +39,7 @@ client.configure_resilience(retry_policy=RetryPolicy(max_attempts=5))
 ```python
 from agentflow import AsyncAgentFlowClient
 
+
 async def main() -> None:
     async with AsyncAgentFlowClient("http://localhost:8000", api_key="dev-key") as client:
         order = await client.get_order("ORD-20260404-1001")

src/quality/monitors/metrics_collector.py

@@ -18,7 +18,6 @@
 import yaml
 from confluent_kafka import KafkaException
 from prometheus_client import Gauge
-from pyiceberg.exceptions import NoSuchPropertyException, RESTError, ValidationError
 
 from src.serving.backends import BackendExecutionError
 
@@ -379,6 +378,25 @@ def _check_iceberg(self) -> ComponentHealth:
                 source=CheckSource.PLACEHOLDER,
             )
 
+        # Optional cloud extra: only import pyiceberg when a config file exists.
+        # Local/core installs can construct HealthCollector(include_external=False)
+        # without requiring the cloud dependency at module import time.
+        try:
+            from pyiceberg.exceptions import (
+                NoSuchPropertyException,
+                RESTError,
+                ValidationError,
+            )
+        except ImportError as exc:
+            return ComponentHealth(
+                name="iceberg",
+                status=HealthStatus.DEGRADED,
+                message=f"Iceberg unavailable: {exc}",
+                last_check=datetime.now(UTC),
+                metrics={"row_counts": {}},
+                source=CheckSource.PLACEHOLDER,
+            )
+
         try:
             from src.processing.iceberg_sink import IcebergSink
 

tests/unit/test_contract_dependencies.py

@@ -207,6 +207,15 @@ def test_docker_build_contexts_prepare_root_package_metadata():
             assert "--require-hashes -r /tmp/requirements-docker.lock" in text
             assert 'pip install --no-cache-dir --no-deps "${wheel}"' in text
             assert "pip check" in text
+            # Runtime image must not ship installer tooling: Trivy SBOM tied
+            # GHSA-6v7p-g79w-8964 (msgpack via pip vendor) and CVE-2025-47273
+            # (setuptools) to the final-layer pip/setuptools/wheel tree, not
+            # to requirements-docker.lock. Uninstall only after installs and
+            # pip check so the hash-locked environment is still verified.
+            final_stage = text.rsplit("FROM ", maxsplit=1)[-1]
+            uninstall = "python -m pip uninstall --yes pip setuptools wheel"
+            assert uninstall in final_stage
+            assert final_stage.index("pip check") < final_stage.index(uninstall)
             # Q0.2 / S9: scale profile needs the Postgres control-plane driver
             # baked in (kind 2-pod previously required a docker-commit psycopg
             # hack). The wheel installs --no-deps, so the guarantee now lives

tests/unit/test_local_only_mode.py

@@ -2,6 +2,10 @@
 
 from __future__ import annotations
 
+import os
+import subprocess
+import sys
+import tempfile
 from pathlib import Path
 
 import pytest
@@ -14,6 +18,8 @@
 from src.serving.api.main import app
 from src.serving.cache import QueryCache
 
+PROJECT_ROOT = Path(__file__).resolve().parents[2]
+
 
 def test_local_health_collector_keeps_only_embedded_checks() -> None:
     collector = HealthCollector(include_external=False)
@@ -25,6 +31,66 @@ def test_local_health_collector_keeps_only_embedded_checks() -> None:
     ]
 
 
+def test_metrics_collector_imports_without_pyiceberg() -> None:
+    """Core/local mode must import HealthCollector when pyiceberg is absent."""
+    script = """
+import os
+import sys
+
+# Force pyiceberg absent even if the host env has the cloud extra installed.
+sys.modules["pyiceberg"] = None
+sys.modules["pyiceberg.exceptions"] = None
+
+from src.quality.monitors.metrics_collector import (
+    CheckSource,
+    HealthCollector,
+    HealthStatus,
+)
+
+collector = HealthCollector(include_external=False)
+assert [check.__name__ for check in collector._checks] == [
+    "_check_serving",
+    "_check_freshness",
+    "_check_quality_score",
+]
+
+# Existing config + blocked pyiceberg must yield degraded placeholder, not
+# the missing-config path (config check runs before the lazy import).
+health = collector._check_iceberg()
+assert health.name == "iceberg"
+assert health.status == HealthStatus.DEGRADED
+assert health.source == CheckSource.PLACEHOLDER
+assert health.message.startswith("Iceberg unavailable")
+assert "Iceberg config not found" not in health.message
+print("ok")
+"""
+    with tempfile.NamedTemporaryFile(
+        mode="w",
+        suffix=".yaml",
+        delete=False,
+        encoding="utf-8",
+    ) as handle:
+        handle.write("catalog:\n  name: test\n")
+        config_path = handle.name
+
+    env = os.environ.copy()
+    env["AGENTFLOW_ICEBERG_CONFIG"] = config_path
+    try:
+        result = subprocess.run(
+            [sys.executable, "-c", script],
+            cwd=PROJECT_ROOT,
+            capture_output=True,
+            text=True,
+            check=False,
+            env=env,
+        )
+    finally:
+        Path(config_path).unlink(missing_ok=True)
+
+    assert result.returncode == 0, result.stderr
+    assert "ok" in result.stdout
+
+
 def test_disabled_query_cache_does_not_construct_redis(
     monkeypatch: pytest.MonkeyPatch,
 ) -> None: