chore: narrow the CI secret's blast radius and pin actions by SHA

Two mitigations for the long-lived DATABRICKS_CLIENT_SECRET, plus the context
for why it is still there at all.

The secret moves from the workflow-level env block, where it was visible to
every step including dependency installation, to the three steps that actually
talk to Databricks. Dependency install and unit tests are exactly where
third-party code executes, and they never needed it. The host and client id
stay at workflow level — they are identifiers, not credentials.

Every third-party action is now pinned to a full commit SHA rather than a tag,
since tags are mutable and whoever controls an action's repo can repoint one at
code that then runs on the runner with that environment. databricks/setup-cli
resolves the CLI version from a VERSION file at the pinned ref, so the SHA pin
keeps the CLI on 1.9.0 exactly as the tag did — verified, not assumed.

The architecture spec now records why OIDC / workload identity federation is
the mechanism Databricks recommends instead, and why this repo cannot use it:
the trust anchor is a service principal federation policy, an account-level
API, and Free Edition has no account-level API access. The exact policy command
and workflow delta are documented for anyone forking onto a paid account.
Neither mitigation shortens the credential's lifetime; only OIDC does that.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: andre-salvati

.github/workflows/onpush.yml

@@ -21,28 +21,34 @@ concurrency:
   group: deploy-${{ github.ref }}
   cancel-in-progress: false
 
+# `DATABRICKS_CLIENT_SECRET` is deliberately NOT here: a workflow-level env var is visible
+# to every step, including dependency installation, where third-party action and package
+# code executes. It is injected per-step, only on the three steps that talk to Databricks.
+# (The host and client ID are identifiers, not credentials — leaking them grants nothing.)
 env:
   DATABRICKS_HOST: ${{ secrets.DATABRICKS_HOST }}
   DATABRICKS_CLIENT_ID: ${{ secrets.DATABRICKS_CLIENT_ID }}
-  DATABRICKS_CLIENT_SECRET: ${{ secrets.DATABRICKS_CLIENT_SECRET }}
   TEMPLATE_ALERT_EMAILS: ${{ secrets.TEMPLATE_ALERT_EMAILS }}
 
 jobs:
   ci:
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@v4
+      # Third-party actions are pinned to a full commit SHA, not a tag: tags are mutable,
+      # so whoever controls the action's repo can repoint one at new code that runs on this
+      # runner. The trailing comment records which release the SHA corresponds to.
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262  # v4
 
       - name: Set up Python
-        uses: actions/setup-python@v5
+        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065  # v5
         with:
           python-version: '3.12'
 
       - name: Install uv
-        # Pinned to the full release tag: setup-uv stopped publishing moving major
-        # tags (v1..v7) after v7, so `@v9` does not resolve — use the exact tag.
-        uses: astral-sh/setup-uv@v9.0.0
+        # setup-uv stopped publishing moving major tags (v1..v7) after v7, so `@v9` does
+        # not resolve — the comment records the exact release this SHA belongs to.
+        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9  # v9.0.0
         with:
           version: "0.11.31"
           enable-cache: true
@@ -62,22 +68,28 @@ jobs:
 
       - name: Upload coverage report
         if: always()
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
         with:
           name: coverage-report
           path: reports/coverage/
           retention-days: 14
 
-      # Pin the CLI to a tagged release so an upstream change can't silently break CI.
+      # Pinned so an upstream change can't silently break CI; comment records the release.
       - name: Install Databricks CLI
-        uses: databricks/setup-cli@v1.9.0
+        uses: databricks/setup-cli@8b7b124dc4f5b9621e959ea61cf4d3dd4e421f67  # v1.9.0
 
       - name: Deploy on staging
+        env:
+          DATABRICKS_CLIENT_SECRET: ${{ secrets.DATABRICKS_CLIENT_SECRET }}
         run: make deploy env=staging
 
       - name: Run integration tests on staging
+        env:
+          DATABRICKS_CLIENT_SECRET: ${{ secrets.DATABRICKS_CLIENT_SECRET }}
         run: make run env=staging
 
       - name: Deploy on prod
         if: github.ref == 'refs/heads/main'
+        env:
+          DATABRICKS_CLIENT_SECRET: ${{ secrets.DATABRICKS_CLIENT_SECRET }}
         run: make deploy env=prod

specs/architecture.md

@@ -101,7 +101,55 @@ and `specs/**`. There is **no `bundle validate` step** — `make deploy` regener
 trigger allows re-running CI without a new commit.
 
 Requires `DATABRICKS_HOST`, `DATABRICKS_CLIENT_ID`, `DATABRICKS_CLIENT_SECRET`, and
-`TEMPLATE_ALERT_EMAILS` repo secrets. CLI and action versions are pinned.
+`TEMPLATE_ALERT_EMAILS` repo secrets. Every third-party action is pinned to a **full commit SHA**
+(with the release in a trailing comment) rather than a tag — tags are mutable, so whoever controls an
+action's repository can repoint one at code that then runs on the runner. `databricks/setup-cli`
+resolves the CLI version from a `VERSION` file at the pinned ref, so the SHA pin also keeps the CLI
+version fixed.
+
+### CI authentication — why the client secret, and what replaces it
+
+CI authenticates as `template-sp` with a **long-lived OAuth client secret** stored as a GitHub
+secret. Databricks explicitly recommends against this: the preferred mechanism is
+[workload identity federation](https://docs.databricks.com/aws/en/dev-tools/auth/provider-github)
+(OIDC), where the runner mints a short-lived JWT signed by GitHub and Databricks exchanges it for an
+OAuth token — nothing stored, nothing to rotate, useless if leaked.
+
+**This template cannot use it, because it runs on Databricks Free Edition**, which has
+[no access to the account console or account-level APIs](https://docs.databricks.com/aws/en/getting-started/free-edition-limitations).
+The trust anchor for OIDC is a *service principal federation policy*, and that is an account-level
+API — every call returns `unauthenticated` on Free Edition regardless of workspace admin rights.
+
+Two mitigations do apply here, and are in place. First, `DATABRICKS_CLIENT_SECRET` is injected
+**per-step**, only on the three steps that talk to Databricks — a workflow-level `env:` would expose
+it to every step, including dependency installation, which is exactly where third-party code
+executes. Second, the SHA pinning described above closes the other route to that runner environment.
+Neither shortens the credential's lifetime, which is what only OIDC fixes.
+
+If you fork this template onto a paid account, this is the first hardening to apply. An account
+admin creates the policy once:
+
+```bash
+databricks account service-principal-federation-policy create <TEMPLATE_SP_NUMERIC_ID> --json '{
+  "oidc_policy": {
+    "issuer": "https://token.actions.githubusercontent.com",
+    "subject_claim": "repository",
+    "subject": "<github-org>/<repo>"
+  }
+}'
+```
+
+then in `onpush.yml` add `permissions: {id-token: write, contents: read}`, set
+`DATABRICKS_AUTH_TYPE: github-oidc`, and drop `DATABRICKS_CLIENT_SECRET`. Match on the `repository`
+claim rather than the default `sub`: `sub` embeds the branch
+(`repo:org/repo:ref:refs/heads/main`) and policy subjects are exact matches with no wildcards, but
+CI deploys to staging on **every** branch — one policy on `repository` covers them all. That does
+not widen prod access, which is gated by `if: github.ref == 'refs/heads/main'` in the workflow. (A
+per-environment policy, `...:environment:staging` / `...:environment:production`, is tighter and
+pairs naturally with GitHub Environments and required reviewers.)
+
+Local development is unaffected either way — it authenticates through the `dev`/`staging`/`prod`
+profiles in `.databrickscfg`.
 
 <!-- Diagram source: assets/ci_cd.drawio (edit in https://app.diagrams.net, then export over assets/ci_cd.png) -->
 <img src="../assets/ci_cd.png" alt="CI/CD: local dev loop and the onpush.yml GitHub Actions pipeline">