feat(postgres): support configurable SSL/TLS connections

[hvn2k1]

Fixes #2763

What this PR does

Makes the PostgreSQL SSL/TLS connection configurable instead of hardcoded to sslmode=disable, and defaults it to require so connections are encrypted unless a deployment explicitly opts out. Local development (Docker Compose and minikube) now serves TLS with a development certificate so it exercises the same path.

Background / Why this PR is needed

postgres.NewClient built its DSN with ?sslmode=disable hardcoded, so there was no way to reach a PostgreSQL server that mandates TLS — the reporter of #2763 could not deploy against Amazon RDS without forking or putting a proxy in front of it.

Rather than only making it configurable, the default is now require: prefer would silently fall back to plaintext, which leaves operators believing a connection is encrypted when it may not be. Failing loudly is the safer default while PostgreSQL support is still marked in development, so this is the right moment to change it.

Points

• Breaking for existing PostgreSQL deployments. Anyone self-hosting against a plaintext PostgreSQL must set sslMode: disable on upgrade, otherwise the services fail to connect at startup. Needs a release note.
• All four libpq parameters are exposed (sslmode, sslrootcert, sslcert, sslkey) via --postgres-ssl-* flags on api/web/batch/subscriber, plus the data warehouse configs, which are a separate connection. global.operationalDatabase.postgres.sslSecretName mounts a secret at /usr/local/certs/postgres in every service, since verify-ca/verify-full need the CA file for servers like RDS whose CA is not in the system trust store.
• The DSN is now built with net/url. This also fixes a latent bug: a password containing URL-reserved characters (@, /, ?, %) previously produced a broken DSN.
• The mode is validated client-side against the six libpq values, so a typo fails with a clear error instead of an obscure driver one.
• Local TLS uses a committed self-signed development certificate (tools/dev/cert/postgres-tls.*, generated by make -C tools/dev generate-postgres-tls-certificate). It is CA:TRUE, so the same file is both the server certificate and the root that clients verify, and its SANs cover postgres, localhost, 127.0.0.1 and *.default.svc.cluster.local. SANs are required: Go rejects CN-only certificates.
• The certificate is installed by an init step, not mounted directly. PostgreSQL only accepts a key owned by its own user and not group readable, which neither a bind mount, a git clone (git does not preserve 0600), nor fsGroup can guarantee — the localenv image is overridden to timescale/timescaledb, where postgres is uid 70, not the 999 the chart's fsGroup assumes. Both Compose and the chart therefore copy the files with install -o postgres from the postgres image itself, which is correct for any image variant.
• manifests/localenv/charts/postgresql-0.1.0.tgz is repackaged and must stay in sync with dependencies/postgresql/. Helm deploys the packaged copy, so editing only the source has no effect. The diff is a binary blob, so it is invisible in review.
• TLS is offered locally, not enforced (pg_hba.conf untouched). Plaintext still works, so hack/delete-e2e-data-postgres and hack/delete-postgres-data-warehouse keep working — those two are separate Go modules whose vendored dependencies are already out of sync with their go.mod on main, so they cannot reference the new API until they are re-vendored.
• Makefile defaults to require, not verify-full, because the minikube targets reach PostgreSQL over the node IP and the certificate cannot name it. The Compose targets go over localhost and verify in full.

[Copilot]

Warning

• Copilot's review of this pull request may be incomplete because some of the changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.

Pull request overview

This PR makes PostgreSQL SSL/TLS configurable (instead of hardcoded sslmode=disable) across Bucketeer services, defaulting connections to sslmode=require and updating local environments to serve TLS so the encrypted path is exercised.

Changes:

• Add a PostgreSQL TLS configuration surface (sslmode, sslrootcert, sslcert, sslkey) with client-side mode validation and DSN building via net/url (fixing reserved-character credential escaping).
• Wire new SSL/TLS flags/env vars into api/web/batch/subscriber and extend data-warehouse Postgres configs to carry SSL/TLS settings.
• Update Helm localenv and Docker Compose to serve PostgreSQL over TLS using a development certificate, plus Make targets to generate/install the cert and create Kubernetes secrets.

Reviewed changes

Copilot reviewed 28 out of 30 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tools/dev/postgres.config	Adds OpenSSL config for generating a dev Postgres TLS cert with required SANs.
tools/dev/Makefile	Adds targets to generate the Postgres TLS certificate and create a minikube secret from it.
tools/dev/cert/postgres-tls.crt	Commits the dev Postgres certificate used by local TLS setups.
tools/dev/cert/postgres-tls.key	Dev Postgres private key (content excluded from review by policy).
pkg/storage/v2/postgres/client.go	Introduces SSLConfig + DSN builder using net/url, validates sslmode, defaults to require.
pkg/storage/v2/postgres/client_test.go	Adds unit tests for DSN building, credential escaping, and SSL defaulting behavior.
pkg/api/cmd/server.go	Adds Postgres SSL/TLS CLI flags and passes SSLConfig into Postgres client creation.
pkg/web/cmd/server/server.go	Adds Postgres SSL/TLS CLI flags and extends DWH Postgres config to include SSL/TLS fields.
pkg/batch/cmd/server/server.go	Adds Postgres SSL/TLS CLI flags and passes SSLConfig into Postgres client creation.
pkg/subscriber/cmd/server/server.go	Adds Postgres SSL/TLS CLI flags and passes SSLConfig into Postgres client creation (incl. DWH).
pkg/subscriber/cmd/server/datawarehouse_config.go	Extends subscriber onDemandProcessors Postgres DWH JSON schema to include SSL/TLS fields.
manifests/bucketeer/values.yaml	Adds operational + data warehouse Postgres SSL/TLS values (incl. secret mount hook).
manifests/bucketeer/values.dev.yaml	Configures dev values to use verify-full and mount the dev Postgres cert secret; switches migration image repo.
manifests/bucketeer/charts/api/values.yaml	Exposes Postgres SSL/TLS values for the api chart.
manifests/bucketeer/charts/api/templates/deployment.yaml	Plumbs Postgres SSL/TLS env vars and mounts the optional Postgres cert secret.
manifests/bucketeer/charts/web/values.yaml	Exposes Postgres SSL/TLS values for the web chart.
manifests/bucketeer/charts/web/templates/deployment.yaml	Plumbs Postgres SSL/TLS env vars and mounts the optional Postgres cert secret.
manifests/bucketeer/charts/web/templates/datawarehouse-configmap.yaml	Adds Postgres SSL/TLS fields into the web datawarehouse configmap when DWH type is postgres.
manifests/bucketeer/charts/batch/values.yaml	Exposes Postgres SSL/TLS values for the batch chart.
manifests/bucketeer/charts/batch/templates/deployment.yaml	Plumbs Postgres SSL/TLS env vars and mounts the optional Postgres cert secret.
manifests/bucketeer/charts/subscriber/values.yaml	Exposes Postgres SSL/TLS values for subscriber operational DB and on-demand processor DWH configs.
manifests/bucketeer/charts/subscriber/templates/deployment.yaml	Plumbs Postgres SSL/TLS env vars and mounts the optional Postgres cert secret.
manifests/bucketeer/charts/subscriber/templates/subscribers-configmap.yaml	Includes Postgres SSL/TLS fields when rendering onDemandProcessors.json from Helm values.
manifests/localenv/dependencies/postgresql/values.yaml	Enables TLS for the localenv Postgres chart and references a Kubernetes TLS secret.
manifests/localenv/dependencies/postgresql/templates/statefulset.yaml	Installs TLS key/cert via init container (correct permissions/ownership) and enables Postgres SSL.
Makefile	Adds default Postgres SSL envs and ensures local compose/minikube flows create the Postgres cert secret / generate certs.
hack/create-postgres-event-tables/command.go	Adds Postgres SSL/TLS flags to the helper command and passes SSLConfig to the client.
docker-compose/config/subscriber-config/onDemandProcessors.json	Updates compose subscriber on-demand processor DWH Postgres config to include sslMode.
docker-compose/compose.yml	Adds a cert-install helper service + mounts certs and configures services to use verify-full locally.

Files excluded by content exclusion policy (1)

• tools/dev/cert/postgres-tls.key

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Warning

• Copilot's review of this pull request may be incomplete because some of the changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.

Pull request overview

Copilot reviewed 28 out of 30 changed files in this pull request and generated 2 comments.

Files excluded by content exclusion policy (1)

• tools/dev/cert/postgres-tls.key

Suppressed comments (4)

manifests/bucketeer/charts/web/templates/deployment.yaml:54

• Kubernetes Secret volumes default to mode 0644, but lib/pq rejects an sslkey that is group/world-readable. As a result, configuring the newly exposed client certificate and key makes this service fail to connect. Set a restrictive default mode on this secret volume.

          secret:
            secretName: {{ .Values.global.operationalDatabase.postgres.sslSecretName }}

manifests/bucketeer/charts/batch/templates/deployment.yaml:49

• Kubernetes Secret volumes default to mode 0644, but lib/pq rejects an sslkey that is group/world-readable. As a result, configuring the newly exposed client certificate and key makes this service fail to connect. Set a restrictive default mode on this secret volume.

          secret:
            secretName: {{ .Values.global.operationalDatabase.postgres.sslSecretName }}

manifests/bucketeer/charts/subscriber/templates/deployment.yaml:52

• Kubernetes Secret volumes default to mode 0644, but lib/pq rejects an sslkey that is group/world-readable. As a result, configuring the newly exposed client certificate and key makes this service fail to connect. Set a restrictive default mode on this secret volume.

          secret:
            secretName: {{ .Values.global.operationalDatabase.postgres.sslSecretName }}

manifests/bucketeer/charts/api/templates/deployment.yaml:48

• Kubernetes Secret volumes default to mode 0644, but lib/pq rejects an sslkey that is group/world-readable. As a result, configuring the newly exposed client certificate and key makes this service fail to connect. Set a restrictive default mode on this secret volume.

          secret:
            secretName: {{ .Values.global.operationalDatabase.postgres.sslSecretName }}

[Ubisoft-potato]

Nice work! 🚀

[t-kikuc]

LGTM

Let's consider MySQL later.