Skip to content

Commit b6a64de

Browse files
OneTwo3DOneTwo3Dclaude
authored
Installer: no secret on a command line (bootstrap env, GitHub/Cloudflare tokens) with a census (o3d-kb3dq) (#762)
* Installer: no secret on a command line (bootstrap env, GitHub and Cloudflare tokens) with a census Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * Installer: bootstrap exports through a function so the exit status is captured where the deploy-order guard reads it Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * Cron jobs: the bearer leaves curl's command line (installer and in-app scheduler), fail closed; census scans rendered output Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * Cron jobs: align the installer cron-secret test and the test env type with the stdin form Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * Installer: rewrite legacy cron -H bearer lines in an existing managed block (verified, refuse otherwise); wider argv census rules Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * Census canary: no literal user:pass at rest; gitleaks fingerprint for the earlier commit's sample Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * Installer: rewrite legacy cron bearer lines from update.sh and deploy.sh too; render job lines without pattern replacement Codex round 3 on #762: - HIGH: the legacy `curl -H "Authorization: Bearer $CRON_SECRET"` rewrite ran in install.sh only. The rewrite (cron_job_command_into, migrate_legacy_cron_lines, count_legacy_cron_lines, the locked rewrite+verify and the refusal) now lives once in scripts/lib/crontab-lock.sh; install.sh, update.sh and deploy.sh call it. update.sh and deploy.sh run it after the cron restore and before they report completion (never in a dry run) and end the run with an operator-visible error if legacy lines remain. - HIGH: an `&` in a valid legacy log path became `@LOG@` (patsub_replacement). The job line is now built by walking the template once and appending each value, so no value is ever a pattern or a replacement. Audit of every ${var/pat/repl} in scripts/*.sh and scripts/lib/*.sh: that one site was the only variable replacement; a test pins it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * Cron rewrite: roster the new locked bodies in the guards; template without bare $NAME The shared rewrite is a `*_locked` body (errexit suspended), so the crontab-reconcile roster and write census now include it; the deploy-order surface scan reads the library, so the job template is an unquoted heredoc with escaped dollars; install-cron-secret reads the library too. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * Cron rewrite: unmarked historical jobs and a general bearer-header detector Codex round 4 on #762: an older installer (d3ebd53..80bb370) wrote UNMARKED jobs with `curl -H "Authorization: Bearer ${CRON_SECRET}"` and a literal localhost URL. The rewrite now (1) rewrites that shape anywhere in the crontab to the stdin form, keeping schedule, .env path, URL and redirect, and (2) refuses completion, naming crontab line numbers and never the line, when ANY non-comment line still runs curl with an Authorization/Bearer header argument (literal secrets included), in all three scripts. install.sh now checks on every path. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * docs: unmarked historical cron jobs and the general bearer-header refusal Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * Cron rewrite: keep the historical-shape regex free of a command-substitution lookalike; install-cron-secret ignores comments Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * Tests: build the unrecognised-bearer canary lines by concatenation; gitleaks fingerprints for the previous commit's two samples Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb * First rollout: document running the release's own scripts (the predecessor driver cannot run the new rewrite), with a numbers-only check; simulate it Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb --------- Co-authored-by: OneTwo3D <onetwo3dcouk@gmail.com> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
1 parent b3c9397 commit b6a64de

14 files changed

Lines changed: 1637 additions & 57 deletions

‎.gitleaksignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,8 @@ a9b95f858ff43001af527147f8812bbe70612ab7:tests/domain/accounting/accounting-even
88
90a3d71134be9aed3fcaefcedb599be0f48d5b32:tests/concurrency/wc-order-status-jobs.concurrent.test.ts:generic-api-key:1096
99
# o3d-31qrv: a placeholder multi-line value in a re-run test fixture (armor-header-shaped words around fake text), not a key; the fixture no longer contains it.
1010
04e46121af939da1790ed5c05e26b5432ac6682c:tests/scripts/env-rewrite-preservation.test.ts:private-key:176
11+
# o3d-kb3dq: a canary SAMPLE of the shape `curl -u user:pass` in the argv census's own rule test (the census exists to catch that shape); no real credential. The line no longer has the literal form.
12+
1afaaa1b3aa026b4d8920142793c10c58fd0ff13:tests/scripts/installer-secrets-argv.test.ts:curl-auth-user:219
13+
# o3d-kb3dq round 4: a canary SAMPLE of an unrecognised bearer-header cron line in the rewrite's refusal tests (the rewrite exists to refuse that shape); no real credential. Later lines build it by concatenation.
14+
f592b26a762a25caf541b7b0bc1706005c847225:tests/scripts/installer-secrets-argv.test.ts:curl-auth-header:570
15+
f592b26a762a25caf541b7b0bc1706005c847225:tests/scripts/cron-bearer-rewrite-upgrade-paths.test.ts:curl-auth-header:536

‎docs/backup-restore.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -308,12 +308,14 @@ Scheduled backups are triggered via a cron endpoint:
308308
/api/cron/backup
309309
```
310310

311-
Configure your server's cron scheduler to call this endpoint at your preferred time. Cron endpoints require the `CRON_SECRET` bearer header in production, and production startup fails fast if `CRON_SECRET` is unset, blank, or shorter than 32 characters. Localhost bypass is available outside production only when no `CRON_SECRET` is configured; production never accepts localhost cron requests without the bearer header. Rotating `CRON_SECRET` requires updating both `.env` and any external cron scheduler invocations in the same maintenance window because the application reads the environment value on restart. For example, to run backups daily at 02:00:
311+
Configure your server's cron scheduler to call this endpoint at your preferred time. Cron endpoints require the `CRON_SECRET` bearer header in production, and production startup fails fast if `CRON_SECRET` is unset, blank, or shorter than 32 characters. Localhost bypass is available outside production only when no `CRON_SECRET` is configured; production never accepts localhost cron requests without the bearer header. Rotating `CRON_SECRET` requires updating both `.env` and any external cron scheduler invocations in the same maintenance window because the application reads the environment value on restart. The installer and the Scheduler page already write the right line for you. If you must write one by hand, do not put the secret in curl's arguments (`curl -H "Authorization: Bearer $CRON_SECRET"` shows it to every local account in `ps` while the job runs); hand it over on standard input as a curl config line. For example, to run backups daily at 02:00:
312312

313313
```
314-
0 2 * * * curl -s -H "Authorization: Bearer $CRON_SECRET" http://localhost:3000/api/cron/backup
314+
0 2 * * * echo "header = \"Authorization: Bearer $CRON_SECRET\"" | curl -s -K - http://localhost:3000/api/cron/backup
315315
```
316316

317+
(`CRON_SECRET` must be set in the crontab or read from `.env` first, as the installer's lines do; and `%` must never appear unescaped in a crontab line.)
318+
317319
## Activity Log
318320

319321
All backup operations — creation, restore, upload, deletion, and scheduled runs — are recorded in the system activity log for full auditability.

‎docs/installation.md‎

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -864,6 +864,10 @@ order exists to remove. A **first** install fences nothing, deliberately: there
864864
crontab and no data, so there is no writer to stop.
865865

866866

867+
### Secrets are never on a command line
868+
869+
`ps` shows the command line of every process to every local account, so the installer hands secrets to the programs it starts through the environment (readable by root and the owner only) or through a mode-600 file, never as an argument. The bootstrap step exports the default administrator's password, the SMTP password and the WooCommerce key and secret inside a subshell and lets `runuser` pass them to the application account's `provision-instance.mjs` (it used to be `env NAME=value ... node`, which put all of them on the command line for the life of the step). The GitHub deploy-key token (`install.sh`), the Cloudflare token (`provision-ims-tenant.sh`) and the cron jobs' `CRON_SECRET` (the crontab lines the installer and the in-app scheduler write) reach `curl` on its standard input as a `-K -` config line instead of a `-H "Authorization: ..."` argument. A test (`tests/scripts/installer-secrets-argv.test.ts`) runs the shipped bootstrap block with stand-ins that record every command line, runs the RENDERED cron command from both the installer and the app renderer under `sh`, and scans every shell script under `scripts/` and the rendered output of the cron writers for the shapes that put a secret on an argument; it fails on any new one. Two sites remain and are listed in that test with their reasons: `pct create --password` in `provision-ims-tenant.sh` (the Proxmox tool has no other form; it is an operator tool, not the installer) and the unwired legacy `scripts/backup.sh`.
870+
867871
## Configuration Prompts
868872

869873
The installer asks for the following values during setup. Press Enter to accept the default shown in brackets.
@@ -1264,7 +1268,7 @@ Never delete and recreate the lock file — the lock lives on the inode, so a re
12641268

12651269
**The installer bootstraps, it does not reconcile.** Its schedules are defaults, not your committed settings, so on a re-run or upgrade it writes its bootstrap block only when no managed `# --- OTI CRON START ---` block exists. If one is already there the application owns it and the installer leaves it alone — which means a job newly registered by an upgrade is not scheduled until you open Settings → System → Scheduler and press Save & Apply. The scheduler page's warning banner flags a missing or malformed managed block.
12661270

1267-
All cron jobs run under the `imsapp` user and call the application's API endpoints via `curl`. Cron endpoints require the `CRON_SECRET` bearer header in production, and production startup fails fast if `CRON_SECRET` is unset, blank, or shorter than 32 characters. Installer-generated crontab entries read only the `CRON_SECRET=` line from the protected `${APP_DIR}/.env` file at runtime so the cron secret is not embedded directly in the crontab and unrelated environment values are not shell-sourced. The in-app scheduler sync (Settings → System → Scheduler) prefers the runtime-read pattern too, but only selects it when a byte-for-byte emulation of the exact `grep | cut | tr` pipeline proves the app's `.env` yields the *active* process secret; every other case (no readable `.env`, a value that shell-parses differently, binary/`NUL` content, or an env supplied by the service manager and not the file) falls back to embedding the current literal, which is always correct at sync time. In runtime mode a secret rotation needs only the `.env` edit and a service restart — no crontab re-sync — and each job line carries a `[ -n "$CRON_SECRET" ]` guard so a missing file or empty extraction skips the request rather than sending an empty bearer. The generated cron log defaults to `/var/log/one-two-inventory/cron.log` (the installer-owned `LOG_DIR`, already covered by logrotate); override with `OTI_CRON_LOG_PATH`. The installer writes its bootstrap jobs inside the same OTI-marked managed block in this exact format, and on upgrade it removes every previous managed block (including whitespace- or CRLF-suffixed markers) and any legacy bootstrap lines that call `localhost:<app-port>/api/cron/`, while preserving the operator's own crontab entries — including unrelated `/api/cron/` calls to other hosts, a line that coincidentally matches the managed-job format but sits outside any managed block, and any operator lines after a malformed unclosed marker (never deleted to end-of-file; only the block's own remnants within that region are removed). So no duplicate or drifting One Two Inventory cron entries survive an upgrade. The first in-app scheduler save then replaces that block IN PLACE — at its original position among the operator's lines, so it never moves past an operator `PATH`, `SHELL`, or `CRON_TZ` assignment (which apply only to the jobs below them). `OTI_CRON_LOG_PATH` and the `.env` path are rejected if they contain a quote, `%`, or control character, and the embedded-literal fallback is rejected for a secret containing a quote, backslash, backtick, `$`, or newline (rotate to a hex/base64 secret) — so the crontab can never be corrupted by a crafted value. The scheduler page shows a warning banner when the managed block is missing, malformed, carries a stale embedded secret, has a runtime `.env` that no longer matches the running service, or when unmanaged `/api/cron/` lines exist outside the managed block. Rejected cron auth is also recorded (best-effort) as a WARNING activity (`cron_auth_rejected`), deferred off the 401 path and throttled to at most one per route per process per hour, so a stale secret surfaces instead of failing silently. Localhost bypass is available outside production only when no `CRON_SECRET` is configured; production never accepts localhost cron requests without the bearer header. After a valid cron secret, each cron endpoint is rate-limited per job and source IP when a client IP is available: daily/hourly jobs default to one accepted run per hour, 5-minute jobs allow 15 accepted runs per hour, and 15-minute jobs allow 6 accepted runs per hour. The sub-hourly quotas intentionally include scheduling-jitter headroom and should not be tightened to the exact cadence. Rate-limited calls return `429` with `Retry-After`. Single-process installs can use the default in-memory rate-limit backend. Multi-replica or load-balanced installs must set `RATE_LIMIT_BACKEND=redis` and `REDIS_URL` so cron throttles are cluster-wide. Rotating `CRON_SECRET` requires updating both `.env` and any external cron scheduler invocations in the same maintenance window because the application reads the environment value on restart; if an old or leaked secret consumed cron quota, restart the memory backend or clear the Redis rate-limit keys rather than waiting for the one-hour window to expire.
1271+
All cron jobs run under the `imsapp` user and call the application's API endpoints via `curl`. Cron endpoints require the `CRON_SECRET` bearer header in production, and production startup fails fast if `CRON_SECRET` is unset, blank, or shorter than 32 characters. Installer-generated crontab entries read only the `CRON_SECRET=` line from the protected `${APP_DIR}/.env` file at runtime so the cron secret is not embedded directly in the crontab and unrelated environment values are not shell-sourced. The in-app scheduler sync (Settings → System → Scheduler) prefers the runtime-read pattern too, but only selects it when a byte-for-byte emulation of the exact `grep | cut | tr` pipeline proves the app's `.env` yields the *active* process secret; every other case (no readable `.env`, a value that shell-parses differently, binary/`NUL` content, or an env supplied by the service manager and not the file) falls back to embedding the current literal, which is always correct at sync time. In runtime mode a secret rotation needs only the `.env` edit and a service restart — no crontab re-sync — and each job line is a `case` on the extracted secret: an empty value, or one holding a backslash, runs nothing and writes `cron-auth: CRON_SECRET is missing or unusable ...; <job> was not run` to the cron log, so a missing file or empty extraction skips the request rather than sending an unauthenticated one. **The bearer is never on a command line:** the line pipes `echo "header = \"Authorization: Bearer $CRON_SECRET\""` (a shell builtin) into `curl ... -K -`, so `ps` shows neither the secret nor a file path, and no file holds it; the crontab itself holds the secret only in the embedded-literal fallback, as one `CRON_SECRET="..."` line in the user's crontab spool (mode 600 root:crontab on Debian/Ubuntu). Earlier releases wrote `curl -H "Authorization: Bearer $CRON_SECRET"`, which put the secret in curl's argv while a job ran; the installer, `update.sh` and `deploy.sh` now each rewrite them in place when they find them in an existing managed block (one shared function, `scripts/lib/crontab-lock.sh`; `update.sh` and `deploy.sh` run it after the cron restore and before they report completion, never in a dry run): under the crontab lock, each legacy job keeps its schedule (including any you set in the Scheduler), slug, log path and secret source and only its command changes, the result is re-read and verified, and a block it cannot rewrite (or a failed write, or a lock held by another writer that leaves the old lines) ends the run with an error instead of a warning (the service keeps running; fix the cause and re-run, or press Save & Apply in the Scheduler and re-run). It also rewrites the plain, unmarked jobs an older installer appended outside any marker (`curl -fsS -H "Authorization: Bearer ${CRON_SECRET}" http://localhost:<port>/api/cron/<job>`), keeping schedule, `.env` path, URL and redirect, and it refuses to finish while ANY other crontab line still runs `curl` with an `Authorization`/`Bearer` header argument (a literal secret, an extra flag, a different quoting): the error names the crontab line number(s) and never prints the line. The rewrite inserts the log path and job name verbatim: an `&`, backslash, `%` or space in a log path survives it unchanged. The installer and the in-app scheduler render the identical line (a test compares them byte for byte and runs the rendered command under `sh` with a recording `curl`). The generated cron log defaults to `/var/log/one-two-inventory/cron.log` (the installer-owned `LOG_DIR`, already covered by logrotate); override with `OTI_CRON_LOG_PATH`. The installer writes its bootstrap jobs inside the same OTI-marked managed block in this exact format, and on upgrade it removes every previous managed block (including whitespace- or CRLF-suffixed markers) and any legacy bootstrap lines that call `localhost:<app-port>/api/cron/`, while preserving the operator's own crontab entries — including unrelated `/api/cron/` calls to other hosts, a line that coincidentally matches the managed-job format but sits outside any managed block, and any operator lines after a malformed unclosed marker (never deleted to end-of-file; only the block's own remnants within that region are removed). So no duplicate or drifting One Two Inventory cron entries survive an upgrade. The first in-app scheduler save then replaces that block IN PLACE — at its original position among the operator's lines, so it never moves past an operator `PATH`, `SHELL`, or `CRON_TZ` assignment (which apply only to the jobs below them). `OTI_CRON_LOG_PATH` and the `.env` path are rejected if they contain a quote, `%`, or control character, and the embedded-literal fallback is rejected for a secret containing a quote, backslash, backtick, `$`, or newline (rotate to a hex/base64 secret) — so the crontab can never be corrupted by a crafted value. The scheduler page shows a warning banner when the managed block is missing, malformed, carries a stale embedded secret, has a runtime `.env` that no longer matches the running service, or when unmanaged `/api/cron/` lines exist outside the managed block. Rejected cron auth is also recorded (best-effort) as a WARNING activity (`cron_auth_rejected`), deferred off the 401 path and throttled to at most one per route per process per hour, so a stale secret surfaces instead of failing silently. Localhost bypass is available outside production only when no `CRON_SECRET` is configured; production never accepts localhost cron requests without the bearer header. After a valid cron secret, each cron endpoint is rate-limited per job and source IP when a client IP is available: daily/hourly jobs default to one accepted run per hour, 5-minute jobs allow 15 accepted runs per hour, and 15-minute jobs allow 6 accepted runs per hour. The sub-hourly quotas intentionally include scheduling-jitter headroom and should not be tightened to the exact cadence. Rate-limited calls return `429` with `Retry-After`. Single-process installs can use the default in-memory rate-limit backend. Multi-replica or load-balanced installs must set `RATE_LIMIT_BACKEND=redis` and `REDIS_URL` so cron throttles are cluster-wide. Rotating `CRON_SECRET` requires updating both `.env` and any external cron scheduler invocations in the same maintenance window because the application reads the environment value on restart; if an old or leaked secret consumed cron quota, restart the memory backend or clear the Redis rate-limit keys rather than waiting for the one-hour window to expire.
12681272

12691273
For WooCommerce specifically:
12701274

@@ -1709,6 +1713,36 @@ The one-release minimum is deliberate — a driver that re-executed code it had
17091713
earlier would be exactly the defect this arrangement closes. The unbounded part is a consequence of
17101714
refusing to promote bytes nothing vouched for, and the remedy is the digest below, not a shorter lag.
17111715

1716+
### First rollout of the release that takes the cron bearer off the command line
1717+
1718+
Earlier releases scheduled their cron jobs as `curl -H "Authorization: Bearer $CRON_SECRET"`, which shows
1719+
`CRON_SECRET` to every local account in `ps` each time a job runs. This release rewrites those lines
1720+
(`update.sh`, `deploy.sh` and `install.sh` all run the same locked rewrite, and `update.sh`/`deploy.sh`
1721+
refuse to report completion while any such line remains). **But the documented command runs the standing
1722+
root-owned driver, and on the first rollout the standing driver is the PREDECESSOR: it deploys the new code
1723+
and restores the old crontab untouched, because the rewrite exists only in this release's scripts.** An
1724+
update run that way completes, reports success, and the old lines are still scheduled.
1725+
1726+
For this release, run the release's own scripts instead of the standing driver. Fetch the release as root
1727+
into a newly created directory (*[The supported bootstrap](#supported-bootstrap)*) and either run
1728+
`install.sh` from there, or run its `update.sh` (root-owned trees are accepted; only trees another account
1729+
can write are refused):
1730+
1731+
```bash
1732+
sudo bash /root/ims-release-NEW/scripts/update.sh # add IMS_DRIVER_SHA256=<64 hex> to refresh the driver as well
1733+
```
1734+
1735+
Either way the rewrite runs before the run can report completion. If you did update through the standing
1736+
driver, nothing flagged it: check, and then run one of the two commands above. The check prints only crontab
1737+
line NUMBERS (never the lines, which may carry the secret), and nothing when the crontab is clean:
1738+
1739+
```bash
1740+
sudo crontab -u imsapp -l | grep -nE 'curl.*[[:space:]](-H|--header).*([Bb]earer|[Aa]uthorization)' | cut -d: -f1
1741+
```
1742+
1743+
Once the standing driver is this release or later (it is refreshed by `install.sh`, or by an update run
1744+
with `IMS_DRIVER_SHA256`), the documented command carries the rewrite and no special step is needed.
1745+
17121746
To see what is standing before deciding, read its record — which lives beside the tree the pointer
17131747
names, so this always reports the copy that is actually in use:
17141748

0 commit comments

Comments
 (0)