You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit b6a64de
Browse filesBrowse the repository at this point in the historyBrowse files
authored
Installer: no secret on a command line (bootstrap env, GitHub/Cloudflare tokens) with a census (o3d-kb3dq) (#762)
* Installer: no secret on a command line (bootstrap env, GitHub and Cloudflare tokens) with a census
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
* Installer: bootstrap exports through a function so the exit status is captured where the deploy-order guard reads it
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
* Cron jobs: the bearer leaves curl's command line (installer and in-app scheduler), fail closed; census scans rendered output
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
* Cron jobs: align the installer cron-secret test and the test env type with the stdin form
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
* Installer: rewrite legacy cron -H bearer lines in an existing managed block (verified, refuse otherwise); wider argv census rules
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
* Census canary: no literal user:pass at rest; gitleaks fingerprint for the earlier commit's sample
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
* Installer: rewrite legacy cron bearer lines from update.sh and deploy.sh too; render job lines without pattern replacement
Codex round 3 on #762:
- HIGH: the legacy `curl -H "Authorization: Bearer $CRON_SECRET"` rewrite ran in install.sh only. The rewrite
(cron_job_command_into, migrate_legacy_cron_lines, count_legacy_cron_lines, the locked rewrite+verify and the
refusal) now lives once in scripts/lib/crontab-lock.sh; install.sh, update.sh and deploy.sh call it. update.sh and
deploy.sh run it after the cron restore and before they report completion (never in a dry run) and end the run with
an operator-visible error if legacy lines remain.
- HIGH: an `&` in a valid legacy log path became `@LOG@` (patsub_replacement). The job line is now built by walking the
template once and appending each value, so no value is ever a pattern or a replacement. Audit of every
${var/pat/repl} in scripts/*.sh and scripts/lib/*.sh: that one site was the only variable replacement; a test pins it.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
* Cron rewrite: roster the new locked bodies in the guards; template without bare $NAME
The shared rewrite is a `*_locked` body (errexit suspended), so the crontab-reconcile roster and write census now
include it; the deploy-order surface scan reads the library, so the job template is an unquoted heredoc with escaped
dollars; install-cron-secret reads the library too.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
* Cron rewrite: unmarked historical jobs and a general bearer-header detector
Codex round 4 on #762: an older installer (d3ebd53..80bb370) wrote UNMARKED jobs with
`curl -H "Authorization: Bearer ${CRON_SECRET}"` and a literal localhost URL. The rewrite now (1) rewrites that shape
anywhere in the crontab to the stdin form, keeping schedule, .env path, URL and redirect, and (2) refuses completion,
naming crontab line numbers and never the line, when ANY non-comment line still runs curl with an Authorization/Bearer
header argument (literal secrets included), in all three scripts. install.sh now checks on every path.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
* docs: unmarked historical cron jobs and the general bearer-header refusal
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
* Cron rewrite: keep the historical-shape regex free of a command-substitution lookalike; install-cron-secret ignores comments
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
* Tests: build the unrecognised-bearer canary lines by concatenation; gitleaks fingerprints for the previous commit's two samples
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
* First rollout: document running the release's own scripts (the predecessor driver cannot run the new rewrite), with a numbers-only check; simulate it
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01M2fZEeZ5fYPBhB3TEKv9vb
---------
Co-authored-by: OneTwo3D <onetwo3dcouk@gmail.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
# o3d-31qrv: a placeholder multi-line value in a re-run test fixture (armor-header-shaped words around fake text), not a key; the fixture no longer contains it.
# o3d-kb3dq: a canary SAMPLE of the shape `curl -u user:pass` in the argv census's own rule test (the census exists to catch that shape); no real credential. The line no longer has the literal form.
# o3d-kb3dq round 4: a canary SAMPLE of an unrecognised bearer-header cron line in the rewrite's refusal tests (the rewrite exists to refuse that shape); no real credential. Later lines build it by concatenation.
Copy file name to clipboardExpand all lines: docs/backup-restore.md
+4-2Lines changed: 4 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -308,12 +308,14 @@ Scheduled backups are triggered via a cron endpoint:
308
308
/api/cron/backup
309
309
```
310
310
311
-
Configure your server's cron scheduler to call this endpoint at your preferred time. Cron endpoints require the `CRON_SECRET` bearer header in production, and production startup fails fast if `CRON_SECRET` is unset, blank, or shorter than 32 characters. Localhost bypass is available outside production only when no `CRON_SECRET` is configured; production never accepts localhost cron requests without the bearer header. Rotating `CRON_SECRET` requires updating both `.env` and any external cron scheduler invocations in the same maintenance window because the application reads the environment value on restart. For example, to run backups daily at 02:00:
311
+
Configure your server's cron scheduler to call this endpoint at your preferred time. Cron endpoints require the `CRON_SECRET` bearer header in production, and production startup fails fast if `CRON_SECRET` is unset, blank, or shorter than 32 characters. Localhost bypass is available outside production only when no `CRON_SECRET` is configured; production never accepts localhost cron requests without the bearer header. Rotating `CRON_SECRET` requires updating both `.env` and any external cron scheduler invocations in the same maintenance window because the application reads the environment value on restart. The installer and the Scheduler page already write the right line for you. If you must write one by hand, do not put the secret in curl's arguments (`curl -H "Authorization: Bearer $CRON_SECRET"` shows it to every local account in `ps` while the job runs); hand it over on standard input as a curl config line. For example, to run backups daily at 02:00:
(`CRON_SECRET` must be set in the crontab or read from `.env` first, as the installer's lines do; and `%` must never appear unescaped in a crontab line.)
318
+
317
319
## Activity Log
318
320
319
321
All backup operations — creation, restore, upload, deletion, and scheduled runs — are recorded in the system activity log for full auditability.
Copy file name to clipboardExpand all lines: docs/installation.md
+35-1Lines changed: 35 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -864,6 +864,10 @@ order exists to remove. A **first** install fences nothing, deliberately: there
864
864
crontab and no data, so there is no writer to stop.
865
865
866
866
867
+
### Secrets are never on a command line
868
+
869
+
`ps` shows the command line of every process to every local account, so the installer hands secrets to the programs it starts through the environment (readable by root and the owner only) or through a mode-600 file, never as an argument. The bootstrap step exports the default administrator's password, the SMTP password and the WooCommerce key and secret inside a subshell and lets `runuser` pass them to the application account's `provision-instance.mjs` (it used to be `env NAME=value ... node`, which put all of them on the command line for the life of the step). The GitHub deploy-key token (`install.sh`), the Cloudflare token (`provision-ims-tenant.sh`) and the cron jobs' `CRON_SECRET` (the crontab lines the installer and the in-app scheduler write) reach `curl` on its standard input as a `-K -` config line instead of a `-H "Authorization: ..."` argument. A test (`tests/scripts/installer-secrets-argv.test.ts`) runs the shipped bootstrap block with stand-ins that record every command line, runs the RENDERED cron command from both the installer and the app renderer under `sh`, and scans every shell script under `scripts/` and the rendered output of the cron writers for the shapes that put a secret on an argument; it fails on any new one. Two sites remain and are listed in that test with their reasons: `pct create --password` in `provision-ims-tenant.sh` (the Proxmox tool has no other form; it is an operator tool, not the installer) and the unwired legacy `scripts/backup.sh`.
870
+
867
871
## Configuration Prompts
868
872
869
873
The installer asks for the following values during setup. Press Enter to accept the default shown in brackets.
@@ -1264,7 +1268,7 @@ Never delete and recreate the lock file — the lock lives on the inode, so a re
1264
1268
1265
1269
**The installer bootstraps, it does not reconcile.** Its schedules are defaults, not your committed settings, so on a re-run or upgrade it writes its bootstrap block only when no managed `# --- OTI CRON START ---` block exists. If one is already there the application owns it and the installer leaves it alone — which means a job newly registered by an upgrade is not scheduled until you open Settings → System → Scheduler and press Save & Apply. The scheduler page's warning banner flags a missing or malformed managed block.
1266
1270
1267
-
All cron jobs run under the `imsapp` user and call the application's API endpoints via `curl`. Cron endpoints require the `CRON_SECRET` bearer header in production, and production startup fails fast if `CRON_SECRET` is unset, blank, or shorter than 32 characters. Installer-generated crontab entries read only the `CRON_SECRET=` line from the protected `${APP_DIR}/.env` file at runtime so the cron secret is not embedded directly in the crontab and unrelated environment values are not shell-sourced. The in-app scheduler sync (Settings → System → Scheduler) prefers the runtime-read pattern too, but only selects it when a byte-for-byte emulation of the exact `grep | cut | tr` pipeline proves the app's `.env` yields the *active* process secret; every other case (no readable `.env`, a value that shell-parses differently, binary/`NUL` content, or an env supplied by the service manager and not the file) falls back to embedding the current literal, which is always correct at sync time. In runtime mode a secret rotation needs only the `.env` edit and a service restart — no crontab re-sync — and each job line carries a `[ -n "$CRON_SECRET" ]` guard so a missing file or empty extraction skips the request rather than sending an empty bearer. The generated cron log defaults to `/var/log/one-two-inventory/cron.log` (the installer-owned `LOG_DIR`, already covered by logrotate); override with `OTI_CRON_LOG_PATH`. The installer writes its bootstrap jobs inside the same OTI-marked managed block in this exact format, and on upgrade it removes every previous managed block (including whitespace- or CRLF-suffixed markers) and any legacy bootstrap lines that call `localhost:<app-port>/api/cron/`, while preserving the operator's own crontab entries — including unrelated `/api/cron/` calls to other hosts, a line that coincidentally matches the managed-job format but sits outside any managed block, and any operator lines after a malformed unclosed marker (never deleted to end-of-file; only the block's own remnants within that region are removed). So no duplicate or drifting One Two Inventory cron entries survive an upgrade. The first in-app scheduler save then replaces that block IN PLACE — at its original position among the operator's lines, so it never moves past an operator `PATH`, `SHELL`, or `CRON_TZ` assignment (which apply only to the jobs below them). `OTI_CRON_LOG_PATH` and the `.env` path are rejected if they contain a quote, `%`, or control character, and the embedded-literal fallback is rejected for a secret containing a quote, backslash, backtick, `$`, or newline (rotate to a hex/base64 secret) — so the crontab can never be corrupted by a crafted value. The scheduler page shows a warning banner when the managed block is missing, malformed, carries a stale embedded secret, has a runtime `.env` that no longer matches the running service, or when unmanaged `/api/cron/` lines exist outside the managed block. Rejected cron auth is also recorded (best-effort) as a WARNING activity (`cron_auth_rejected`), deferred off the 401 path and throttled to at most one per route per process per hour, so a stale secret surfaces instead of failing silently. Localhost bypass is available outside production only when no `CRON_SECRET` is configured; production never accepts localhost cron requests without the bearer header. After a valid cron secret, each cron endpoint is rate-limited per job and source IP when a client IP is available: daily/hourly jobs default to one accepted run per hour, 5-minute jobs allow 15 accepted runs per hour, and 15-minute jobs allow 6 accepted runs per hour. The sub-hourly quotas intentionally include scheduling-jitter headroom and should not be tightened to the exact cadence. Rate-limited calls return `429` with `Retry-After`. Single-process installs can use the default in-memory rate-limit backend. Multi-replica or load-balanced installs must set `RATE_LIMIT_BACKEND=redis` and `REDIS_URL` so cron throttles are cluster-wide. Rotating `CRON_SECRET` requires updating both `.env` and any external cron scheduler invocations in the same maintenance window because the application reads the environment value on restart; if an old or leaked secret consumed cron quota, restart the memory backend or clear the Redis rate-limit keys rather than waiting for the one-hour window to expire.
1271
+
All cron jobs run under the `imsapp` user and call the application's API endpoints via `curl`. Cron endpoints require the `CRON_SECRET` bearer header in production, and production startup fails fast if `CRON_SECRET` is unset, blank, or shorter than 32 characters. Installer-generated crontab entries read only the `CRON_SECRET=` line from the protected `${APP_DIR}/.env` file at runtime so the cron secret is not embedded directly in the crontab and unrelated environment values are not shell-sourced. The in-app scheduler sync (Settings → System → Scheduler) prefers the runtime-read pattern too, but only selects it when a byte-for-byte emulation of the exact `grep | cut | tr` pipeline proves the app's `.env` yields the *active* process secret; every other case (no readable `.env`, a value that shell-parses differently, binary/`NUL` content, or an env supplied by the service manager and not the file) falls back to embedding the current literal, which is always correct at sync time. In runtime mode a secret rotation needs only the `.env` edit and a service restart — no crontab re-sync — and each job line is a `case` on the extracted secret: an empty value, or one holding a backslash, runs nothing and writes `cron-auth: CRON_SECRET is missing or unusable ...; <job> was not run` to the cron log, so a missing file or empty extraction skips the request rather than sending an unauthenticated one. **The bearer is never on a command line:** the line pipes `echo "header = \"Authorization: Bearer $CRON_SECRET\""` (a shell builtin) into `curl ... -K -`, so `ps` shows neither the secret nor a file path, and no file holds it; the crontab itself holds the secret only in the embedded-literal fallback, as one `CRON_SECRET="..."` line in the user's crontab spool (mode 600 root:crontab on Debian/Ubuntu). Earlier releases wrote `curl -H "Authorization: Bearer $CRON_SECRET"`, which put the secret in curl's argv while a job ran; the installer, `update.sh` and `deploy.sh` now each rewrite them in place when they find them in an existing managed block (one shared function, `scripts/lib/crontab-lock.sh`; `update.sh` and `deploy.sh` run it after the cron restore and before they report completion, never in a dry run): under the crontab lock, each legacy job keeps its schedule (including any you set in the Scheduler), slug, log path and secret source and only its command changes, the result is re-read and verified, and a block it cannot rewrite (or a failed write, or a lock held by another writer that leaves the old lines) ends the run with an error instead of a warning (the service keeps running; fix the cause and re-run, or press Save & Apply in the Scheduler and re-run). It also rewrites the plain, unmarked jobs an older installer appended outside any marker (`curl -fsS -H "Authorization: Bearer ${CRON_SECRET}" http://localhost:<port>/api/cron/<job>`), keeping schedule, `.env` path, URL and redirect, and it refuses to finish while ANY other crontab line still runs `curl` with an `Authorization`/`Bearer` header argument (a literal secret, an extra flag, a different quoting): the error names the crontab line number(s) and never prints the line. The rewrite inserts the log path and job name verbatim: an `&`, backslash, `%` or space in a log path survives it unchanged. The installer and the in-app scheduler render the identical line (a test compares them byte for byte and runs the rendered command under `sh` with a recording `curl`). The generated cron log defaults to `/var/log/one-two-inventory/cron.log` (the installer-owned `LOG_DIR`, already covered by logrotate); override with `OTI_CRON_LOG_PATH`. The installer writes its bootstrap jobs inside the same OTI-marked managed block in this exact format, and on upgrade it removes every previous managed block (including whitespace- or CRLF-suffixed markers) and any legacy bootstrap lines that call `localhost:<app-port>/api/cron/`, while preserving the operator's own crontab entries — including unrelated `/api/cron/` calls to other hosts, a line that coincidentally matches the managed-job format but sits outside any managed block, and any operator lines after a malformed unclosed marker (never deleted to end-of-file; only the block's own remnants within that region are removed). So no duplicate or drifting One Two Inventory cron entries survive an upgrade. The first in-app scheduler save then replaces that block IN PLACE — at its original position among the operator's lines, so it never moves past an operator `PATH`, `SHELL`, or `CRON_TZ` assignment (which apply only to the jobs below them). `OTI_CRON_LOG_PATH` and the `.env` path are rejected if they contain a quote, `%`, or control character, and the embedded-literal fallback is rejected for a secret containing a quote, backslash, backtick, `$`, or newline (rotate to a hex/base64 secret) — so the crontab can never be corrupted by a crafted value. The scheduler page shows a warning banner when the managed block is missing, malformed, carries a stale embedded secret, has a runtime `.env` that no longer matches the running service, or when unmanaged `/api/cron/` lines exist outside the managed block. Rejected cron auth is also recorded (best-effort) as a WARNING activity (`cron_auth_rejected`), deferred off the 401 path and throttled to at most one per route per process per hour, so a stale secret surfaces instead of failing silently. Localhost bypass is available outside production only when no `CRON_SECRET` is configured; production never accepts localhost cron requests without the bearer header. After a valid cron secret, each cron endpoint is rate-limited per job and source IP when a client IP is available: daily/hourly jobs default to one accepted run per hour, 5-minute jobs allow 15 accepted runs per hour, and 15-minute jobs allow 6 accepted runs per hour. The sub-hourly quotas intentionally include scheduling-jitter headroom and should not be tightened to the exact cadence. Rate-limited calls return `429` with `Retry-After`. Single-process installs can use the default in-memory rate-limit backend. Multi-replica or load-balanced installs must set `RATE_LIMIT_BACKEND=redis` and `REDIS_URL` so cron throttles are cluster-wide. Rotating `CRON_SECRET` requires updating both `.env` and any external cron scheduler invocations in the same maintenance window because the application reads the environment value on restart; if an old or leaked secret consumed cron quota, restart the memory backend or clear the Redis rate-limit keys rather than waiting for the one-hour window to expire.
1268
1272
1269
1273
For WooCommerce specifically:
1270
1274
@@ -1709,6 +1713,36 @@ The one-release minimum is deliberate — a driver that re-executed code it had
1709
1713
earlier would be exactly the defect this arrangement closes. The unbounded part is a consequence of
1710
1714
refusing to promote bytes nothing vouched for, and the remedy is the digest below, not a shorter lag.
1711
1715
1716
+
### First rollout of the release that takes the cron bearer off the command line
1717
+
1718
+
Earlier releases scheduled their cron jobs as `curl -H "Authorization: Bearer $CRON_SECRET"`, which shows
1719
+
`CRON_SECRET` to every local account in `ps` each time a job runs. This release rewrites those lines
1720
+
(`update.sh`, `deploy.sh` and `install.sh` all run the same locked rewrite, and `update.sh`/`deploy.sh`
1721
+
refuse to report completion while any such line remains). **But the documented command runs the standing
1722
+
root-owned driver, and on the first rollout the standing driver is the PREDECESSOR: it deploys the new code
1723
+
and restores the old crontab untouched, because the rewrite exists only in this release's scripts.** An
1724
+
update run that way completes, reports success, and the old lines are still scheduled.
1725
+
1726
+
For this release, run the release's own scripts instead of the standing driver. Fetch the release as root
1727
+
into a newly created directory (*[The supported bootstrap](#supported-bootstrap)*) and either run
1728
+
`install.sh` from there, or run its `update.sh` (root-owned trees are accepted; only trees another account
1729
+
can write are refused):
1730
+
1731
+
```bash
1732
+
sudo bash /root/ims-release-NEW/scripts/update.sh # add IMS_DRIVER_SHA256=<64 hex> to refresh the driver as well
1733
+
```
1734
+
1735
+
Either way the rewrite runs before the run can report completion. If you did update through the standing
1736
+
driver, nothing flagged it: check, and then run one of the two commands above. The check prints only crontab
1737
+
line NUMBERS (never the lines, which may carry the secret), and nothing when the crontab is clean:
0 commit comments