|
| 1 | +--- |
| 2 | +date_created: 2026-08-15 |
| 3 | +date_modified: 2026-08-15 |
| 4 | +title: "Every frontend deploy had been failing for twelve days on one missing COPY" |
| 5 | +lede: "Production was serving a build from August 3rd. Every deploy since had failed at the same line, in all six frontend Dockerfiles, for the same reason — and nothing was watching." |
| 6 | +publish: true |
| 7 | +authors: |
| 8 | + - Michael Staton |
| 9 | +augmented_with: |
| 10 | + - Claude Code on Claude Opus 5 |
| 11 | +files_changed: |
| 12 | + - shell/Dockerfile |
| 13 | + - apps/chat/Dockerfile |
| 14 | + - apps/corpora-curator/Dockerfile |
| 15 | + - apps/org-workbench/Dockerfile |
| 16 | + - apps/search-and-add/Dockerfile |
| 17 | + - apps/search-results/Dockerfile |
| 18 | +tags: |
| 19 | + - Augment-It |
| 20 | + - Deployment |
| 21 | + - Docker |
| 22 | + - Debugging |
| 23 | + - Build-Pipeline |
| 24 | +--- |
| 25 | + |
| 26 | +# Every frontend deploy had been failing for twelve days |
| 27 | + |
| 28 | +## Why Care? |
| 29 | + |
| 30 | +We went looking for something else entirely — confirming which environment |
| 31 | +variable names a service still supplied, during a rename — and found that the |
| 32 | +service had not deployed successfully since **2026-08-03**. Four consecutive |
| 33 | +failures. The most recent one was the release commit. |
| 34 | + |
| 35 | +Nothing was broken on the site, which is exactly why nobody noticed: the last |
| 36 | +good container kept serving. A failed deploy on this setup is silent. The old |
| 37 | +build just keeps running, the URL keeps answering, and the only evidence is a |
| 38 | +red entry in a dashboard nobody had open. |
| 39 | + |
| 40 | +The cause was one missing line, in six files, and it had been wrong the whole |
| 41 | +time. |
| 42 | + |
| 43 | +## What's New? |
| 44 | + |
| 45 | +Every frontend Dockerfile now copies `tsconfig.base.json` into the build image. |
| 46 | +That's the entire fix: |
| 47 | + |
| 48 | +```dockerfile |
| 49 | +COPY package.json pnpm-lock.yaml pnpm-workspace.yaml ./ |
| 50 | +COPY tsconfig.base.json ./ # ← this |
| 51 | +COPY packages ./packages |
| 52 | +COPY apps ./apps |
| 53 | +COPY shell ./shell |
| 54 | +``` |
| 55 | + |
| 56 | +All six frontends — `shell`, `chat`, `corpora-curator`, `org-workbench`, |
| 57 | +`search-and-add`, `search-results` — now build clean, verified with a real |
| 58 | +`docker build` of each rather than by reading the diff. |
| 59 | + |
| 60 | +## How we found it |
| 61 | + |
| 62 | +The build log ends like this: |
| 63 | + |
| 64 | +``` |
| 65 | +[build 9/9] RUN pnpm --filter @augment-it/strategy-curator build |
| 66 | +> rsbuild build |
| 67 | +
|
| 68 | +error Build errors: |
| 69 | + × Tsconfig not found /monorepo/tsconfig.base.json |
| 70 | +
|
| 71 | +File: @module-federation/runtime/rspack.js:1:1-236 |
| 72 | + × Tsconfig not found /monorepo/tsconfig.base.json |
| 73 | +``` |
| 74 | + |
| 75 | +Every app's `tsconfig.json` is three lines: |
| 76 | + |
| 77 | +```json |
| 78 | +{ "extends": "../../tsconfig.base.json", "include": [...], "exclude": [...] } |
| 79 | +``` |
| 80 | + |
| 81 | +The Dockerfiles copy `package.json`, the lockfile, the workspace manifest, |
| 82 | +`packages/`, `apps/` and `shell/`. They never copied the file every one of those |
| 83 | +tsconfigs extends. `pnpm install` succeeds — it doesn't care. The failure lands |
| 84 | +one layer later, at `rsbuild build`, and reads as a TypeScript problem rather |
| 85 | +than a missing-file problem. |
| 86 | + |
| 87 | +## Under the Hood: the split that hid it |
| 88 | + |
| 89 | +The interesting part is that **the ten service Dockerfiles were fine**. Only the |
| 90 | +six frontends were broken, and the reason is structural rather than careless. |
| 91 | + |
| 92 | +| | tsconfig shape | Build context | |
| 93 | +|---|---|---| |
| 94 | +| `services/*` | own self-contained `tsconfig.json`, extends nothing | its own directory | |
| 95 | +| `shell`, `apps/*` | three-line stub extending `../../tsconfig.base.json` | the repo root | |
| 96 | + |
| 97 | +Services copy `tsconfig.json` and are done — that file is complete on its own. |
| 98 | +The frontends inherit from a root file that lives outside anything they copy. So |
| 99 | +"does this Dockerfile copy a tsconfig?" returns **yes** for all sixteen, and the |
| 100 | +six that are broken are broken by what they *don't* reach for. |
| 101 | + |
| 102 | +A per-file audit finds nothing. Only building one finds it. |
| 103 | + |
| 104 | +## What this says about the setup |
| 105 | + |
| 106 | +Two things worth fixing beyond the one line: |
| 107 | + |
| 108 | +**A failed deploy is invisible.** The previous container keeps serving, the |
| 109 | +health check keeps passing, and the dashboard is the only place the failure is |
| 110 | +recorded. Twelve days is how long that stayed unnoticed while active work |
| 111 | +shipped daily. |
| 112 | + |
| 113 | +**Nothing builds the containers in CI.** `pnpm build` on a developer machine |
| 114 | +works, because `tsconfig.base.json` is right there on disk. The Docker build is |
| 115 | +the only place the missing COPY exists, and it only ran in production. A build |
| 116 | +of each image on PR would have caught this the day it was introduced. |
| 117 | + |
| 118 | +Neither is fixed here. Both are logged. |
| 119 | + |
| 120 | +## What's Next |
| 121 | + |
| 122 | +The immediate consequence: the deployed frontends are twelve days stale. Once |
| 123 | +this merges, the next deploy will be the first successful one since August 3rd — |
| 124 | +so it will ship twelve days of accumulated change at once. Worth watching rather |
| 125 | +than assuming. |
0 commit comments