Skip to content

Commit b5832c5

Browse files
docs(specs): scheduled purge runs on production only; purge takes private data
v0.10.2's sandbox ran the sweep against the published branch it shares with production, and purge never deleted private data. Only the deployment with CFP_SCHEDULED_PURGE=true sweeps; purge deletes the person's private profile, legacy password, reset tokens, claim requests and sessions in the same transaction; an admin endpoint clears private data left by earlier purges, counts first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TGqwPjt3v4eMfiHpPmCKnR
1 parent d174d7e commit b5832c5

5 files changed

Lines changed: 118 additions & 2 deletions

File tree

‎docs/operations/deploy.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -273,6 +273,7 @@ comments. Production pod gets these mounted:
273273
| `CFP_DATA_RELOAD_SECRET` | **Secret** (`codeforphilly-secrets`) | Shared bearer token for the hot-reload webhook; when unset the `/api/_internal/reload-data` endpoint returns 503. Must match the `CFP_DATA_RELOAD_SECRET` Actions secret in the data repo. See [runbook.md](runbook.md#hot-reload-webhook). |
274274
| `CFP_WEB_DIST_PATH` | ConfigMap | `/app/apps/web/dist` |
275275
| `CFP_SITE_HOST` | ConfigMap (patched per cluster) | Public-facing host: `next.codeforphilly.org` in prod until cutover, `codeforphilly.org` after; `next-v2.codeforphilly.org` in sandbox. Drives the markdown renderer's external-link transform, canonical URLs in email, and the SAML IdP metadata's `SingleSignOnService` endpoint URLs. |
276+
| `CFP_SCHEDULED_PURGE` | ConfigMap (patched per cluster) | `"true"` **only in cfp-live-cluster**; unset (`"false"`) everywhere else. Turns on the daily [scheduled spam purge](../../specs/behaviors/person-lifecycle.md#scheduled-spam-purge). Sandbox and production share `published`, so a second deployment with it on would be a second destructive writer on production data. |
276277
| `SAML_ENTITY_ID` | ConfigMap | Optional. Stable SAML IdP entity ID / assertion `Issuer` (default `https://codeforphilly.org/api/saml/slack/metadata`). Leave unset everywhere Slack should keep trusting the production IdP identity — it deliberately does **not** follow `CFP_SITE_HOST`, so flipping the host at cutover doesn't require re-registering with Slack. Only set it when standing up a separate IdP registration. See [specs/api/saml.md](../../specs/api/saml.md#idp-identity-and-hosts). |
277278
| `POSTMARK_SERVER_TOKEN` | **Secret** (`codeforphilly-secrets`) | Postmark server API token for outbound notifications. When unset, the email notifier falls back to a no-op LoggingNotifier — convenient for dev + tests but means no real emails go out. |
278279
| `POSTMARK_MESSAGE_STREAM` | ConfigMap | Postmark message stream for outbound mail (default `outbound`). Must exist on the server the token belongs to. |

‎docs/operations/runbook.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -204,6 +204,33 @@ The helper script resolves the current pod by label selector, so it survives res
204204

205205
**Read-only by design** — `git upload-pack` only serves fetch; pushing back to the pod would bypass gitsheets + in-memory state and fight the push daemon. Pull what you need to your local clone, reason about it there, then push to `origin` if appropriate.
206206

207+
## Private data left behind by purges
208+
209+
Purge deletes a person's private data (profile, legacy password, reset tokens,
210+
claim requests, sessions) in the same transaction since v0.10.3. People purged
211+
or pruned before that still have private data. An administrator clears it from
212+
the running API — never with an out-of-process script, because the API holds
213+
the private store in memory and would write the old files back:
214+
215+
```js
216+
// In the browser console on the site, signed in as an administrator.
217+
const run = (apply) => fetch('/api/admin/private-data/purge-orphans', {
218+
method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ apply }),
219+
}).then((r) => r.json()).then((j) => console.log(j.data));
220+
run(false); // counts only — check them
221+
run(true); // deletes; cannot be undone (the private store is not versioned)
222+
```
223+
224+
It answers counts only, never emails. Spec:
225+
[api/moderation.md](../../specs/api/moderation.md#post-apiadminprivate-datapurge-orphans).
226+
227+
## Scheduled spam purge runs on production only
228+
229+
`CFP_SCHEDULED_PURGE=true` is set in cfp-live-cluster alone
230+
([deploy.md](deploy.md)). Sandbox shares `published`, so a sandbox pod with
231+
the flag on would delete production data. A pod logs
232+
`scheduled spam purge disabled` at boot when the flag is off.
233+
207234
## Helpful commands
208235

209236
```bash
Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
---
2+
status: in-progress
3+
depends: [scheduled-spam-purge]
4+
specs:
5+
- specs/behaviors/person-lifecycle.md
6+
- specs/api/moderation.md
7+
issues: []
8+
---
9+
10+
# Plan: Gate the scheduled purge to production; purge private data too
11+
12+
## Scope
13+
14+
v0.10.2 shipped the scheduled spam purge early (2026-10-15). The sandbox ran
15+
the sweep against `published`, which it shares with production, and purged
16+
27 spam-voted accounts there. The accounts were the right ones, but two
17+
flaws showed: every pod runs the sweep, and purge never touched the private
18+
store, so purged people's private profiles stayed behind.
19+
20+
**In:** `CFP_SCHEDULED_PURGE` (default off; on only in cfp-live-cluster), and
21+
a pod with it off never sweeps and says so at boot; Purge (button and sweep)
22+
deletes private profile, legacy password, reset tokens, claim requests and
23+
session metadata in the same transaction; `POST
24+
/api/admin/private-data/purge-orphans` (admin, `apply` false/true, counts
25+
only) for private data left by earlier purges and prunes.
26+
27+
**Out:** an out-of-process script for the cleanup (the API's in-memory
28+
private store would write the old data back).
29+
30+
## Implements
31+
32+
- [behaviors/person-lifecycle.md](../specs/behaviors/person-lifecycle.md#purge-cascading-hard-delete-admin-only)
33+
- [api/moderation.md](../specs/api/moderation.md#post-apiadminprivate-datapurge-orphans)
34+
- docs/operations/deploy.md (env table), runbook.md.
35+
36+
## Approach
37+
38+
1. `PrivateStoreTx.deletePersonData(personId)`, staged by `store.transact`
39+
like the other private writes; `purgePeople` calls it per person.
40+
2. `SessionMetadataStore.removeAllFor(personId)` after the commit.
41+
3. `scheduledPurgeEnabled(config)` gates the plugin.
42+
4. The orphan cleanup runs under `publicRepo.withLock`, so a sign-up that
43+
has written its private profile but not its public record is never
44+
counted.
45+
46+
## Validation
47+
48+
- [ ] `npm run type-check && npm run lint && npm test` clean.
49+
- [ ] Tests: the gate; `deletePersonData` persisted to disk, other people
50+
untouched; Purge deletes the private profile; the cleanup is
51+
admin-only, counts without emails, deletes on `apply`, then counts 0.
52+
53+
## Risks / unknowns
54+
55+
- `apply: true` can't be undone; the runbook says to count first.
56+
57+
## Notes
58+
59+
## Follow-ups

‎specs/api/moderation.md‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,3 +156,32 @@ as the caller, so the vote carries the voter's pseudonymous git identity.
156156
deactivate side effect and its reversal rule.
157157
- [api/people.md](people.md) — `deactivate`, `reactivate`, `purge`, and the
158158
staff-only fields reused here.
159+
160+
## POST /api/admin/private-data/purge-orphans
161+
162+
Administrator only (`404` for anyone else, like the rest of `/api/admin`).
163+
Deletes the private data of every person id the private store knows that has
164+
no `people` record: private profile, legacy password, password-reset tokens,
165+
account-claim requests naming them, and session metadata — what
166+
[Purge](../behaviors/person-lifecycle.md#purge-cascading-hard-delete-admin-only)
167+
now deletes in its own transaction, for everyone purged or pruned before it
168+
did.
169+
170+
### Request
171+
172+
```json
173+
{ "apply": false }
174+
```
175+
176+
`apply` is required: `false` only counts; `true` deletes and returns the same
177+
counts. Run under the write lock, so a member mid-sign-up (private profile
178+
written, public record not yet) is never counted.
179+
180+
### Response — 200
181+
182+
```json
183+
{ "success": true, "data": { "applied": false, "orphanPeople": 27, "profiles": 27, "legacyPasswords": 0, "claimRequests": 0, "sessions": 0 } }
184+
```
185+
186+
Counts only — never emails, names or ids. The private store is not versioned,
187+
so `apply: true` cannot be undone; run `false` first.

‎specs/behaviors/person-lifecycle.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ The privacy / self-removal path — members should be able to remove themselves;
2424
The garbage-collection path for spam — the runtime sibling of the offline spam-prune (behaviors/spam-exclusion.md).
2525

2626
- **Who:** administrators only.
27-
- **Mechanism:** one write-mutex transaction that hard-deletes: the `people` record; their `project-membership`; their `help-wanted-interest`; their person `tag-assignment`; AND their authored `project-update`, `project-buzz`, and `blog-post` records (unlike the prune which nulls authorId — purge DELETES the content, it's garbage).
27+
- **Mechanism:** one write-mutex transaction that hard-deletes: the `people` record; their `project-membership`; their `help-wanted-interest`; their person `tag-assignment`; AND their authored `project-update`, `project-buzz`, and `blog-post` records (unlike the prune which nulls authorId — purge DELETES the content, it's garbage). Their **private** data goes in the same transaction: private profile, legacy password credential, password-reset tokens, account-claim requests naming them, and their session metadata. A purged person leaves nothing behind in either store.
2828
- **Projects only they touched go too.** In the same transaction, purge hard-deletes every project the person created that nobody else has touched — spam accounts create spam projects, and a project left behind with no members is a soft-deleted husk an admin can't clear. A project qualifies when both hold:
2929
1. **Created by the person:** `createdById` is the person; for a project with no `createdById` (imported from laddr, or created before the field existed), the person is its `maintainerId`.
3030
2. **No one else touched it:** every membership is the person's, every update was authored by them, every buzz posted by them, every help-wanted role posted by them with no interest from anyone else, and no RSVP from anyone else names the project.
@@ -38,7 +38,7 @@ The garbage-collection path for spam — the runtime sibling of the offline spam
3838

3939
A `spam` vote on the admin members screen deactivates the person at once; three days later, if nobody has disagreed, the site purges them on its own. Staff shouldn't have to come back and press a second button for every spam account, and three days is long enough for another staff member to look and object.
4040

41-
- **When:** an in-process job runs once a day (and once shortly after boot).
41+
- **When:** an in-process job runs once a day (and once shortly after boot) — **only on the deployment where `CFP_SCHEDULED_PURGE=true`, which is production alone.** Sandbox and production serve the same `published` branch, so any pod that runs the sweep is deleting production data; exactly one may. With the flag off the pod never sweeps and logs that at boot.
4242
- **Who it takes:** every person who
4343
1. has at least one human `spam` vote, and every human vote on them is `spam` — a `legit` or `uncertain` vote from anyone holds them back;
4444
2. got their newest human vote at least **3 days** ago;

0 commit comments

Comments
 (0)