Skip to content

Commit b7fdfd7

Browse files
benmfzenclaude
andcommitted
Record scale-out design as ADR-002 (proposed, not implemented); link from README
Documents, precisely, what removing the single-operator assumptions would take — locking, plan versioning, durable queue, tenant isolation, audit log, human-approval placement — as an explicit unbuilt decision rather than leaving the boundary implicit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 501bad4 commit b7fdfd7

2 files changed

Lines changed: 151 additions & 0 deletions

File tree

README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,12 @@ story, including what operation taught the design:
218218
- **`plan.json` writes are not atomic** (single-operator assumption; a torn read
219219
falls back to config defaults rather than crashing).
220220

221+
What all of the single-operator assumptions above would concretely take to remove —
222+
locking, plan versioning, a durable queue, tenant isolation, a real audit log, and
223+
where human approval for plan changes should live at scale — is recorded as an
224+
explicit, unimplemented architecture decision in
225+
[ADR-002](docs/adr/002-scale-out-design.md), not left implicit.
226+
221227
## Non-goals
222228

223229
No content generation, no analytics dashboard, no live posting. This is the thin,

docs/adr/002-scale-out-design.md

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
# ADR-002: Scale-out design — from single-operator to multi-team
2+
3+
**Status:** proposed (not implemented) · **Date:** 2026-07-16
4+
5+
## Context
6+
7+
postpeer-pilot is built, tested, and running as a **single-operator tool**: one
8+
channel, one config directory, one process at a time. That scope is deliberate — see
9+
the README's "Design notes & known limits" and [ADR-001](001-deterministic-planner.md).
10+
But it's worth recording, precisely, what would have to change for a shared,
11+
multi-team, or multi-tenant deployment — so the boundary between "works today" and
12+
"would need work" is an explicit decision, not something a reader has to infer from
13+
absence.
14+
15+
This ADR documents the decision **not to build** the items below yet, and what each
16+
one would concretely require if the day comes. Nothing here is implemented.
17+
18+
## The six gaps, and what closing each one actually requires
19+
20+
### 1. Distributed locking around slot assignment
21+
22+
**Today:** `plan.free_slots()` re-reads live Postpeer occupancy on every call (tested,
23+
invariant I7), so a *second, later* scheduler run never double-books a slot the first
24+
run already has on Postpeer. But two runs that read occupancy **at the same instant**
25+
— before either has written anything back — can both see the same free slot. This is
26+
explicitly called out in the README as acceptable for one operator and unsafe for
27+
concurrent ones.
28+
29+
**What closing it requires:** a lock scoped to `(account_id, date)`, held for the
30+
read-check-write window around slot assignment. Not a generic distributed lock
31+
service — the contention unit is narrow (one account, one day), so something like a
32+
row in a shared table with a conditional write (`UPDATE ... WHERE slot IS NULL`) or a
33+
Postgres advisory lock keyed on `hash(account_id, date)` is enough. A general-purpose
34+
lock manager (Redis/Zookeeper) would be solving a bigger problem than this one has.
35+
36+
### 2. Atomic `plan.json` writes and versioning
37+
38+
**Today:** `planner.apply()` calls `PLAN_FILE.write_text(...)` directly
39+
(`postpeer_pilot/planner.py:124`) — a torn read during a crash mid-write falls back to
40+
config defaults rather than corrupting state, which is safe but silent: there's no
41+
record of what the plan *was* before the write, only what it is now.
42+
43+
**What closing it requires:** write-to-temp-then-`os.replace()` for atomicity (cheap,
44+
should probably happen regardless of scale), plus a `plan_history.jsonl` — one
45+
append-only line per applied plan (proposal, stats, whether forced, timestamp)
46+
instead of overwriting in place. That turns "what changed and why" from "not
47+
recorded" into "queryable," which matters the moment more than one person is asking
48+
"why did Tuesday's volume change last week."
49+
50+
### 3. A durable job queue instead of direct, synchronous API calls
51+
52+
**Today:** `schedule_video` uploads and schedules inline, in the calling process; a
53+
crash mid-batch leaves whatever was already scheduled recorded in the ledger
54+
(idempotent re-runs pick up where it left off — tested in
55+
`tests/test_reliability.py`), but there's no persistent, inspectable "N videos are
56+
queued to be scheduled" state between the request and completion.
57+
58+
**What closing it requires:** a real queue (even SQLite-backed would do — this
59+
doesn't need Kafka) so that `schedule_video` enqueues and returns immediately, a
60+
worker processes it with the existing retry policy, and `queue_status` can report
61+
in-flight work, not just completed slots. This is the change most tied to *why* you'd
62+
scale: a single MCP call blocking on N sequential uploads is fine for a handful of
63+
videos and wrong for a content team's daily batch.
64+
65+
### 4. Idempotency at the API boundary — but not the way that phrase usually means
66+
67+
The generic version of this advice is "add idempotency keys to your API calls." That
68+
doesn't apply cleanly here: **Postpeer's API has no idempotency-key mechanism to send
69+
one to** (see the README's "Postpeer API quirks" — there's no reservation primitive,
70+
full stop). Today's idempotency is built entirely in the layer above the API: the
71+
local ledger records what this tool has scheduled, keyed by post id once one exists,
72+
and a re-run skips ledger entries in future slots (`allow_duplicate` opts out).
73+
74+
**What scaling it requires:** the same pattern, made crash-safe and shared — a
75+
ledger entry written *before* the upload call, in a `pending` state, so a second
76+
worker (or a retry after a crash between "media uploaded" and "post created") can see
77+
the in-flight attempt and either wait or resume from the orphaned upload
78+
(`orphaned_upload` already surfaces the reusable URL) instead of re-uploading. The
79+
idempotency key, in other words, needs to be **ours**, generated before the first
80+
network call, not Postpeer's.
81+
82+
### 5. A structured, queryable audit log
83+
84+
**Today:** `scheduled.jsonl` is an audit trail for *scheduling* decisions only —
85+
what was scheduled, when, under what series. Plan changes live inside `plan.json`'s
86+
`basis` field (only the most recent one, per gap 2). There's no unified log of "who
87+
(which operator, which agent session) asked for what, and what did the tool layer
88+
actually do" across both tools.
89+
90+
**What closing it requires:** one append-only, structured log (JSONL is enough,
91+
doesn't need a database) written by the tool layer itself — not the MCP client — for
92+
every call: `{ts, tool, args, actor, result, refused_reason?}`. `actor` matters more
93+
here than in most audit logs: the whole point of this tool is that an agent can act,
94+
so the log needs to distinguish "operator ran this from the CLI" from "Claude called
95+
this via MCP" from (eventually) "operator B's agent called this."
96+
97+
### 6. Tenant isolation
98+
99+
**Today:** one `POSTPEER_PILOT_HOME` (default `~/.config/postpeer-pilot`), one
100+
`accounts.json`, one `plan.json`, one ledger. Nothing in the code path takes a tenant
101+
identifier — `config.HOME` is a module-level constant.
102+
103+
**What closing it requires:** the actual work here is smaller than it sounds, because
104+
the data model already partitions cleanly by account. Threading a `tenant_id`
105+
through `config.HOME`-equivalent lookups (or moving from files to a
106+
tenant-keyed table) turns one directory into N; the planner, gates, and invariants
107+
don't change at all, because they were already written per-account. The real design
108+
question isn't storage, it's **auth**: who can call `schedule_video` for tenant A,
109+
and how does an MCP server (currently one process, one identity) authenticate a
110+
caller as acting on behalf of a specific tenant. That's the part worth designing
111+
before writing any tenant-partitioning code.
112+
113+
## Human approval for plan changes, at scale
114+
115+
`planner.apply(force=True)` already exists as an escape hatch for a human who wants
116+
to override the damping guard on thin data — but `server.py` never passes it
117+
(`return planner.apply(force=False) if args.get("apply") else planner.propose()`),
118+
so **no MCP tool call can reach it today**. That's not an accident to fix; it's a
119+
property worth keeping deliberately. At scale, the right shape isn't to expose
120+
`force` as a tool parameter an agent can flip — it's a **separate, explicitly-named
121+
tool** (e.g. `plan_force_apply`) that a human calls directly, outside agent-driven
122+
flows, logged with `forced: true` and a required reason string. The override should
123+
stay reachable by a human and unreachable as a side effect of an agent being asked
124+
to "just make it work."
125+
126+
## Decision
127+
128+
None of the six gaps above are implemented. This ADR is the artifact instead of the
129+
code: a single-operator tool that is honest about exactly what "single-operator"
130+
excludes, and what each exclusion would concretely cost to remove, so that decision
131+
is made once, on purpose, rather than discovered piecemeal under load.
132+
133+
## Consequences
134+
135+
- A reader (or a reviewer) can evaluate "is this production-ready for a team" without
136+
guessing — the answer is in this file, itemized, not implied by the absence of a
137+
section.
138+
- If any of these become real requirements, the corresponding invariant tests
139+
(`tests/test_invariants.py`) are the right place to encode the *new* guarantees
140+
(e.g. "two concurrent scheduler runs never both win the same slot") before writing
141+
the implementation — the same discipline ADR-001 already applies to the planner.
142+
- Building all six pre-emptively would be over-engineering for the tool's actual
143+
current usage (one channel, one operator, 85+ posts published without a single
144+
double-book). The trigger for revisiting this ADR is a second concurrent operator
145+
or tenant, not a hypothetical one.

0 commit comments

Comments
 (0)