Skip to content

Latest commit

 

History

History
31 lines (20 loc) · 7.38 KB

File metadata and controls

31 lines (20 loc) · 7.38 KB

CRM/mass-mailer subscriber tables get their own carry/empty gate, default empty

ADR-0014 carved form-submission tables out of the silently-carried content split and gave them a standalone carry/empty gate, default empty, because they are non-regenerable and full of third-party personal data. A recognised mass-mailer's or CRM's subscriber store is the same kind of data — real names and email addresses of people who never developed against this site — and carries an additional, sharper hazard the form tables do not: with the mail=live and cron-runs defaults (ADR-0009), a CRM's standing funnel/automation sequences over a carried subscriber list can mail those real people from the developer's laptop. The mass-send valve does not catch this: it watches for a poised bulk campaign, a campaign-shaped risk, and says nothing about standing automations over the subscriber data itself. So a subscriber list carried in full is a live hazard the valve is structurally blind to.

The live smoke test of 2026-07-23 (run 2, finding R2-5) made this concrete: classify.py classed 17 wp_fc_* tables as full and the transfer carried 584 real FluentCRM subscribers (plus 1319 subscriber_meta, 93 subscriber_notes, 1376 subscriber_pivot, 1118 campaign_emails rows). Nothing landed in the empty split. The run was safe only because the operator explicitly chose mail=capture (Mailpit); the default combination mail=live + cron=run is the dangerous one, and it is the default.

So crm_subscribers is a sixth classification family, a sibling to user_submissions: matched the same way as the operational four and the user-submission family (after prefix-stripping, exact-or-startswith, first match wins — ADR-0005's pattern), kept structurally separate in classify.py, and given its own standalone decision in resolve_plan.py — a plain carry/empty gate, default empty, sitting immediately after user_submissions in the ordered decision list. Default empty is the same privacy minimisation ADR-0014 settled: the common case is a developer who needs a working local copy, not a copy of every subscriber's address, and the gate is the way back for the operator who genuinely needs the real list (debugging a funnel locally). The choice persists in the saved plan under crm_subscribers and replays with every other settled decision.

Definitions vs persons — the line the family draws

The family gates the tables that hold persons, not the tables that hold definitions. Only these carry third-party addresses and only these are the automation's ammunition:

  • FluentCRMfc_subscribers and its family (fc_subscriber_meta, fc_subscriber_pivot, fc_subscriber_notes), fc_campaign_emails, fc_campaign_url_metrics, fc_funnel_subscribers, fc_funnel_metrics.
  • MailPoetmailpoet_subscribers and its siblings.
  • Mailstermailster_subscribers and its siblings, mailster_queue.
  • The Newsletter Plugin — the bare newsletter subscriber table (matched exact-only) and its per-person delivery/tracking tables newsletter_sent, newsletter_stats, newsletter_user_logs. Its newsletter_emails table holds campaign bodies, not addresses, and carries in full as a definition (below), which is why newsletter is gated by exact match rather than a prefix that would also sweep newsletter_emails.

Deliberately not matched — carried in full as site config, exactly like posts and options: fc_campaigns, fc_funnels, fc_funnel_sequences, fc_lists, fc_tags, fc_terms, fc_term_relations, fc_meta, fc_url_stores, and Newsletter's newsletter_emails. These make the site work locally and hold no third-party addresses. The split is not arbitrary: emptying fc_funnel_subscribers is precisely what stops a local funnel automation from having anyone to mail, while keeping fc_funnels and fc_funnel_sequences means the funnel still exists to build and test against. Draw the line at the person rows and the copy is both usable and safe by default; draw it at the whole plugin and the copy loses its CRM configuration for no privacy gain.

Every engine in the mass-send recognition registry (bootstrap_parse.py's MAILER_ENGINES: fluentcrm / mailpoet / newsletter) has subscriber patterns here, so a recognised mailer's address store can never carry in full — a test (test_every_recognised_mailer_engine_has_a_gated_subscriber_store) pins that direction against drift. Mailster is gated in addition: it is a known on-site mailer with no poised-scan entry in MAILER_ENGINES (its send model needs no bulk-campaign scan), so the gated set is a superset of the recognised engines, never a subset. The registry is additive like USER_SUBMISSION_TABLE_PATTERNS: an unrecognised CRM's subscriber tables carry in full until a pattern is added, the same explicit-recognition posture the whole engine takes.

The operator filed this from the second live smoke test with the fix proposal recorded on issue #39; recording it here settles the policy.

Consequences

  • classify.py's table_category() tags a matched table crm_subscribers, distinct from the four operational categories and from user_submissions, even though the table still lands in the same full/empty split structure — the category tag is the distinction that matters downstream, not a different output shape.
  • The gate is always presented (or walked in replay), independent of whether the discovered site actually runs a recognised CRM — the same posture user_submissions, media_originals, and heavy_blobs already take for decisions that may be a no-op on a given site.
  • A saved carry is a deliberate, sticky per-site choice, not re-evaluated against live state on replay, mirroring every other coarse gate except mail's mass-send valve.
  • The two privacy gates are independent: resolve_plan.py folds each gate's resolved choice into db_table_content over its own category only, so one gate's carry never drags the other's tables out of the empty split. Both the resolved value (this run's actual choice) and the recommendation (the saved/live/built-in layers, never the this-run answer) are folded, so a db_table_content gate walked before the operator answers still shows a split consistent with whatever is already on record — the same double-fold contract ADR-0014 established for user_submissions. The pack script's content/empty lists and the dump-sanity empty-set read db_table_content, never the gate value directly, so they see the carry without any change of their own.
  • The pattern set covers all three recognised on-site engines (FluentCRM, MailPoet, The Newsletter Plugin) plus Mailster. A CRM outside this set (or a cloud sender that never blasts from the local copy and so has no on-site subscriber table) is not yet recognised and its tables carry in full like any other content until a pattern is added.
  • table_category() matches on the table name after the site's own prefix, not after any multisite sub-site prefix — the same pre-existing limitation the operational and user-submission families carry, and currently moot: multisite is out of scope for the whole engine (docs/spec.md, Out of Scope). The privacy rationale would apply with equal force to a sub-site's subscriber tables were multisite ever brought into scope.