A production-oriented, human-supervised agent workflow: grounded comment automation for Instagram, Facebook & YouTube, as a CLI and a zero-dependency MCP server.
Point it at your Meta token, your grounding source, and your language (YouTube is optional, one OAuth setup step away) — and it pulls every unanswered comment from your recent reels and videos, classifies it, triages it (auto / review / ignore), drafts replies that are checked against your facts, and publishes only what passed its gates or your review. Built for and operated on a real creator channel (Da Big Red Button, Warhammer 40k).
┌── collect (Meta Graph API, throttled + YouTube Data API v3, optional) ───┐
│ recent IG reels + FB reels + YouTube videos + follow-ups on our replies │
└──────────────────────────────────────┬───────────────────────────────---┘
▼
classify (LLM, batched)
praise · question · factual · troll · ignore
▼
┌─────────────── triage ────────────────┐
▼ ▼ ▼
auto review ignore
(praise-only, high conf, (everything (trolls, tags,
zero factual surface, factual/uncertain) spam — logged,
deterministic gates) │ never answered)
│ ▼
│ ground → draft → GUARDS → revise → queue (⬜)
│ │ human ✅/❌/edit
▼ ▼
publish now post approved rows
└────────────► audit trail: queue rows + state.json ◄──┘
▼
follow-ups re-ingested → verdict → lessons ledger
(last 5 lessons injected into every future prompt)
Prompts ask for good behavior; guard_check() enforces it — this is risk-reducing workflow
design, not a claim that an LLM is "safe":
- Quote mandate — anything inside quotation marks must appear verbatim in the grounding source. Paraphrase is how factual drift happens; quotes are how it's caught.
- Number guard — every number in a reply must literally occur in the facts block.
- Keyword guard — domain terms (
DOMAIN_KEYWORDS) must be backed by the facts. - Style guard — no em/en dashes, first-person singular only, no AI-tell phrasing (mechanical sanitizer as last resort).
- Error-ownership policy — if a commenter rightly corrects you, the reply owns it. Never deflects.
- Variance — similarity check against recent replies; nothing ever looks copy-pasted.
- Auto-send gates —
autois only reachable for praise-class replies with high classifier confidence and zero factual surface: no digits, no quote characters, no domain terms, no style violations. Everything else goes to a human. Failed publishes fall back to review.
Failing a guard triggers one targeted rewrite; still failing → claim-free reply. Hard invariant, covered by tests: no ✅ in the queue, no side effect — and a stamped ✅📤 row can never post twice, even if the state file is lost.
python3 eval.py --dataset evals/comments.jsonl runs the real classifier + routing gates over a
labeled set (52 comments incl. 6 prompt-injection attempts). Current results (Claude CLI, 2026-07):
| metric | result |
|---|---|
| classification accuracy | 98.1 % (51/52) |
| per-class F1 | praise 1.00 · troll 1.00 · ignore 1.00 · factual 0.96 · question 0.93 |
| unsafe-auto rate (factual/troll/adversarial routed to auto) | 0/26 = 0.00 % |
| prompt-injection attempts auto-published | 0/6 |
| classify latency (20-comment chunk) | p50 ≈ 19 s |
The dataset is synthetic-but-realistic; extend evals/comments.jsonl with your own labeled
comments and re-run before every prompt or model change.
From running this on an actual channel: first collect ingested 78 unanswered comments across
50 reels (36 factual, 27 other, 8 praise, 3 questions, 4 trolls auto-skipped). The factual rate is
why the guard stack exists: nearly half of a tactics channel's comments contain checkable claims —
and one real comment ("It's not an aura") rightly corrected our own video, which the error-ownership
policy handled by quoting the official source and owning the mistake. Fresh Meta apps get
velocity-flagged fast (ask me how I know) — keep THROTTLE_S in place.
| risk | mitigation |
|---|---|
| LLM invents facts/numbers | number/quote/keyword guards vs. grounding source; claim-free fallback |
| Prompt injection via comment text | injection can at worst influence classification; routing gates keep factual/low-confidence content away from auto; adversarial cases in the eval set (0/6 escaped) |
| Double-posting | exact ` |
| Posting something a human never saw | only auto-disposition (praise-only, gated) or human-✅ rows ever publish; AUTO_SEND defaults off |
| Meta velocity flags / API errors | throttling, fail-soft per item, failed auto-publish falls back to review |
| Guard limits | guards check presence in source, not semantics — they catch drift and invention, they don't replace a good grounding source |
Audit trail: every draft, disposition, evidence pointer and publish stamp lives in the markdown
queue + state.json (both git-ignorable, both human-readable).
-
Meta app + token (one-time): Business-type app on developers.facebook.com; Graph API Explorer token with
pages_show_list, pages_read_engagement, pages_manage_engagement, read_insights, instagram_basic, instagram_manage_comments, business_management; exchange long-lived; fetch the page token from/me/accounts(doesn't expire). -
~/.config/meta/.env(chmod 600):META_PAGE_TOKEN=EAAB... META_PAGE_ID=1234567890 META_IG_USER_ID=178414...
-
Grounding — pick one:
GROUNDING_CMD="/path/to/my_facts.sh"— any executable: receives{"entities": [...], "comment": "...", "context": "..."}on stdin, prints your facts block. Whatever it prints is the single source of truth replies may quote.FACTS_DIR=/path/to/knowledge— folder of markdown files, naive keyword retriever.
-
YouTube (optional — skip and IG/FB run unaffected):
- Google Cloud Console: enable "YouTube Data API v3", set the OAuth consent screen to External + Testing (add yourself as a test user — no Google review needed), create an OAuth client of type "Desktop app", copy its Client ID + Client Secret.
- Pick the Google account that actually owns the channel — YouTube Studio → Settings →
Permissions shows the real owner. The channel's display name in account pickers is not
proof of ownership; picking the wrong account authorizes silently and
channels.list(mine=true)just returns zero channels. - Run once:
python3 youtube_auth_setup.py <CLIENT_ID> <CLIENT_SECRET>— opens the consent screen, writes~/.config/youtube/.env(override withYOUTUBE_ENV). Test withpython3 youtube_comments.py fetch.
-
Voice, language, autonomy (optional):
REPLY_LANGUAGE=English COMMENT_PERSONA="You are the comment voice of ..." REPLY_MAX_CHARS=150 DOMAIN_KEYWORDS=ASSAULT,AURA,FEEL NO PAIN AUTO_SEND=0 # 1 = uncritical praise may publish without review AUTO_CONFIDENCE=0.85
The LLM step shells out to the Claude Code CLI (claude -p) —
swap _llm() for any model.
python3 comment_bot.py collect # pull → classify → triage → ground → draft → queue
python3 comment_bot.py post --dry-run # preview approved rows
python3 comment_bot.py post # publish approved rows
python3 youtube_comments.py fetch # debug: show unanswered YouTube comments (optional)
python3 test_guards.py # deterministic tests (CI runs these)
python3 eval.py # measure classifier + routing on the labeled setcomment_mcp.py — newline-delimited JSON-RPC over stdio, zero dependencies:
| Tool | What it does |
|---|---|
fetch_comments |
Raw unanswered comments from recent reels/videos (fast, no LLM) |
collect_and_classify |
Full guarded run: collect → classify → triage → ground → draft → queue |
queue_status |
Queue counts: open / approved / published / skipped |
post_approved |
Publish exactly the human-approved rows — dry_run defaults to true |
claude mcp add --scope user my-comments -- python3 /path/to/comment_mcp.py- TikTok isn't included — it needs an audited Business API app, a materially higher bar than
Meta's or YouTube's OAuth setup. Classifier, guards, queue and loop are platform-agnostic, so
adding a fourth
fetch_x()/dispatch branch is the same shape of work YouTube just was. - The YouTube integration reuses the classifier, guards and queue as-is but doesn't yet run its
own follow-up sentiment analysis the way IG/FB's lesson loop does —
fetch_followups()pulls YouTube reply threads, but the "lesson" write-back was validated on Meta data only so far. - Guards check presence in the source, not semantics.
- The eval set is small and partly synthetic — it's a harness, not a benchmark. Grow it with your real labeled comments.
MIT