Instruction text the Neotoma MCP server sends to clients so agents know how to use MCP actions. The server loads this file at runtime and ships the first fenced code block verbatim; edits here take effect without code changes in src/server.ts. For user-facing docs see docs/specs/MCP_SPEC.md and docs/developer/mcp_overview.md.
The block below is loaded and sent to MCP clients as interaction instructions. It is organized into labelled sections ([TURN LIFECYCLE], [STORE RECIPES], etc.) in the order agents need during a turn. Each section contains one rule per line where possible; mega-rules are split into atomic sub-rules.
[TURN LIFECYCLE]
MUST complete the turn in this order: (1) bounded retrieval, (2) user-phase store, (3) other MCP calls or host tool use, (4) compose assistant reply, (5) closing store of the assistant reply. Do not respond before steps 1 and 2 are complete, and do not end the turn before step 5 unless explicitly waived by the user or the turn produced no user-visible reply.
Short passes are NOT exempt: doc-only edits, `neotoma cli config`, single-file lint or copy tweaks, and other minimal turns still MUST run steps 1–5 in order. FORBIDDEN: skipping steps 1–2 before other tools, or skipping step 5 when there is a user-visible reply, because the turn seems quick, low-risk, non-chat, or "sync only". FORBIDDEN: rationalizing a store skip because the turn is "analysis-only", "in-session bookkeeping", "just an evaluation", or any similar framing — the audit trail is unconditional; no content category exempts a turn from the protocol.
Image-only messages are NOT exempt: when the user message body contains only an image or screenshot (no text), all five steps MUST still execute. Treat the visual content as the user message body for entity extraction and storage purposes. At minimum store the conversation entity, a user conversation_message entity (with content describing what the image shows), and any entities extractable from the image (tasks, events, deadlines, contacts, transactions visible in the screenshot). FORBIDDEN: skipping steps 1–5 because the user message has no text body.
Step 1 — Bounded retrieval: for entities implied by the user message, use retrieve_entity_by_identifier for concrete identifiers and retrieve_entities for category/list queries; use matches to reuse or link existing records. See [RETRIEVAL] for query-shape rules.
Step 2 — User-phase store: store the conversation, the current user message, and any entities implied by the message in ONE **`store`** call per [STORE RECIPES] user-phase. If the user attached a file, include the file in the same request and link per [STORE RECIPES] attachment. MUST NOT skip this for greetings or minimal messages.
Step 3 — Other actions: host IDE tools (read_file, apply_patch, run_terminal_cmd, grep, codebase_search, web fetch, and similar) and other MCP tools may run only after steps 1 and 2 are complete. Host tools are not exempt from store-first unless the user message is only an explicit waiver of persistence or a no-content system ping.
MCP availability detection: do not infer whether Neotoma is available in the live client session from a workspace `.mcp.json`, repo-local config file, or committed MCP manifest alone. If the host/session exposes Neotoma MCP tools or another active Neotoma transport, treat Neotoma as available and follow store-first.
Tool deregistration recovery: if a tool call fails with "tool 'Neotoma:X' is not registered" or "stale tool schema", the tool has fallen out of the host client's local cache — this is a client-side cache eviction, not a server-side removal. Recovery: call `tool_search` with the tool name or a relevant keyword (e.g. `tool_search("submit_issue")`) to reload it, then immediately retry the original call. Do not report this as a Neotoma server bug; it is expected behavior in long sessions with many tool loads.
Step 4 — Compose reply: synthesize the answer using retrieval results and tool output.
Step 5a — Closing store shape: after the user-visible reply is finalized, in the same turn, call **`store`** with entities = [ { entity_type: "conversation_message", role: "assistant", sender_kind: "assistant", content: "<exact assistant reply text as shown to the user>", turn_key: "{conversation_id}:{turn_id}:assistant" } ] and idempotency_key "conversation-{conversation_id}-{turn_id}-assistant-{suffix}". FORBIDDEN: reusing the user-phase `turn_key` (`{conversation_id}:{turn_id}` without the `:assistant` suffix) on the closing message — resolution targets the same `conversation_message` row as the user and corrupts the transcript; the server rejects this with `ERR_CONVERSATION_MESSAGE_ROLE_CONFLICT`.
Step 5b — Closing store relationship: then call create_relationship(PART_OF, source_entity_id = the assistant agent_message entity_id returned by this closing store, target_entity_id = the conversation entity_id returned by this turn's user-phase store). The target is the conversation the user message was linked to — not a new conversation.
Step 5b.1 — Reply-cited entity edges: in the same closing **`store`** call, include REFERS_TO relationships from the assistant agent_message to every entity the reply materially cites or produces during composition (synthesized notes/reports created this turn, and existing entities named in the reply text). Scope to cited-in-reply only; do NOT link every entity returned by retrieval. Skip when the REFERS_TO edge already exists from this same assistant agent_message to that target. Chat bookkeeping (conversation, agent_message) is excluded.
Step 5c — Closing store skip/forbidden: skip the closing store only when there is no user-visible assistant reply or the user explicitly waived persistence. FORBIDDEN: persisting the user message without storing the assistant reply when you did reply; ending the turn without the closing assistant store when you produced a user-visible reply.
Rapid-fire and edit-heavy sessions: the store-first rule applies even during many small, fast instructions (inline edits, DOM-path directives, quick fixes). If performance requires batching, store at minimum every 3–5 turns with all accumulated entities; never go an entire session without storing. Do not batch away closing assistant stores — each user-visible reply must still be stored via the closing action or an explicitly batched equivalent that includes every assistant reply text. Skipping Neotoma for an entire rapid-fire session is the highest-priority failure mode to avoid.
Backfill: if the user asks to store missed earlier turns, run /learn after a persistence gap, or run a compliance pass for prior turns, store every prior user and assistant turn that was missing, partially stored, or inaccurately stored. When the visible transcript is available, store each `conversation_message.content` exactly as shown to the user, including the initial user message and the final assistant reply text; do not substitute a planned draft, summary, or paraphrase. Use the same stable turn_key when correcting an inaccurate stored turn so the new observation repairs that turn instead of creating a disconnected duplicate. If the exact transcript is unavailable, store a concise note listing the unavailable turns and why exact content could not be recovered. Link all backfilled messages to the conversation in the same turn. Do not defer to a later session.
[DATA MODEL]
Neotoma separates **immutable bytes** from **semantic entities**. **Source** is a first-class primitive: rows in the `sources` table hold content-addressed raw bytes (SHA-256 deduplication per user). **Source** is **not** an `entity_type` — you will not see it in entity-type lists. **Interpretation** is an optional versioned extraction run tied to a source. **Observations** attach facts to entities and may carry `source_id` and `interpretation_id`. **Entity snapshots** are reducer output over observations. Canonical MCP tool **`store`** accepts entities-only, file-only (bytes → new source, no entity observations required in that call — use for archival / ingest-before-extract), or combined file + entities + optional `interpretation`. Chat- or tool-sourced structured writes often keep `interpretation_id` null. For file-derived field provenance, use combined `store` with `interpretation` (e.g. `source_ref: "unstructured"`) or `create_interpretation` on an existing `source_id`.
[GUEST ENTITY SUBMISSION]
Use **`submit_entity`** with `entity_type` plus a `fields` object for types that have an active `submission_config` row (operator-seeded only; the repo does not seed default submission rows). Use **`submit_issue`** when the GitHub-first issue orchestration path is required. Submitting an issue to a REMOTE operator instance needs NO prior identity: with no Bearer token, no AAuth signature and no guest token you may still submit, whenever that instance's `issue` guest access policy permits guest writes — the `guest_access_token` is an OUTPUT of the submit, not a precondition for it. Anonymity covers submission only; use the returned token for `add_issue_message` / `get_issue_status`. If a submit returns `AUTH_REQUIRED`, that operator has closed the inbox (`NEOTOMA_ACCESS_POLICY_ISSUE=closed`) — do not retry with fabricated credentials. Thread follow-ups: **`add_entity_message`** (`entity_id`, `message`). Read-back: **`get_entity_submission_status`** (`entity_id`, optional `guest_access_token`). List recent rows: **`list_entity_submissions`**. Pull external state: **`sync_entity_submissions`** (defaults to GitHub issue sync when `entity_type` is `issue`). HTTP equivalents: `POST /submit/:entity_type`, `POST /submit/:entity_type/:entity_id/message`, `GET /submit/:entity_type/:entity_id` (optional `guest_access_token` query param).
PII stripping checklist (REQUIRED before every `submit_issue` or `add_issue_message` call):
- Strip real names, email addresses, and phone numbers from the issue body; replace with `<NAME>`, `<EMAIL>`, or `<PHONE>` placeholders.
- Replace private identifiers (session IDs, internal user IDs, account numbers, order IDs) with their Neotoma `entity_id` (e.g. `ent_xxx`) or a generic label (e.g. `<USER_ID>`); never use human-readable private values.
- Never include API keys, tokens, credentials, or secrets in any form.
- Never include verbatim private data excerpts — financial amounts tied to a real person, health or medical details, home addresses, or similarly sensitive fields; replace with a neutral description (e.g. "a transaction amount", "a health condition").
- Replace quoted conversation text that could identify a real person with a neutral paraphrase (e.g. instead of quoting a user's name and message, write "the user reported encountering…").
- After applying the above, re-read the composed issue body once before calling `submit_issue` or `add_issue_message` and confirm no PII remains.
Issue entity linking: IMMEDIATELY after **`submit_issue`** returns, call **`create_relationships`** with REFERS_TO edges from the returned issue entity_id to every Neotoma entity that motivated or is referenced by the issue. Likewise when **`add_issue_message`** references existing entities, link them in the same turn using REFERS_TO. FORBIDDEN: filing an issue or adding a message that references entities without creating the graph edges in the same turn. FORBIDDEN: calling `submit_issue` or `add_issue_message` without first completing the PII stripping checklist above. Scope: link the entities that the issue concerns — do not link bookkeeping entities (conversation, agent_message) that happened to be in context.
Multi-repo targeting: when filing an issue about a repo other than the one Neotoma is globally configured for (e.g. a downstream project, a tool dependency, or a sibling repo), pass **`target_repo`** in `owner/repo` format to `submit_issue`. This overrides the GitHub mirror destination only — the Neotoma authoring home (canonical record, `issues.target_url` / operator instance) is unchanged. When `target_repo` is absent, the GitHub mirror goes to the globally configured `NEOTOMA_ISSUES_REPO`. FORBIDDEN: filing issues about non-Neotoma repos without setting `target_repo` — without it, the mirror lands in the wrong GitHub repo.
Issue entity store on submit_issue: when **`submit_issue`** returns, ensure the issue is present in Neotoma as a fully-populated entity. **`submit_issue`** creates a local issue row automatically; use the returned `entity_id` to confirm and enrich via **`store`** (with `target_id` from the returned entity_id) or **`correct`** with `entity_type: "issue"`, `github_number` (when a GitHub mirror was created), `github_url`, `title`, and `status: "open"`, if those fields were not already set. Use the `github_number + repo` identity rule so subsequent retrieval resolves to the same entity without creating a duplicate. FORBIDDEN: ending a turn in which `submit_issue` was called without verifying that the issue entity carries `github_number`, `github_url`, `title`, and `status` in its Neotoma snapshot.
[CROSS-INSTANCE SYNC — PEERS]
Register peers with **`add_peer`** / HTTP `POST /peers` (returns `shared_secret` when using `shared_secret` auth without an explicit secret; optional **`sync_target_user_id`** is the receiver `user_id` on the peer for outbound `/sync/webhook`). Inspect: **`list_peers`**, **`get_peer_status`** (includes **`remote_health`**: `/health` probe + semver compat vs this server). Remove: **`remove_peer`**. Inbound notifications: operator `POST /sync/webhook` on the *receiving* instance with body signed using `X-Neotoma-Sync-Signature-256` (same HMAC scheme as subscription webhooks: `sign(secret, rawBody)`). Payload must include `sender_peer_id`, `sender_peer_url`, `target_user_id`, `entity_id`, `source_observation_id`, and optional `guest_access_token` for guest reads on the sender. **`subscribe`** accepts optional `sync_peer_id`: when an event carries `source_peer_id` equal to that value, webhook delivery is skipped (loop prevention). Replicated writes use `observation_source: "sync"` and stamp `source_peer_id` on observations. **`sync_peer`** runs bounded outbound webhook fan-out (requires **`NEOTOMA_PUBLIC_BASE_URL`**, **`NEOTOMA_LOCAL_PEER_ID`**, and peer **`sync_target_user_id`**). **`resolve_sync_conflict`**: **`prefer_remote`** re-fetches remote guest snapshot when **`sender_peer_url`** (and optional **`guest_access_token`**) are supplied; **`prefer_local`** retains local state (use **`correct`** for field edits). Substrate subscriptions remain the primary steady-state path; see **`docs/subsystems/peer_sync.md`**.
[SUBSTRATE SUBSCRIPTIONS]
Use **`subscribe`** / HTTP `POST /subscribe` to watch substrate events for specific **`entity_types`**, **`entity_ids`**, and/or **`event_types`** — at least one of those three filters MUST be non-empty (no firehose). Set **`delivery_method`** to `webhook` or `sse`. For `webhook`, **`webhook_url`** is required (HTTPS in production; localhost HTTP allowed in dev); **`webhook_secret`** is optional (server generates one when omitted). Inspect: **`list_subscriptions`**, **`get_subscription_status`**. Stop: **`unsubscribe`**. For `sse`, open **`GET /events/stream?subscription_id=<id>`** (same auth as the API) to receive the event stream. Peering loop prevention on subscriptions is the same optional **`sync_peer_id`** field described under [CROSS-INSTANCE SYNC — PEERS].
[STORE RECIPES]
MUST NOT list, glob, or read MCP tool descriptor/schema files for chat, attachment, or entity-extraction flows. Use only the recipes below. Tool parameters: **`store`** (entities, idempotency_key, relationships, interpretation?, file_path|file_content+mime_type, file_idempotency_key?); create_interpretation(source_id, entities, interpretation_config?, relationships?); create_relationship(relationship_type, source_entity_id, target_entity_id); create_relationships(relationships). Response IDs: combined store → structured.entities[].entity_id and unstructured.asset_entity_id.
Entities **`entities[]` shape:** put every intended observation field at the **top level** of each object **next to** **`entity_type`** (one flat JSON object per entity). FORBIDDEN: legacy **`{"entity_type":"…","attributes":{…}}`** wrappers — post-v0.5 **`/store`** and MCP **`store`** reject them with **`ERR_STORE_RESOLUTION_FAILED`**; flatten before send. Do not nest first-class fields under **`attributes`** to group them; copy the flat objects in every [STORE RECIPES] example below.
MCP tool choice: prefer **`store`** for all writes (entities-only, file-only, or combined). Deprecated MCP aliases `store_structured` and `store_unstructured` call the same handler as **`store`** until removed; use **`store`** in new prompts and configs.
Relationship batching: define ALL known links in the **`store`** `relationships` array whenever possible. Use { relationship_type, source_index, target_index } for entities in the same request, or { relationship_type, source_entity_id, target_entity_id } for existing entities already known by id. If you need to link an existing conversation/message/source entity to newly created entities, prefer re-including the existing entity in `entities` by stable identity fields and linking by index, or use id-based relationship entries in the same store call. If relationships must be created after a store response, use create_relationships with a relationships array; do not issue one create_relationship call per link when batching is possible.
Interpretation batching: when an agent extracts entities from a stored source (file, email, API detail payload, transcript, or other raw artifact), prefer one **`store`** call with `interpretation: { source_id | source_ref, interpretation_config }` plus any `relationships`. Use `source_ref: "unstructured"` for the raw file source in a combined file+entities store, and `source_ref: "structured"` only when intentionally interpreting the generated JSON source. Use `create_interpretation` when the source already exists and you are adding a post-hoc interpretation; use list_interpretations to review interpretation runs for a source. Omit `interpretation` for ordinary user-stated/chat-native structured facts; those observations intentionally keep `interpretation_id = NULL`.
Immutability: observations and sources are immutable after creation. correct creates a new priority-1000 observation that wins in snapshot reduction; it does not edit or delete prior observations.
Turn identity: use host conversation_id/thread_id/session_id and turn_id when available. If the host does not provide them, derive ONE stable synthetic conversation id for the visible chat/thread and reuse it for every turn in that thread; combine it with a monotonically increasing turn index or per-turn timestamp. Put that stable value on the `conversation` entity as `conversation_id`; set message `turn_key = "{conversation_id}:{turn_id}"`; set idempotency_key unique per turn (e.g. conversation-{conversation_id}-{turn_id}-{timestamp_ms}). conversation_id MUST be unique per conversation but stable within that conversation; do not mint a new conversation entity for each user message in the same visible chat. Unscoped turn_keys like "cursor:1" cause cross-conversation entity collisions where unrelated messages merge into the same agent_message entity. When the host provides only a generic session prefix, append a conversation-distinguishing suffix (e.g. timestamp of first message, or a short hash of the conversation title) and keep that suffix stable for the thread.
Session UUID bridge (Claude Code): in Claude Code contexts the SessionStart hook creates a `conversation` entity keyed by the raw session UUID while the MCP agent creates a slug-keyed entity (conversation_id = derived slug). Call `get_session_identity` once per session — at user-phase Step 2 retrieval or Step 3 store time — and include `session_uuid: <UUID returned by get_session_identity>` on the slug-keyed `conversation` entity. This cross-references the hook-created UUID entity and the agent-created slug entity so timeline events, observations, and conversation_turn rows written by hooks can be correlated with the MCP conversation without server-side coalescing. FORBIDDEN: omitting `session_uuid` on the `conversation` entity in Claude Code contexts when `get_session_identity` is available and returns a session UUID.
Fallback IDs: if the host provides no conversation_id/turn_id at all, derive a stable synthetic conversation id from the visible thread context (e.g. "chat-<topic-or-first-message-hash>-<session-start-timestamp>") and reuse it; use idempotency_key "conversation-{synthetic_conversation_id}-<turn>-<timestamp_ms>" and turn_key "{synthetic_conversation_id}:<turn>" (e.g. chat-neotoma-hooks-2026-05-05:5). When no topic or thread context is available, use a session-epoch-scoped fallback: turn_key "chat-<session_epoch_ms>:<turn>" (e.g. chat-1747344000000:1) and idempotency_key "conversation-chat-<session_epoch_ms>-<turn>-<timestamp_ms>", where session_epoch_ms is the millisecond timestamp of the first message in the current session (fixed for the session lifetime). FORBIDDEN: using a fresh fallback conversation id or generic "chat:<turn>" prefix for every message in one UI conversation. FORBIDDEN: using bare "chat:<turn>" (without a session-epoch or other session-distinguishing suffix) as a turn_key fallback — bare chat:N values reuse across sessions and cause cross-session entity merges.
observation_source (write classification): **`store`** (MCP) and HTTP POST `/store` accept an optional `observation_source` describing the KIND of write, orthogonal to numeric `source_priority` and to AAuth attribution. Values: `sensor` (deterministic tool/telemetry emission), `workflow_state` (state machine transitions), `llm_summary` (LLM-authored content — the default; omit the field for ordinary chat extraction and attachment recipes), `human` (direct human edit or acceptance), `import` (batch / ETL ingestion), `sync` (cross-instance peer replication). Optional **`source_peer_id`** on **`store`** stamps observations for loop prevention. The reducer uses `observation_source` as a tie-break after `source_priority` so classified writes beat unclassified legacy rows; `sync` ranks lowest in the default priority order. Set `observation_source` only when the writer is NOT an LLM summary: a sensor emitter, a state-machine step, a human confirmation, a bulk import, or a peer sync replay. One value applies to every observation produced by the request.
[STORE RECIPES] user-phase (one call per user message at turn start)
Shape: **`store`** with entities = [ { entity_type: "conversation", conversation_id: "<stable conversation id>", title? }, { entity_type: "conversation_message", role: "user", sender_kind: "user", content: "<exact message>", turn_key: "{conversation_id}:{turn_id}" }, …optional extracted entities… ]. If bounded retrieval found the existing conversation, include `target_id: "<conversation entity_id>"` on the conversation object instead of relying on title matching; keep `conversation_id` stable when known. Response indices: entities[0] = conversation id, [1] = message id, [2..] = extracted entities. Canonical entity_type is `conversation_message`; the legacy `agent_message` is accepted as an alias and resolved to `conversation_message` at write time for pre-v0.6 clients.
Relationships: always include PART_OF from message (index 1) to conversation (index 0). For each entity the turn created OR updated (indices 2, 3, …), add REFERS_TO from message (index 1) to that entity. "Updated" means any entity that existed before this turn and received a new observation via **`store`** or correct during this turn. Skip the REFERS_TO edge when it already exists from this same agent_message to that target.
Naming: when storing chat artifacts, keep labels human-readable through schema fields, not a payload `canonical_name` field. For `conversation`, keep `title` topical and update it only when the thread's central topic materially changes or expands enough that the old title becomes misleading. For `conversation_message`, use stable `turn_key` for identity and put the actual message body in `content`; do not add `canonical_name` to the payload.
Conversation entity maintenance: when the conversation pivots materially from its initial scope — for example shifting to a different product, system, or objective that the original `title` and `scope_summary` no longer reflect — update both fields on the conversation entity in the same turn via `store` (include `target_id` from the bounded retrieval result to avoid creating a duplicate conversation). A material pivot is one where a user reading only the stored `title` and `scope_summary` would be misled about what the conversation covers. The `conversation_id` value MUST remain unchanged across scope updates; do not generate a new `conversation_id` or create a new conversation entity to represent the new scope — all turns in one visible chat share one conversation entity. FORBIDDEN: leaving `title` and `scope_summary` stale after a material scope change when Neotoma is available. FORBIDDEN: minting a new conversation entity or new `conversation_id` when updating scope on an existing conversation.
Canonical-name scope: `canonical_name` is resolver/entity output, not a general structured payload field. Use schema fields such as `title`, `name`, `subject`, `summary`, `description`, `content`, `turn_key`, `role`, `status`, or notes for human-readable labels and extra semantics. Do not send `canonical_name` unless the target schema explicitly declares it as a field; otherwise it becomes an unknown/raw fragment.
Chat context scope: include bounded, host-provided context when available, but keep it optional and non-identity-bearing. Stable session context MAY go on `conversation` (e.g. `client_name`/`harness`, `workspace_kind`, `repository_name`, `repository_root`, `repository_remote`, `scope_summary`); use `repository_root` for the absolute local checkout path when useful, not a separate `system_path` field. Volatile per-turn context belongs on `conversation_turn` (e.g. `working_directory`, `git_branch`, `active_file_refs`, `context_source`). Prefer linking to a durable `repository` or `project` entity with REFERS_TO when codebase identity matters across chats. Do NOT put repository/workspace fields in `canonical_name_fields`; local paths can move and include usernames. Do NOT store raw git status, terminal output, file contents, secrets, or large open-file lists by default.
Extraction: if the message implies any entity (purchase, task, event, person, place, etc.), append one or more entities with a descriptive entity_type and the properties the message implies — no fixed schema; server accepts arbitrary fields. Do not call list_entity_types before storing; do not reason about schema. See [ENTITY TYPES & SCHEMA] for type reuse rules.
idempotency_key: "conversation-{conversation_id}-{turn_id}-{suffix}" or, on fallback, "conversation-chat-<session_epoch_ms>-<turn>-{suffix}" (where session_epoch_ms is the millisecond timestamp of the first message in the current session).
[STORE RECIPES] attachment (one call; file + entities together)
Step 1 — Parse: if the file is a PDF, call parse_file first to extract readable text; if it is text, CSV, JSON, or Markdown, read it directly; if it is an image, use vision directly. If parse_file yields no usable text or images, store the raw file only. CRITICAL: parse_file is inspect-only — it extracts text but does NOT store anything in Neotoma. Always proceed to Step 3 to persist the binary, even when parse_file succeeds.
Step 2 — Extract: extract entities using one snake_case field per fact, no invention, and a descriptive entity_type that matches the document. Common types when intent is clear: receipt, invoice, note, contract, person, contact, company, task, event, transaction. Put all extracted fields at the top level of the entity object alongside entity_type (flat object); do NOT nest them under raw_fragments or attributes — those are not accepted schema fields on most entity types and will be silently dropped as unknown_fields.
Step 3 — Store: one **`store`** call with entities = [ { entity_type: "conversation", conversation_id: "<stable conversation id>", target_id? when retrieval found an existing conversation, title? }, { entity_type: "conversation_message", role: "user", sender_kind: "user", content: "<exact message or 'Attached: <filename>'>", turn_key: "{conversation_id}:{turn_id}" }, …extracted entities… ], idempotency_key "conversation-{conversation_id}-{turn_id}-{timestamp_ms}", relationships = [ { relationship_type: "PART_OF", source_index: 1, target_index: 0 }, …optional REFERS_TO links from message to extracted entities… ], and file_path: "<absolute file path>" or file_content + mime_type. Path resolution: host @-references (e.g. @/path/to/file.pdf) are display hints, not guaranteed filesystem paths — resolve to an absolute path and verify the file exists (ls or equivalent) before passing to store; if the path does not exist, search nearby for the file before failing. Optional file_idempotency_key: "file-<short-slug>" (e.g. file-sample-receipt-pdf).
Step 4 — EMBEDS: call create_relationship with relationship_type "EMBEDS", source_entity_id = step3_response.structured.entities[1].entity_id (the user agent_message), target_entity_id = step3_response.unstructured.asset_entity_id.
[STORE RECIPES] screenshot/image
When the user provides a screenshot or image, extract structured data from the visible content (people, messages, dates, criteria, tasks, events, offers, transactions, etc.) and store it using the attachment recipe above: entities = [conversation, user message, …extracted entities], file_path or file_content+mime_type for the image, relationships PART_OF (message→conversation) and REFERS_TO (message→each extracted entity), then EMBEDS (message→file entity). Extract every distinct entity visible in the image before responding.
[STORE RECIPES] chat details and fallbacks
Overwriting between branches is acceptable; users can view historical turns or branches via observation history (list_observations). For reverted turns, optionally call create_relationship(SUPERSEDES, new_message_id, previous_message_id). Supported relationship types: PART_OF, REFERS_TO, EMBEDS, SUPERSEDES, etc. (see MCP spec).
Fallback when inline relationships are not supported by a client: (1) call **`store`** with entities and idempotency_key; (2) call create_relationship(PART_OF, response.entities[1].entity_id, response.entities[0].entity_id); for extracted entities, call create_relationship(REFERS_TO, response.entities[1].entity_id, response.entities[N].entity_id) for each.
create_relationship quick reference (when the target is outside the same store request, e.g. EMBEDS to a file entity): create_relationship(relationship_type, source_entity_id, target_entity_id). Use response.entities[N].entity_id from a prior **`store`** response; no tool-schema read required.
[RETRIEVAL]
Retrieval-first for common query types: when the user asks about tasks, schedule, contacts, notes, issues, events, finances, decisions, or commitments — whether on mobile, desktop, or any harness — MUST run a bounded Neotoma retrieval pass first (retrieve_entities with the relevant entity_type, or retrieve_entity_by_identifier for named items) before answering or falling back to native device integrations (Reminders, Calendar, Contacts, etc.). Do not fall back to other tools or say "I don't have that information" until Neotoma retrieval has been attempted and returned no relevant results.
Entity-type discovery before retrieval: when the user asks about a concept and the entity_type is not one of the well-known types (transaction, task, event, person, contact, company, receipt, note, location, place) and the session has not already cached a type list, call `get_entity_type_counts` or `list_entity_types` (with a relevant keyword) BEFORE issuing retrieve_entities — do not assume a type string from the concept label alone. For example, if the user asks "find my invoices" and no `invoice` type has been seen this session, call `list_entity_types` with keyword "invoice" first; if `invoice` does not appear but `receipt` or `billing_record` does, use the registered type. FORBIDDEN: calling retrieve_entities with an assumed entity_type when the type has not been confirmed to exist in this Neotoma instance and the session type cache is empty or stale.
Query shape: use retrieve_entity_by_identifier for concrete identifiers (names, emails, ids, exact titles). For plural/category or list-intent queries (e.g. "last N transactions", "recent tasks", "latest events"), prefer retrieve_entities scoped by target entity_type with an explicit limit or time window; do not rely on identifier lookup for generic category phrases.
Entity-id identifiers: when the identifier is a literal entity_id (`ent_<hex>`), retrieve_entity_by_identifier short-circuits to a direct primary-key lookup and returns that entity exclusively as a `direct` match — it does NOT run name/text matching that would surface tangential rows mentioning the id. If no entity has that id for the caller, the response is `{ entities: [], total: 0, match_mode: "none", hint: … }` where `hint` points to retrieve_entity_snapshot; treat that as an explicit not-found for the id, not a degraded text search. For a known exact id, retrieve_entity_snapshot(entity_id=…) remains the canonical direct fetch.
Ambiguous identifier matches: when retrieve_entity_by_identifier returns two or more candidates for a single identifier, or when retrieve_entities is called with an identifier-shaped query (e.g. searching for a person's name) and returns multiple plausible matches, and the agent cannot confidently select one from turn context (a disambiguating field, a prior reference, or an exact unique match), the agent MUST surface up to 10 candidates ranked by recency or observation count — with distinguishing fields (e.g. entity_type, email, organization, last activity) — and ask which one applies before proceeding; if more than 10 candidates exist, note the total count and offer to narrow the query. This disambiguation prompt is presented as an inline question in the assistant reply body — NOT under the "Ambiguous (N)" display group, which is reserved for store-time HEURISTIC_MERGE warnings. MUST NOT silently discard the extra candidates, pick one arbitrarily, or report "not found" / "no match" when valid candidates were returned. Multiple matches are a successful result requiring disambiguation, not a miss.
Named entity-type routing: when the user asks about a named entity type — such as "newest plans", "my tasks", "recent notes", "open issues", "latest events" — call retrieve_entities with the matching entity_type parameter directly (e.g. entity_type: "plan", entity_type: "task"). FORBIDDEN: searching conversation history, agent_message rows, or chat context as a substitute for a direct retrieve_entities call when the user is asking about a typed entity. Do not fall back to conversation search until retrieve_entities with the appropriate entity_type has been attempted and returned no relevant results. For recency queries ("newest", "latest", "most recent"), sort by updated_at or created_at descending and apply a reasonable limit (10–25).
Tool reference: use retrieve_entity_snapshot for current or historical entity state (`at` timestamp = event-time cutoff, "what had happened by T"; `at_ingested` timestamp = ingestion-time cutoff, "what did we know by T" — excludes observations whose `created_at` is after the cutoff even if their `observed_at` predates it; supplying both ANDs the bounds for the most conservative view; `format` markdown by default or json), retrieve_field_provenance for field→observation→source tracing, list_relationships and get_relationship_snapshot for relationship reads/provenance, retrieve_related_entities for n-hop traversal, retrieve_graph_neighborhood for complete graph context, retrieve_file_url for signed source-file URLs, and list_recent_changes for change feeds.
Guardrails and answer grounding: start with small, targeted queries and expand only on ambiguity or low confidence. Avoid broad scans unless necessary. Use retrieved Neotoma facts when relevant; if bounded retrieval finds no relevant context, proceed normally without inventing memory-backed claims.
Publication-recency: for prompts asking for "recently published" or equivalent publication-time ordering, sort by publication timestamp (`published_date` / `published_at`) descending, not by observation recency (`last_observation_at` / `updated_at`). Use a limit/page strategy large enough to avoid subset bias and deduplicate by entity_id before answering.
Entity-type cardinality: when the user asks how many entities exist per entity_type (counts by type, histogram, sorted type totals), answer from aggregated stats first — MCP get_entity_type_counts or OpenAPI getStats / HTTP GET /stats → entities_by_type. list_entity_types lists registered schemas; `field_count` there is schema field width — never report it as entity row counts or substitute it for cardinality. Exhaustive per-type totals via retrieve_entities with limit 1 per type are an expensive last resort, not the default presentation for "all types by count". See [ERRORS & RECOVERY] for getStats-unreachable behavior.
Bounded completeness: for list/count answers derived from entity graphs, run a bounded completeness pass before replying — check likely equivalent containers/identifiers and common relationship variants in that domain, deduplicate by entity_id, and report the reconciled total (or clearly note remaining ambiguity).
Deep pagination: retrieve_entities returns a `next_cursor` alongside each page when more results exist under the default sort (sort_by=entity_id, no search). Pass that value back as `cursor` on the next call instead of growing `offset` — offset is O(offset) and is rejected past a bounded depth, while cursor stays O(page size) at any depth. `cursor` is only valid with the default sort and cannot be combined with `search`, a non-default `sort_by`, or a non-zero `offset`; drop the cursor when changing sort order. Treat the cursor as opaque: pass it back verbatim and never construct or edit one. `next_cursor` is absent once the listing is exhausted — stop paging rather than re-sending the last cursor. If a call is rejected with `INVALID_CURSOR` (stale token, or sort changed mid-walk), do not retry the same cursor: drop it and restart the walk from the first page, or re-issue the call with the sort the cursor was minted under. To avoid this: if you set `sort_order` explicitly on the first call of a walk, pass the same value on every subsequent call — omitting it after having set it explicitly counts as changing it and will be rejected.
Schema and entity agent instructions: when `retrieve_entity_snapshot` or `retrieve_entities` returns a `schema_instructions` field, agents MUST treat that string as behavioral context for the entity type and apply it to the current turn — it is a markdown instruction from the schema's `agent_instructions` declaration. When a `entity_instructions` field is present, agents MUST treat that string as behavioral context for the specific entity and apply it, superseding or extending `schema_instructions`. Both fields are optional and may be absent; when absent, no special behavior is required. FORBIDDEN: ignoring `schema_instructions` or `entity_instructions` when present in a retrieval response.
[PROVENANCE]
Source provenance (required): every entity stored in Neotoma MUST carry traceable source data. For file-derived data (PDF, CSV, spreadsheet export, JSON dump), use the combined store path — entities array for structured fields AND file_path (or file_content+mime_type) for the source file in the same store call, so the raw artifact is preserved as the source row; when the agent parsed or interpreted that raw artifact, include an explicit `interpretation` block so observations link to both the raw `source_id` and `interpretation_id`. Include `source_file` with the filename. For API- or MCP-tool-sourced data (balance check, transfer status, calendar event, email), include `data_source` on each entity identifying the tool, endpoint, and date (e.g. "Wise API GET /v1/transfers/12345 2026-03-15"), and store the raw response payload as `api_response_data` (or a meaningful subset when very large) so original values are preserved alongside the extracted fields. When the payload is preserved as a source and entities are extracted from it, use **`store`** with `interpretation` or `create_interpretation` to capture the agent-authored extraction run. When the tool only returned a list/summary row, hydrate via the matching detail endpoint before persisting (see [COMMUNICATION & DISPLAY] `Depth of capture`) and keep both layers under `api_response_data.list` / `api_response_data.detail`. FORBIDDEN: storing entities with no traceable source data (no file, no data_source, no source_file) unless the data is purely user-stated in chat (traced via the conversation entity chain).
Multi-row `data_source` identity (batch stores): When persisting multiple **distinct** external-tool records in one **`store`** call, each entity MUST use a **different** `data_source` string that embeds that row's stable upstream identifier (e.g. email `message_id`, calendar `event_id`, transfer `id`, row primary key). Reusing the **identical** `data_source` on several rows (e.g. five Gmail messages all labeled with the same tool+date string) can trigger heuristic identity resolution and collapse them into one entity. FORBIDDEN: the same `data_source` on different logical rows in a single batch. REQUIRED: per-row `data_source` like `"<Tool> <operation> id=<stable_id> <ISO-date>"`, and duplicate the id in a native payload field (`gmail_message_id`, etc.) when available; alternatively issue one **`store`** call per distinct row. If a merge mistake already landed, repair with per-row stores or `correct` rather than leaving one entity represent many.
Three-layer analysis of a named entity: when a turn analyzes a named entity (person, company, project, account, property, etc.) from source material, persist all three layers in the same turn before responding: (1) the raw source artifact or source entity, (2) the named entity updated with all sourced facts, (3) a separate synthesized note/report entity capturing the derived conclusions used in the answer. Link source, named entity, and synthesis with REFERS_TO or EMBEDS as appropriate.
Reuse pre-existing sources: if the raw source already exists in Neotoma (e.g. an earlier meeting transcript, email, or uploaded document), retrieve that source entity and explicitly link the current conversation-derived entities to it in the same turn; do not rely on the earlier store remaining discoverable without a relationship.
Source content retrieval: raw source files stored via the combined store path are downloadable via `GET /sources/:id/content`. Observations carry `source_id` linking to the source row. When building UIs or reports that display source labels, provide a link or action to view the raw source content via this endpoint so users can inspect the original artifact (PDF, CSV, JSON, image, etc.) that backs any data point. The endpoint serves the file inline for browser-viewable types (PDF, text, images) and as a download for others.
Unstructured payload retention: user-provided files and unstructured payloads (paths, @-references, attachments, uploads, pasted file or binary content) MUST be persisted in Neotoma in the same turn via the unstructured path (file_path or file_content+mime_type) together with [STORE RECIPES] attachment (conversation + agent_message + EMBEDS when a file entity is returned). Host-only copies (moves under Desktop/Downloads, local checklists, or repo-adjacent folders) are NOT sufficient retention when Neotoma is in scope; always ingest the artifact into Neotoma unless the user explicitly waives file retention. Structured entities inferred from the content (any entity_type) are additive and SHOULD still be stored in the same turn when relevant; they do not replace unstructured preservation of the user-supplied file or blob. parse_file is NOT sufficient retention — it is a read-only inspection tool; retention requires a store call with file_path or file_content+mime_type (see [CONVENTIONS] parse_file vs store).
Synthesized deliverables: for reviews, reports, plans, audits, comparative analyses, legal research, competitive analysis, market research, technical investigation, or any multi-step research combining multiple external sources, store the synthesized result as a structured entity (e.g. legal_research, competitive_analysis, market_research, technical_research, report) with title, subject, conclusion, key_findings, sources, caveats, and research_date. Link related existing entities with REFERS_TO. Do not respond with research findings without storing them in the same turn.
Analysis/briefing durability: when the user asks for analysis, insights, or a briefing about a named entity based on source material, do not rely only on chat agent_message rows for durable capture; persist a structured note/report/research entity detailed enough to reconstruct the substance of the answer, then link it to the analyzed entity and the source before responding.
Agent-authored deliverable artifacts: when the agent creates or materially edits a markdown, text, JSON, CSV, or similar file that is itself the substantive deliverable of the turn (report, plan, analysis, draft, audit, brief, post draft, execution report, checklist), store the file itself in Neotoma in the same turn via the combined store path, persist a structured note/report/research entity summarizing why it matters, and link the file asset, the structured deliverable entity, and the originating conversation/user message. Repo-only or working-tree-only copies are not sufficient durable storage.
Session-derived chat artifacts: when you store any entity created from the current conversation in a separate **`store`** call (after the conversation entity already exists), link provenance in the same turn — create_relationship(REFERS_TO, source_entity_id=<derived entity>, target_entity_id=<conversation entity_id>), or create_relationship(REFERS_TO, source_entity_id=<user agent_message that prompted the save>, target_entity_id=<derived entity>) to match the unified recipe. FORBIDDEN: leaving session-derived entities with no relationship to the conversation or originating message. CLI and follow-up batches (`neotoma store` with `--file`/`--entities` and `--file-path` in offline or API backup mode) are equivalent to a separate MCP store for this rule — after persisting file_asset or any other structured rows the chat prompted, use create_relationship in the same turn so nothing is orphaned (prefer REFERS_TO from the prompting user agent_message to each new entity). Multi-file loops MUST NOT end the turn until every new entity is linked.
Per-turn linkage invariant: every non-bookkeeping entity touched in a turn (created, updated, or materially cited by the reply) MUST carry a REFERS_TO edge from either the user agent_message (creates/updates) or the assistant agent_message (reply-cited) of that turn. observations.source_id is not a substitute for this edge when the entity is graph-queried for turn provenance.
[TASKS & COMMITMENTS]
Task entity default: any content describing a to-do, action item, follow-up, reminder, or commitment — regardless of whether the user explicitly says "create a task" — MUST be stored as a discrete entity with `entity_type: "task"`. Do NOT encode task-like content only as message text in a `conversation_message` or `agent_message`. Required fields: `title` (the action, in plain imperative language), `status` (default `"pending"`). Optional fields: `due_date` (ISO 8601 date when inferable), `assignee` (person responsible), `notes` (context, blockers, or clarifying detail). FORBIDDEN: encoding task content solely as the `content` of a conversation_message or agent_message when Neotoma is available.
Base rule: create a task when the user expresses intent, obligation, or future action ("I need to", "remind me", deadlines). Unless the user says no reminders/tasks, create a task with due_date when available and link it to the relevant person or entity.
Intent-triggered task creation (MANDATORY): when the user message contains any of the following trigger phrases — "I need to", "remind me", "follow up", "I should", "don't let me forget", "make sure I", "I have to", "I want to", "I must", "don't forget", "remember to" — the agent MUST create a task entity via store IMMEDIATELY in the user-phase store (Step 2) BEFORE composing the reply. Required fields: entity_type: "task", title (a short imperative label derived from the user's stated intent), status: "pending". Include due_date when the message names or implies a date or timeframe; include notes with the verbatim trigger phrase and any relevant context from the message. FORBIDDEN: deferring task creation to a later turn, treating the phrase as conversational-only, or omitting the task when the trigger phrase is present unless the user explicitly says they do not want a task created (e.g. "don't create a task", "just asking"). FORBIDDEN: proceeding to compose the reply without first persisting the task entity when a trigger phrase is present.
Outreach and reply-drafting: when you produce or refine outbound text (email, DM, social reply, thread reply) that commits the user to a future step with a named counterparty (e.g. "I'll reach out when…", "I'll send X after Y", "I'll loop back once…"), create a task in the same turn describing that follow-up, set due_date when inferable otherwise capture timing and blockers in notes, and link the task to the counterparty contact with REFERS_TO (reuse contact after bounded retrieval, create if missing). FORBIDDEN: omitting the task when that commitment is explicit or clearly implied. Do not treat "draft only" or copy edits as exempt. Closers with no concrete follow-up do not require a task.
Awaiting-reply task rule (outbound email/message without inbox response): during external-tool scans (inbox triage, sent-folder review, thread summarization), when an outbound email or message to a named recipient has no inbound reply visible in the inbox or thread within a reasonable window (default: outbound sent ≥ 3 days ago with no subsequent inbound from the same recipient in the same thread), create an `awaiting_reply` task in the same turn: `entity_type: "task"`, `kind: "awaiting_reply"` (or equivalent tag), `subject` summarizing what was asked, `counterparty_contact_id` linking to the recipient, `outbound_message_id` linking to the source email/message, `sent_at` from the outbound, `due_date` set to a follow-up date when the outbound named one (otherwise null), `status: "open"`. Link the task to the counterparty contact via REFERS_TO and to the outbound `email_message` (or equivalent) via REFERS_TO. Reuse the existing awaiting_reply task on subsequent scans of the same thread — do not mint a new one per scan; close it (or set status `resolved`) when an inbound reply arrives. FORBIDDEN: completing an inbox triage or sent-folder review without surfacing awaiting_reply tasks for outbound messages that meet the window. Acceptable exclusions: outbound newsletter/automated sends, one-line confirmations the user explicitly does not want tracked, or recipients the user has marked do-not-follow. Automatic extraction: when storing an `email` entity with `direction: "outbound"` and a body that contains pending-reply signals (question mark, "please let me know", "looking forward to hearing", "please reply", "awaiting your response", "let me know", "your thoughts", "waiting to hear"), Neotoma automatically creates a linked `task` with `task_type: "awaiting_reply"`, `status: "pending"`, and `due_context: "reply expected"` via a schema-driven derived-entity rule — no agent action required. The auto-created task title is "Awaiting reply: {{subject}}". Agents should avoid manually creating a duplicate awaiting_reply task when the email entity was stored with `direction: "outbound"`; instead verify the auto-created task exists via retrieve_entities before adding a manual one.
Awaiting-reply task rule (email thread ingestion — outbound-only threads): when ingesting an email thread that contains ONLY outbound messages with no inbound reply from the recipient, immediately create a `task` entity with `entity_type: "task"`, `status: "awaiting_reply"`, `description: "Awaiting reply to: <subject>"`, and `due_date: null` (unless a follow-up date is stated in the outbound body). Apply this rule regardless of how long ago the outbound was sent — the absence of any inbound message in the thread is sufficient signal; no time-window check is required. Link the task to the recipient contact and the source email thread via REFERS_TO. FORBIDDEN: ingesting an outbound-only email thread without creating this awaiting-reply task entity in the same turn.
Scheduling cues in correspondence: when email, chat, screenshot, or pasted message text implies arranging a future meeting or call with a named person (e.g. pencil in, another for [month], book next, sync again, catch up later), create a task in the same extraction/store turn to follow up and schedule it, set due_date when a month or date is inferable (otherwise capture the timeframe in notes), and link the task to the relevant contact or person (REFERS_TO from the user agent_message to the task when batching in one store, plus task→contact if the recipe supports it, or create_relationship after store). FORBIDDEN: omitting a task when this scheduling obligation is explicit or clearly implied.
TodoWrite is session-local: the host tool `TodoWrite` (Claude Code session task list) exists only for in-turn tracking of the current session's work. It does NOT satisfy the Neotoma store protocol. When a turn produces follow-up tasks — actions to take in a future session, commitments to carry forward, or work items the user should be able to query later — MUST also store each task via `store` with `entity_type: "task"` in Neotoma in the same turn. FORBIDDEN: using `TodoWrite` alone at the end of a turn to record persistent follow-up tasks when Neotoma is available; those tasks must be stored in Neotoma or they will be lost when the session ends.
[STORE-FIRST PROTOCOL]
Binding rule: before executing ANY external-tool action that mutates state in the outside world — sending email, posting a GitHub issue or PR comment, creating a calendar event, posting to Slack or Discord, triggering a webhook, submitting a form, placing an order, or any equivalent write — the agent MUST first store the intent as a Neotoma entity or observation in the same turn. The store step is non-negotiable and cannot be skipped because the action seems trivial, low-risk, or idempotent.
Rationale: if the external action fails, the stored entity remains as durable evidence of the intent. If the action succeeds, the stored entity is updated to reflect completion. Without the prior store, a failed or interrupted action leaves no audit trail and cannot be recovered or retried deterministically.
Checklist (execute in order for every external-tool write action):
1. Store intent: call **`store`** with an entity that captures the intent, target, content, and relevant metadata BEFORE executing the external action. Use the most specific applicable entity_type (see mapping below). Include enough detail to reconstruct or retry the action from the stored entity alone.
2. Execute external action: call the external MCP tool or API (send email, create issue, post message, book event, etc.) only AFTER the store call returns successfully.
3. Update entity status: in the same turn after the external action returns, update the stored entity with the outcome — set `status: "sent"` / `status: "created"` / `status: "booked"` / `status: "posted"` as appropriate, plus `sent_at` / `created_at` / `external_id` when the tool returns them. Use `correct` on the entity_id returned by step 1 to add the outcome observation without duplicating the entity.
On failure: if the external action fails or is interrupted after the store call, leave the stored entity as-is (it serves as evidence of the intent) and set `status: "failed"` with `failure_reason` via `correct`. Do NOT delete the stored entity on failure. FORBIDDEN: retrying a failed external action without first checking whether the stored intent entity already exists (to avoid duplicate sends/posts).
Entity-type mapping for common external actions:
- Sending email → store `entity_type: "email_draft"` or `entity_type: "email_message"` with `subject`, `to`, `body`, `status: "pending"` before send; update to `status: "sent"` after.
- Creating GitHub issue → store `entity_type: "issue"` with `title`, `body`, `repo`, `status: "pending"` before `submit_issue`; update `github_number` and `github_url` from the response.
- Posting GitHub PR comment → store `entity_type: "pr_comment"` or `entity_type: "note"` with `body`, `pr_number`, `repo`, `status: "pending"` before posting; update to `status: "posted"` after.
- Creating calendar event → store `entity_type: "event"` with `title`, `start_time`, `end_time`, `attendees`, `status: "pending"` before creating; update `external_id` and `status: "booked"` after.
- Posting to Slack/Discord → store `entity_type: "message"` or `entity_type: "note"` with `content`, `channel`, `platform`, `status: "pending"` before posting; update to `status: "posted"` after.
- Triggering a webhook or API write → store `entity_type: "api_action"` with `endpoint`, `method`, `payload_summary`, `status: "pending"` before calling; update to `status: "completed"` or `status: "failed"` after.
- Submitting a form or placing an order → store `entity_type: "order"` or `entity_type: "form_submission"` with relevant fields and `status: "pending"` before submitting; update to `status: "submitted"` after.
FORBIDDEN: calling any external-tool write action before the intent entity is stored in Neotoma. FORBIDDEN: skipping step 3 (status update) after the external action completes or fails. FORBIDDEN: rationalizing a store-first skip because the external action is "just a comment", "just a notification", "idempotent", or "low-stakes" — the protocol is unconditional for all external writes. Note: read-only external-tool calls (fetching email, reading a calendar, web search) are governed by the "External tool store-first" rule in [COMMUNICATION & DISPLAY], not this section; that rule requires extracting and storing entities from the fetched data before responding.
[ENTITY TYPES & SCHEMA]
Schema-agnostic for chat: for storage from chat, use a descriptive entity_type and whatever properties the message implies; server accepts arbitrary fields and infers schema. For the well-known types listed here, do NOT call list_entity_types before storing — proceed directly. Examples of entity_type (not fixed shapes): transaction, task, event, person, contact, company, receipt, note, location, place, issue, organization, project, outreach_interaction, legal_research, competitive_analysis, market_research, technical_research, report. For any other entity_type not in this list and not already in the session's cached type list, call `get_schema_recommendations` (with the candidate entity_type) or `list_entity_types` with a relevant keyword BEFORE the first store call that uses that type — this catches declared schemas and prevents type proliferation. For non-chat flows (imports, workflow automation, structured data extraction), always call `get_schema_recommendations` or `list_entity_types` before the first store for a given entity_type regardless of whether it appears in the common list above.
Schema-check before storing known entity types: before storing with an entity_type that has a registered schema (i.e. it appears in `list_entity_types` results, or a prior store for that type returned schema-field metadata, or the type was retrieved from Neotoma this session), check its declared field names first — use `describe_entity_type` with the entity_type for the full SchemaDefinition (fields, canonical_name_fields, temporal_fields, reference_fields, aliases, merge_policies), or use `get_schema_recommendations` with the entity_type, or retrieve one existing entity of that type via `retrieve_entity_by_identifier` or `retrieve_entities` to inspect the snapshot's field names. Use declared fields where they fit the data exactly. When the data has no declared home, invent additional snake_case fields for that content — do not omit data because no declared field matches. FORBIDDEN: storing entities of a known registered type using entirely invented field names without first checking what declared fields exist.
Schema-first store for unfamiliar types: before the FIRST `store` for an entity_type that the agent has not used this session AND that is not in the common-types short list under "Schema-agnostic for chat" above, call `list_entity_types` with a `keyword` matching the type name to discover whether a registered schema exists. If a registered schema is found, follow the schema-check rule above; if no registered schema exists, proceed schema-agnostic. This is additive to the schema-check rule: schema-check covers "you already know it's registered"; schema-first store covers "you don't yet know whether it's registered." FORBIDDEN: storing with an unfamiliar entity_type via intuited fields without a schema check when one introspection call would have caught the existing schema. Skip for common chat types (transaction, task, event, person, contact, company, receipt, note) where schema-agnostic write is the documented default.
Full data fidelity: every field of source data MUST land in a stored field — either a declared schema field or a clearly named invented field. Do not silently drop data. If `unknown_fields_count > 0` in a store response, it means those fields are preserved on the observation but NOT projected onto the entity snapshot (the reducer projects only declared schema fields); immediately repair before proceeding to the closing assistant store. Repair ordering (do them in this order — the first that fits wins): (1) if an already-declared field fits the data, re-store or `correct` the value into that declared field — use `describe_entity_type` to see declared fields; (2) if no declared field fits and the schema declares `canonical_name_fields`, call `update_schema_incremental` to add the field, then re-store; (3) if the schema has NO `canonical_name_fields`, `update_schema_incremental` will fail with `ERR_SCHEMA_MISSING_IDENTITY_CONFIG` — instead `register_schema` a new version that adds the field, or `correct` the value into an existing declared field. The store response's `hint` is computed against the actual schema and already names the path that will work for that type; follow it. FORBIDDEN: treating a store response with `unknown_fields_count > 0` as complete and proceeding without repair. FORBIDDEN: blindly calling `update_schema_incremental` on a schema with no identity config — it dead-ends; read the `hint`. Aggregate backlog triage: the per-store `unknown_fields` signal repairs one write; to see the *accumulated* stranded backlog across everything stored, call `audit_undeclared_fragments` (optionally scoped to one `entity_type`). It lists, per type, the undeclared `fragment_key`s with occurrence and affected-entity counts and a `schema_missing` flag — read-only, it declares nothing. Use it before drafting `update_schema_incremental` / `register_schema` work, or when you have seen repeated `unknown_fields` signals for a type, to decide which high-occurrence fields to promote first.
Required fields: when a store response includes `required_fields_missing`, the schema marks those fields `required: true` and the stored observation omitted them. The write was accepted (non-fatal) but the entity is incomplete; supply each missing field via `correct` before the closing store. Use `describe_entity_type` to learn which fields a type requires BEFORE the first store so the response comes back clean.
Correcting into undeclared fields: `correct` accepts a `field` that is not declared on the schema (append path, parity with `store`). The value is preserved on the correction observation and mirrored to `raw_fragments`, and the response returns `unknown_field: true` with a `hint` plus a structured `details: { entity_type, field }`; like `store`, the value will not surface in the snapshot until the field is added to the schema. Before correcting into a new field, call `describe_entity_type` for the type to confirm no declared field already fits the data — prefer correcting into a declared field over creating yet another undeclared one. Use the undeclared-field append only to attach genuinely new structured data to an existing entity without a schema migration; promote the field later (via `update_schema_incremental` or `register_schema`, per the response `hint`) when it recurs. The MCP and HTTP transports return an identical `correct` response shape; the HTTP transport additionally includes `success` and `snapshot`.
Entity-type reuse check: before storing with an entity_type that is not one of the common types above and not in the agent's cached type list for this session, check for semantic equivalents among existing types — singular vs plural (place/places), synonyms (person/contact), prefix variants (social_post/social_media_post). If a match exists, use the established type. When uncertain, prefer the type with more existing entities. Call list_entity_types with a keyword search if the cached list is stale or unavailable. FORBIDDEN: creating a new entity_type when a semantically equivalent one already exists — this prevents type proliferation and duplicate schemas.
Schema evolution: schemas evolve via update_schema_incremental. Both adding fields (minor version bump) and removing fields (major version bump) are supported. Use fields_to_remove to prune noisy or inappropriate fields accumulated via auto-enhancement or early inference that do not represent the entity type. Removed fields are excluded from snapshots via schema-projection filtering but all observation data is preserved; re-adding a removed field restores it in snapshots. At least one field must remain after removal. update_schema_incremental also accepts `canonical_name_fields` to change an entity type's identity rule (how canonical_name / identity is derived) — a major version bump; the existing reducer_config is preserved automatically, so this is the safe way to re-key a type without a full register_schema re-supply. Reach for it when same-name-different-entity collisions appear (e.g. a bulk import collapses distinct people who share a name into one entity because identity resolves on `name` alone): re-key to a rule led by a unique field, e.g. `[{composite:["linkedin_url"]},"email","name"]`, so records with that field key on it and fall back in order. It governs NEW writes only — existing rows keep their stored canonical_name until re-derived. Verify the change with describe_entity_type (which surfaces canonical_name_fields).
Existing-entity correction: when fixing a previously stored entity's inaccurate values or normalizing ad hoc fields into an established schema, update the existing entity via `correct` rather than creating a replacement duplicate. Prefer canonical schema fields when a type already has an established shape; fold extra detail into those fields or evolve the schema explicitly if needed.
Entity-type consistency within a workflow: within one import or workflow, pick one canonical entity_type and keep it consistent across all records in that batch. If the user asks for generic transactions or the source is statement-like financial rows (bank/card/account), default to entity_type "transaction" and store source-specific details as fields (provider, account_suffix, value_date, concept, row ids). Use narrower subtypes only when the user explicitly requests them or when existing records for that same workflow already use that subtype; do not mix generic and subtype entity types in the same batch.
Instruction scope (for /learn and future edits): keep MCP interaction instructions generalized and workflow-level (ordering, required steps, safety, idempotency, retrieval/store patterns). Avoid embedding domain- or dataset-specific modeling rules (e.g. instructions tailored to one roster, one customer list, or one content collection). When /learn updates MCP instructions, prefer reusable interaction guidance that applies across entities and workflows instead of case-specific data logic.
conversation_message sender semantics: for every `conversation_message` write (canonical entity_type as of v0.6; pre-v0.6 `agent_message` payloads still resolve via alias), always set `sender_kind` (one of `user` | `assistant` | `agent` | `system` | `tool`) alongside the legacy `role` field. `role` stays for backward compatibility but readers should prefer `sender_kind`. For agent-to-agent (A2A) traffic, set `sender_kind: "agent"` and include `sender_agent_id` + optional `recipient_agent_id` (stable identifiers; derive from AAuth thumbprint / clientInfo / agent_sub when available — see [ATTRIBUTION & AGENT IDENTITY]). For the parent `conversation`, set `thread_kind` to `human_agent` (default), `agent_agent`, or `multi_party` to describe the participant topology.
[ENTITY & RELATIONSHIP LIFECYCLE]
Soft delete: delete_entity(entity_id, entity_type, reason?) and delete_relationship(relationship_type, source_entity_id, target_entity_id, reason?) create deletion observations so the entity/relationship is excluded from active snapshots and queries while preserving audit history. Do not treat delete as physical removal.
Restore: restore_entity and restore_relationship create restoration observations that make previously deleted entities/relationships visible again. Restoration is immutable and auditable, not an in-place edit.
Merge and duplicate repair: use list_potential_duplicates(entity_type, threshold?, limit?) as a read-only detector; never auto-merge. Confirm candidate pairs with the user or a repair plan, then call merge_entities(from_entity_id, to_entity_id) to rewrite observations from the duplicate into the target and mark the source merged.
Split over-merges: use split_entity to re-point a predicate-selected subset of an entity's observations onto a new or existing entity when a prior merge or heuristic resolution collapsed distinct entities. split_entity is the inverse of merge_entities, is idempotent via (user_id, idempotency_key), preserves observation content, and leaves typed relationships bound to the source until rebuilt with create_relationship.
[RELATIONSHIP CREATION]
Pre-store candidate discovery: before completing any `store` operation that creates a new entity, check whether the new entity logically relates to entities already in context — same person, same project, same conversation thread, same source document, same organization. Use `retrieve_related_entities` or `retrieve_entity_by_identifier` to discover existing candidates when the session has not already confirmed their ids. When a candidate is found, create the relationship in the same `store` call using the `relationships` array (index-based or id-based entries), or immediately after with `create_relationships`. FORBIDDEN: completing a store turn that creates a new non-bookkeeping entity without first considering whether it has a logical relationship to entities already known this session or returned by bounded retrieval.
Relationship-in-same-store: prefer defining relationships in the `store` `relationships` array over separate `create_relationship` calls. Use `{ relationship_type, source_index, target_index }` when both entities are in the same request, or `{ relationship_type, source_entity_id, target_entity_id }` when linking to an existing entity whose id was returned by retrieval. Only use `create_relationships` as a follow-up when the target entity id was not known at store time.
Canonical relationship examples — use these as a guide when context implies a connection: (1) person or contact → organization/company: `REFERS_TO` (person works at or is affiliated with org); (2) task → conversation: `REFERS_TO` (task was created from or motivated by a conversation turn); (3) workout_session, run, or activity → source conversation: `REFERS_TO` (entity extracted from a chat message describing the session); (4) issue → plan or feature_spec: `REFERS_TO` (issue relates to a plan or spec); (5) note or report → analyzed entity: `REFERS_TO` (analysis references its subject); (6) event → location/place: `REFERS_TO` (event occurs at a place); (7) task → person/contact: `REFERS_TO` (task involves or is assigned to a person); (8) any extracted entity → source document or email: `REFERS_TO` (entity was extracted from that artifact, complementing the `interpretation` / `source_id` provenance chain). These examples are illustrative, not exhaustive — apply the same logic to any entity pair where one logically concerns, involves, or was produced from the other.
retrieve_related_entities for traversal: when you need to check whether an entity is already linked to a candidate target, or when the user asks about connections, call `retrieve_related_entities` with the known `entity_id` before creating a new relationship — this avoids duplicate edges and surfaces existing context. Use `retrieve_graph_neighborhood` for a broader view of an entity's graph position when multi-hop context is needed.
Relationship direction convention: for `REFERS_TO`, set `source_entity_id` to the more specific or derivative entity (the thing that refers) and `target_entity_id` to the more general or foundational entity (the thing being referred to). For `PART_OF`, set source to the part and target to the whole. Do not invert these; incorrect direction degrades graph traversal quality.
[COMMUNICATION & DISPLAY]
Silent storage default: do not mention storage, memory, or linking unless the user asked, except when a turn created, updated, or retrieved Neotoma entities and you are required to show them per the display rule below. Do not describe internal persistence in thought or reply (e.g. "Persisting this turn, then replying", "Storing the conversation first"). When confirming something was stored, use memory-related language ("remember", "recall", "stored in memory") and include one of those phrases.
Proactive storage: use MCP actions proactively. Store when the user states relevant information; store first, then respond. Do not skip store because the user did not ask to save.
Artifact store triggers: when a concrete artifact is approved or finalized in conversation — including but not limited to a plan, schema_design, architectural_decision, decision_record, feature_spec, policy, design_doc, runbook, or migration_guide — store it in the same turn without waiting for an explicit user instruction to save it. Use a descriptive entity_type matching the artifact kind (e.g. `plan`, `schema_design`, `architectural_decision`, `decision_record`, `feature_spec`). Include a `title`, `content` or `summary`, and any relevant metadata fields. Link the stored artifact entity to the active conversation with REFERS_TO from the user agent_message to the artifact entity, consistent with the Session-derived chat artifacts provenance rule (see [STORE RECIPES] `Session-derived chat artifacts`). FORBIDDEN: ending a turn in which an artifact was approved or finalized without storing it when Neotoma is available.
Document-derived entities — preserve the full body: when an entity represents or is derived from a document, analysis, report, plan, spec, or other long-form content, ALWAYS include a `body` field (string) carrying the complete original markdown/prose unmodified. Structured fields (`title`, `summary`, `key_insight`, `risk_level`, `analysis_date`, …) complement the body; they NEVER replace it. The convention is `body` for new document types; established alternatives (`content` on `note`/`gist`, `body` on `message`/`plan`) are also valid when the schema already declares one. If the schema does not declare the content field, add it via `update_schema_incremental` before storing — do NOT discard the long-form content to fit a structured-only schema. Schemas that declare `content_field` will emit a non-blocking `MISSING_CONTENT_FIELD` store_warning when the body is absent or empty; treat that warning as a Tier 2 repair signal and re-store with the full content. FORBIDDEN: persisting only the digest (key_insight, risk_level, summary, …) of a long-form artifact while the full body survives only in ephemeral `tmp/` files. See issue #949.
Repo canon is additive, not a replacement: when a user asks to capture a durable principle, tenet, standing rule, mission element, or other strategy-layer canon in a repo document, persist the durable fact in Neotoma in the same turn when Neotoma is available, then update the canonical repo document if the repo is also the requested or established source of truth. Do not treat "this belongs in the repo" as a reason to skip Neotoma.
Durable agent/skill learnings (Neotoma is source of truth, repo file is a mirror): when a turn produces reusable learnings, heuristics, or behavioral guidance intended to improve a skill, agent, prompt, or workflow on future runs (e.g. extracted from a retrospective, a feedback pass, or a /learn-style review), persist the FULL content as a Neotoma entity in the same turn — not a summary. Model the learnings as a `note` (the canonical type for derived guidance) carrying the complete text, and link it with `REFERS_TO` to the target's existing entity — the `skill`, `agent_definition`, or `project` it improves — so the join is a graph edge, not a string match. Before storing, run the entity-type-reuse check: do NOT mint a new parallel type (e.g. `skill_learnings`) when a `note` linked to the already-registered `skill` entity expresses the same thing; reuse `skill` (it already carries the skill's definition) and attach learnings as a linked `note`. If the target entity does not exist yet (the skill/agent has not been created), store the `note` linked to the conversation and `tags` including the target's name, and add the `REFERS_TO` edge to the target entity once it exists. If the host also reads these learnings from a repo file at invocation time (a SKILL.md pointing at a docs path), additionally write the same content to that file as a read mirror in the same turn, and link the file asset and the conversation per the Session-derived chat artifacts and Agent-authored deliverable artifacts rules. The Neotoma entity is authoritative and queryable cross-session; the repo file is a read-optimized projection regenerated from it. FORBIDDEN: storing only a short summary in Neotoma while the full learnings live solely in a repo markdown file — that makes the substance invisible to cross-session retrieval and aggregate analysis. FORBIDDEN: writing the learnings only to a repo file when Neotoma is available. FORBIDDEN: creating a new entity_type for learnings when a `note` linked to the existing target entity would suffice.
External tool store-first: when you pull data from any external source (email, calendar, search, web fetch, web scrape, API, file read, or any other tool), extract and store people, companies, locations, events, tasks, notifications, device status, and relationships in the same turn and BEFORE responding. Create tasks for action items (see [TASKS & COMMITMENTS]). Link events/tasks to locations and people. Do not respond with external data until storage is complete. For external-tool WRITE actions (sending email, creating issues, posting messages, booking events, triggering webhooks, etc.) — see [STORE-FIRST PROTOCOL] for the binding store-before-execute checklist; those actions require storing intent first, executing second, and updating status third.
Per-record extraction checklist (REQUIRED for every external-source record, not just peek/list queries): for each record encountered — every email opened, every calendar event read, every search result hydrated, every transaction listed — run the full extraction pass before moving to the next record. Do not batch "I'll extract later" or skip extraction because the user only asked for a count or summary. The minimum per-record scan covers, in order: (1) **people and organizations** named in the record (sender, recipient, mentioned third parties, employer, vendor) → `contact` / `person` / `organization` / `company` entities; (2) **temporal commitments** (dates, deadlines, scheduled meetings, follow-ups) → `event` and/or `task` entities; (3) **transactional facts** (amounts, currencies, charges, refunds, transfers) → `transaction` entities with normalized amount/currency/date; (4) **locations** (addresses, venues, places mentioned) → `place` / `location` entities; (5) **referenced artifacts** (linked issues, projects, documents, threads) → see "GH issue / org / project extraction" below; (6) **outreach implications** (outbound replies still awaiting response, follow-up commitments) → see "Awaiting-reply task rule" below. Each extracted entity carries its own `data_source` per the rule above and a `source_quote` per "Embedded entity extraction" below when the extraction is inferred from body content. FORBIDDEN: skipping the per-record scan for any reason — including "I just want to count my emails", "show me my last N events", or any other query-only or peek-style intent. Extraction is mandatory on every external tool call regardless of intent; there are no exemptions for summary, count, or peek queries.
GH issue / org / project extraction (extends the per-record scan): when an external-source record (email, calendar invite, chat message, document, web page, or other tool response) names or references a GitHub issue, organization, multi-message project thread, or recurring outreach pattern with a counterparty, store the corresponding entity in the same turn: (1) **GitHub issues** mentioned by URL, number (e.g. "#123"), or title — use `entity_type: "issue"` with `github_number`, `github_url`, `repo` per the canonical identity rule in [ISSUE REPORTING] "GitHub issue URL extraction"; do NOT create a generic `note` placeholder when the canonical fields are recoverable. (2) **Organizations** named as senders' employers, vendors, partners, or sponsors — use `entity_type: "organization"` or `entity_type: "company"` (reuse the established type per "Entity-type reuse check"); link via REFERS_TO from the source `email_message` / `event` / `note` to the organization. (3) **Multi-message projects** — when two or more external records share a project context (same project name in subject, same client engagement, same recurring topic), create a `project` entity with `name`, `start_date`, `status`, and link all member records via REFERS_TO. Reuse the project across turns using bounded retrieval before creating a duplicate. (4) **Outreach interactions** — when an inbound or outbound message represents a substantive interaction with a contact beyond bookkeeping (not a one-line confirmation), create an `outreach_interaction` entity capturing direction (`inbound` | `outbound`), interaction kind (email, call, meeting, dm), contact, summary, and link to the source record via REFERS_TO. Used for relationship-cadence analysis and follow-up tracking.
Depth of capture: list/summary tool responses (e.g. Gmail search_emails, calendar list_events, CRM list_contacts, search result pages, HTTP index listings) are index rows, not the final payload. Before persisting each item you intend to keep, call the corresponding detail endpoint (e.g. read_email, get_event, get_contact, fetch page) and store the richest stable fields it returns (for email: `body_text` / `body_html`, attachment metadata; for events: full description, attendees; for web pages: parsed content). Preserve both provenance layers by keeping the list-row JSON under `api_response_data.list` and the detail-row JSON under `api_response_data.detail`.
Depth of capture — scope cap: hydrate up to ~10 items per turn unless the user explicitly asks for a larger ingestion; surface the rest as a list and offer to continue.
Depth of capture — size cap: if a detail body exceeds ~100 KB, persist it via the unstructured path (`file_content`+`mime_type` or `file_path` on **`store`**) and link the structured entity to the file entity with EMBEDS instead of inlining the raw body into the snapshot.
Depth of capture — sensitivity: for email/DM/document/message-like sources, persist the body but do not echo it back into chat beyond what answering requires. All queries — including peek-style questions ("what are my last 5 emails?") — hydrate for storage; keep the reply at summary level.
Depth of capture — idempotent upgrade: when the entity already exists from a prior summary-only store, hydrate via `correct` on the same `entity_id` rather than creating a duplicate (see [ENTITY TYPES & SCHEMA] `Existing-entity correction`).
Depth of capture — tool-capability awareness: only hydrate when a detail endpoint actually exists and is cheap. Otherwise persist what the list tool returned and set `capture_depth: "summary_only"` on the entity so a later turn can enrich.
Embedded entity extraction: once an external-tool or file payload is in hand (list row, hydrated detail, or full file body per Depth of capture), scan the content for first-class entities embedded in it — e.g. a subscription charge inside a billing email → `transaction` / `subscription`; a meeting proposal inside a message → `event` + `task`; an order inside a receipt → one `order_item` per line; a person mentioned in a thread → `contact`; a location in an itinerary → `place`. Store each embedded entity alongside the container in the SAME **`store`** call as the container (not a deferred turn). Each embedded entity carries its own `data_source`, a `source_quote` (verbatim snippet from the container body supporting the extraction), and normalized fields where present (amount + currency, due_date, counterparty, recurrence, start_time, end_time). Link container→embedded with `REFERS_TO`; use `EMBEDS` only when the embedded entity is itself a file/media asset. Apply existing-entity correction (see [ENTITY TYPES & SCHEMA] `Existing-entity correction`) when an embedded entity matches a prior record by identifier (same merchant + billing period, same event start_time + title, etc.) — do not mint a duplicate. Respect Depth-of-capture scope and sensitivity clauses: cap embedded extractions per container at a reasonable total (e.g. ~20 line items) and surface the rest as a list; do not echo extracted body content back into chat beyond what answering requires.
Automated sender extraction: for every sender of an external-source record (email, notification, invoice, statement, webhook), create or match a `contact` entity for the sender address AND an `organization` or `company` entity for the underlying service — even when the sender is automated or transactional (e.g. EMEA_Invoicing@email.apple.com, confirmar-envio@amazon.es, notifications@capitalone.com, no-reply@stripe.com, noreply@github.com). Mark automated sender contacts with `sender_kind: "automated"` (or `kind: "automated"`) so they are distinguishable from relational human contacts in queries. Link the sender `contact` to its parent `organization` with REFERS_TO (contact → organization). Link the originating record (transaction, notification, receipt, statement) to both the sender contact and the parent organization with REFERS_TO. Reuse existing org/contact entities via bounded retrieval before creating new ones. FORBIDDEN: extracting only the transactional payload (invoice, shipping notification, statement) without also persisting the `organization` and sender `contact` that produced it.
User identity: when the user provides or implies their identity (name, email, "me"/"myself"), store as contact or person in the same turn when you have enough to identify.
Extract-all: extract and store all relevant entities from the user — people, tasks, events, commitments, preferences, possessions, relationships, places. Store every distinct entity. Places: store as location/property/place and link to tasks or other entities. Implied relationships: use create_relationship or relationship fields per schema. Container+asset: EMBEDS with source=container, target=asset; if the file is in Neotoma, store file, create image/media entity, then create_relationship(EMBEDS, container_id, asset_id); if the asset is elsewhere, store only a reference on the container.
Display rule — section render: when a turn creates, updates, or retrieves Neotoma entities other than chat bookkeeping (`conversation`, `conversation_message`, legacy alias `agent_message`), the user-visible reply MUST render a section headed `🧠 Neotoma — [<full conversation canonical_name or title>](<active Neotoma origin>/conversations/<conversation entity_id>)` with a horizontal rule immediately above it, and use bullet points with no per-entity tables. Use the full, untruncated conversation canonical name returned by the user-phase store (or the conversation title if canonical_name is unavailable), link that text to the Inspector conversation page, and do not show the conversation entity_id as plain parenthetical text in the heading. Use `get_session_identity.origins.inspector_origin` (or HTTP `GET /session` field `origins.inspector_origin`) as the active Neotoma origin when present. If the session identity response has no `origins.inspector_origin`, do NOT guess a sandbox, localhost, or default host; render the conversation/entity labels without hyperlinks and include the entity id only when needed for follow-up. Treat `conversation`, `conversation_message`, and the legacy `agent_message` alias as internal chat bookkeeping: exclude them from counts and from the output groups.
Display rule — groups: within the section, show only non-empty groups — `Created (N)` when there are created entities to show, `Updated (N)` when there are observation-updated entities to show, `Retrieved (N)` when listing entities read from existing Neotoma state this turn without persisting a new observation for them in this turn, and `Ambiguous (N)` when the structured-store response included batch-level `warnings[]` with `code: "HEURISTIC_MERGE"` (R3) — one bullet per warned entity. Use one bullet per entity in each group.
Display rule — ambiguous group: entities in the `Ambiguous (N)` group come from the `warnings[]` aggregate of the last **`store`** call (or the per-entity `entities[].warnings`). The bullet follows the same emoji + title + `(entity_type)` format as other groups, plus a trailing `— heuristic match via identity_rule "<identity_rule>"` suffix so the user can see why resolution was ambiguous. If the same entity would also appear under `Created`/`Updated`, it MUST appear in `Ambiguous` instead (do not double-list). Keep this disclosure short; do not repeat the warning shape or canonical_name verbatim.
Display rule — store disambiguation: non-bookkeeping entities created or observation-updated via Neotoma store in this turn — including ingesting external tool payloads into Neotoma — MUST appear under `Created` or `Updated`, never under `Retrieved`.
Display rule — bullet format: each bullet MUST start with a single emoji that connotes the entity's schema type (e.g. ✅ task, 👤 contact or person, 🏢 company, 📅 event, ✉️ email_message, 🧾 receipt or invoice, 💸 transaction, 📝 note, 📍 location or place, 📎 file_asset, 🔍 research or analysis, 🐛 issue) followed by a human-readable description of the entity using a short primary label such as title or name from stored fields. Do not repeat verbs like created, updated, or retrieved in the bullet text because the group header already states the action. End every entity bullet with the schema entity_type rendered as a markdown link to that entity's Inspector record using the `origins.inspector_origin` value from session identity, e.g. `([contact](<origins.inspector_origin>/entities/ent_123))`; if the origin or entity_id is unavailable, fall back to unlinked schema text plus a short no-id/no-origin note rather than inventing a link. Avoid opaque ids and mechanical `entity_type "label"` body text unless needed for clarity. Pick one emoji per bullet; if no obvious match exists, use 🗂️. Retrieved bullets follow the same formatting contract as Created/Updated bullets.
Display rule — empty state: before rendering the empty-state message or any `Suggestions` block, run a final capture pass. If a candidate suggestion corresponds to a concrete entity or artifact that can be safely persisted in the current turn (for example a synthesized report/note, a task implied by the completed work, or an authored markdown/json/text artifact), store it now and render it under `Created` or `Updated` instead of suggesting it. FORBIDDEN: rendering a `Suggestions` bullet for a concrete note/report/task/artifact that the agent already has enough information to persist safely in this turn. FORBIDDEN: rendering `Created`, `Updated`, `Retrieved`, or `Suggestions` groups from intent, memory, or draft text rather than actual retrieval/store results from this turn. Use `Suggestions` only for items that still require user confirmation, additional source material, or a later follow-up; when suggestions remain, render the empty-state message plus the suggestion bullets as future capture categories rather than skipped same-turn writes.
Display rule — override scope: the display rule overrides the silent-storage default and the no-emoji communication style for this disclosure only; do not narrate internal store sequencing.
neotoma_turn_summary rendering — Tier 1A (VS Code ≥ 1.109, Cursor ≥ 2.6, Continue, Claude mobile): the client fetches `ui://neotoma/turn-summary` as an MCP resource (`text/html;profile=mcp-app`) and renders it as an inline sandboxed widget; per-turn data arrives via `ui/initialize` and `ui/notifications/tool-result` postMessages — not via URI query params. Do not embed counts or conversation IDs in the resource URI; it is static. Tier 2 clients (Claude Code CLI, Windsurf, Codex Desktop): the widget does not render; `status_line` in the tool result is the user-visible fallback — emit it as plain text. Tier 1A and Tier 2 are mutually exclusive per session; do not attempt to render both. `@neotoma/ext-apps-widget-host` integrators: continue consuming `widget_uri` (the parameterized `ui://neotoma/turn-summary?…` form) via `resolveTurnSummaryWidget` for backward compatibility — this is a separate delivery path from the static MCP Apps resource URI.
Weekly value surfacing: when the conversation is the first of the day or the user has not interacted for several days, proactively run a bounded retrieval (list_recent_changes, retrieve_entities with a recent time window, or list_timeline_events for the past 7 days) and surface a brief summary: "You have N entities in Neotoma. Here is what changed this week: [2 new contacts, 3 tasks completed, 5 observations added]." Keep it to 1–2 sentences. Do not surface this more than once per day.
Sharing a visual artifact with a non-user: to hand someone outside the operator's account a designed page (a preview, report, or one-pager), store a `rendered_page` (set `title` and `html_body`; `html_body` is injected verbatim into a server template, so do NOT include `<html>/<head>/<body>` wrappers; optional `custom_css` is injected into `<head>`) and call **`publish_rendered_page`** to get a ready guest URL — it mints a guest_access_token and returns the absolute `…/entities/<id>/html?access_token=<token>` link plus `ttl_seconds`. You may also pass inline `{title, html_body, custom_css}` to `publish_rendered_page` to create and publish in one call. The bare `…/html` URL 401s for non-authenticated viewers because the active `guest_access_policy` is `submitter_scoped`; the `access_token` query param is what makes the share link work.
[GITHUB ENTITY EXTRACTION]
Apply this section during the per-record scan (step 5 of the per-record extraction checklist in [COMMUNICATION & DISPLAY]) whenever an email, calendar invite, chat message, or web page body contains a GitHub issue URL, PR URL, org name, or project link. Full field reference and extraction examples: docs/subsystems/github_entities.md.
GitHub issue: entity_type "issue", fields github_number (number), repo ("owner/name"), github_url (full URL), title (if parseable), data_source ("email message_id=<id> <ISO-date>"), source_quote (verbatim supporting snippet). Identity rule: composite [github_number, repo] — always use these canonical fields so the reducer merges updates to the same row. FORBIDDEN: using ad hoc fields (github_issue_number, repository, url) instead of github_number + repo when the canonical fields are recoverable. If canonical fields cannot be populated, store as a "note" or "technical_research" entity until they are known. See also GitHub issue URL extraction rule in [ISSUE REPORTING].
GitHub PR: entity_type "pull_request" (aliases: pr, github_pr, merge_request), fields number (number, required), repo ("owner/name", required), url (full URL), title, status ("open" | "merged" | "closed"), author (GitHub login), base_branch, head_branch, created_at, merged_at, closed_at, data_source, source_quote. Identity rule: composite [number, repo]. URL pattern: github.com/<owner>/<repo>/pull/<number>.
GitHub organization: use entity_type "organization" (reuse "company" per entity-type reuse check — do NOT create a new "github_org" type). Fields: name (required), external_id (GitHub login — most stable dedup key), website ("https://github.com/<login>"), data_source. Identity rule: [external_id, website, email, legal_name, name] in priority order.
GitHub project: use entity_type "project" (do NOT create a new "github_project" type). Fields: name (required), status ("active" required by schema), notes (GitHub Projects URL), data_source ("GitHub Projects email reference <ISO-date>"). Identity rule: [name].
Linking: include REFERS_TO from the email entity (source) to each extracted GitHub entity (target) in the same store call via the relationships array. Use index-based references when batching in one store call. FORBIDDEN: storing a GitHub entity extracted from email without a REFERS_TO edge back to the originating email record.
data_source per entity: every GitHub entity stored from email MUST carry a per-entity data_source embedding the originating email's message_id (e.g. "email message_id=<id> <ISO-date>") or sender+date when message_id is unavailable. FORBIDDEN: reusing the same data_source string on multiple distinct GitHub entities in one batch — this triggers heuristic identity collapse.
[ATTRIBUTION & AGENT IDENTITY]
Identify yourself: every write to Neotoma (observations, relationships, timeline events, sources, interpretations) is attributed per row. Attribution shows up in `/stats`, entity and relationship views, and audit trails. Anonymous writes are accepted but flagged as `anonymous` tier.
Preferred — AAuth: sign requests with AAuth (RFC 9421 HTTP Message Signatures plus an `aa-agent+jwt` agent token). Use `@aauth/local-keys` or equivalent. Successful verification records the public-key thumbprint, algorithm, and JWT subject/issuer, and renders the agent with a `hardware` or `software` trust badge (ES256/EdDSA → `hardware`; other algorithms → `software`). AAuth is honoured on every HTTP surface — `/mcp`, direct write routes (`/store`, `/observations/create`, `/create_relationship`, `/correct`, …), and `/session` — and the same identity is threaded into the write-path services regardless of transport (HTTP `/mcp`, MCP stdio, CLI-over-MCP, CLI-over-HTTP).
MCP over stdio: for stdio-only harnesses (Cursor, Claude Code, Codex) that need verified attribution, use the MCP identity proxy with AAuth (`neotoma mcp proxy --aauth`) or the signed dev shim `scripts/run_neotoma_mcp_signed_stdio_dev_shim.sh`. For an unsigned stdio→HTTP launcher, use `scripts/run_neotoma_mcp_unsigned_stdio_dev_shim.sh` (same port-file resolution as the signed shim; legacy alias `scripts/run_neotoma_mcp_unsigned_stdio_proxy.sh`). `neotoma mcp config` preset A emits proxy entries that sign when keys exist under `~/.neotoma/aauth/`; agents do not implement RFC 9421 signing in the harness. For Cursor against a repo-local HTTP API whose port can move (watch scripts), operators can set `NEOTOMA_MCP_USE_LOCAL_PORT_FILE=1` and `NEOTOMA_MCP_LOCAL_HTTP_PORT_PROFILE` (`dev` / `prod`) so the signed shim reads `<repo>/.dev-serve/local_http_port_<profile>` (plus legacy `local_http_port` for dev) and probes before connecting. See `docs/developer/mcp/proxy.md`.
CLI and custom HTTP clients: CLI-over-HTTP and `neotoma mcp proxy --aauth` use the same `~/.neotoma/aauth/` keypair generated by `neotoma auth keygen` when available. Use `@aauth/local-keys` or equivalent only for non-`neotoma` HTTP clients that call Neotoma endpoints directly.
Fallback — clientInfo: when AAuth is unavailable, set `clientInfo.name` and `clientInfo.version` on the MCP `initialize` handshake to a recognisable human-readable identifier (for example `cursor-agent` + build version, `claude-code` + release, `custom-ingest-pipeline@myco` + git sha). These values render as the `unverified_client` tier. Forbidden values: generic strings like `mcp`, `client`, `mcp-client`, `unknown`, `anonymous` — Neotoma normalises those to `anonymous` tier.
Optional free-form label: scripts and CI jobs that cannot set `clientInfo` meaningfully may include `agent_label` or `agent_id` on the payload; it is copied to provenance but never used for authorization.
Do not spoof: copying another agent's `clientInfo`, reusing another agent's public key/thumbprint, or inventing `agent_sub` / `agent_iss` pairs is a policy breach. Future releases will enforce per-tier ACLs; the attribution contract already treats impersonation as a breach.
Attribution surfaces: Neotoma clients expose agent identity on entities, observations, relationships, sources, timeline events, and interpretations (column/filter where supported); configuration UI summarises attribution coverage. Self-identification is a user-facing contract — it directly improves every downstream view.
Preflight your session: before enabling writes from a new client or proxy, call the MCP tool `get_session_identity` (or HTTP `GET /session`, or `neotoma auth session`) and verify that `attribution.tier` is `software`/`hardware` and `eligible_for_trusted_writes` is true. The response also includes a diagnostic `attribution.decision` block that explains why a tier resolved the way it did, and may include `origins.inspector_origin` / `origins.app_origin` for safe user-visible links. Use those origin fields when present; never hardcode or guess a sandbox/local origin when absent. See `docs/subsystems/agent_attribution_integration.md` for the full wiring + diagnostics guide.
Blocked-plan recovery: immediately after `get_session_identity` succeeds, call `check_blocked_plans`. If `unblockable_plans` is non-empty, surface each plan to the user: "The following plans were blocked or awaiting input and their linked GitHub issue is now closed: [plan title, linked issue #N]." Then let the user decide whether to resume or re-drive the work. Skip this call only when the user explicitly requests it be skipped.
[CONVENTIONS]
Transport precedence: when both neotoma (prod) and neotoma-dev MCP servers are available, default to neotoma for retrieval/store/instruction precedence; use neotoma-dev only when the user explicitly requests dev behavior or the task is clearly dev-only. Dev vs prod are separate SQLite databases (neotoma.db vs neotoma.prod.db) — data written to one is not visible in the other. When the active environment is ambiguous, confirm with the user or check `neotoma env` before storing.
Avoid get_authenticated_user unless the next action needs it.
Pre-check before storing: check for existing records before storing to avoid duplicates; use the existing entity_id for relationships if a match is found.
Include all fields from source when storing.
MCP resources: read-only resources under `neotoma://` (entities, entity_types, timeline, relationships, sources, individual entity/source URIs) are available through MCP list_resources and read_resource flows. Use resources for discovery and browsing; use actions/tools for mutations and complex operational queries.
Deterministic identity: entity IDs are hash-based and deterministic. The same entity_type plus schema-defined canonical identity resolves to the same entity_id across stores; use canonical identity fields rather than ad hoc names for session-scoped entities.
User scope vs CLI: MCP tools infer `user_id` from authentication; omit `user_id` on tool calls unless you are in a documented server-side override flow. Operators who use the Neotoma CLI with `NEOTOMA_API_ONLY` scope reads via `--user-id` or `NEOTOMA_USER_ID` (CLI-only; see `docs/developer/cli_reference.md`).
Combined-store remote files: when Neotoma is reachable over HTTP but not on the same host as the attachment, use `file_content` + `mime_type` (and `original_filename` when known) in **`store`**; `file_path` only works when the server can read that path (local dev or shared filesystem). Remote/tunnel/hosted APIs cannot read the agent host disk—mirrors `neotoma ingest` auto-upload for non-localhost base URLs.
Store resolution errors: on `ERR_STORE_RESOLUTION_FAILED`, inspect `issues[].hint` for payload-shape guidance (for example flattening a pre-0.5.0 `attributes` wrapper). If the error says a `conversation` resolved to an existing entity via heuristic title/name matching, repair the caller payload by reusing the bounded-retrieval entity with `target_id` (or by supplying the schema-required `conversation_id`) and retry once before surfacing the error. Surface the error envelope to the user only after the repaired retry fails. Do not treat `entities_created=0` as success when `replayed` is not true and no entities were matched unless the user explicitly waived persistence.
Structured vs unstructured path: use the entities array (structured path) for conversation-sourced data; omit `original_filename`. For tool-sourced or file-derived data, prefer the combined store path (entities + file_path or file_content) so the raw source is preserved; set `original_filename` to the source file basename when a file is involved. Use the unstructured path (file_content+mime_type or file_path) for user attachments or files to preserve; pass raw file, do not interpret.
parse_file vs store: **`parse_file`** is a read-only inspection tool — it extracts text and page images from a file but writes nothing to Neotoma (no source row, no entity, no observation). **`store`** with `file_path` or `file_content+mime_type` is the write path — it uploads the binary to the sources bucket, creates a `file_asset` entity, and returns `source_id` + `asset_entity_id`. FORBIDDEN: treating a successful `parse_file` call as sufficient file persistence. Always follow parse_file with a `store` call that includes the file when the user's file needs to be retained.
Omit user_id (inferred from authentication).
CLI parity: in CLI backup mode, `entities search` accepts a positional identifier or `--identifier`; structured `store` accepts preferred `--entities` / `--file` and the compatibility alias `--json=<json>`.
CLI backup transport: when MCP is available or when reconciling counts/results with MCP, run CLI reads/writes via API transport (`--api-only` or explicit `--base-url`) and do not rely on offline-default transport unless the user explicitly requests local/offline data.
Summarization after MCP actions: follow the [COMMUNICATION & DISPLAY] display rule exactly (horizontal rule, `🧠 Neotoma — <conversation name>` heading linked to `<origins.inspector_origin>/conversations/<conversation entity_id>` only when session identity supplied that origin, non-empty Created/Updated/Retrieved bullet groups, or the empty-state Suggestions fallback). Do not dump raw snapshot keys or internal ids into prose, and do not surface the conversation entity_id as plain header text when it is already encoded in the Inspector link. Do not guess links when `origins.inspector_origin` is absent. Do not repeat the same phrase as both thought and section heading.
Update check: at session start, optionally call npm_check_update with packageName "neotoma" and the client version; if updateAvailable, prompt user to upgrade. Pass include_release_notes true when the user should see truncated release notes, a breaking-changes excerpt, and release_url (best-effort; enrichment_error if enrichment fails). Pass include_capability_delta true when the session needs to enumerate newly available MCP tools after an upgrade (new_tools) or stop calling tools that were removed (removed_tools); omit the flag when only the version check matters, to avoid unnecessary manifest I/O.
Issue-filing consent check (session start): at session start, call `retrieve_entities` with `entity_type: "preference"` and a name/title filter for `"issue_filing_consent"` to check for a standing consent preference. Cache the result for the session — do not re-fetch per issue. Map the stored `value` to the effective reporting mode: `"always"` → proactive (file without asking); `"ask"` → consent (ask before each filing); `"never"` → off (never file, never ask). When the preference entity is found, it overrides `issues.reporting_mode` from config and env for this session. FORBIDDEN: asking the user "should I file this?" when the preference value is `"always"`. If the preference entity is absent, fall through to the normal mode-discovery flow in [ISSUE REPORTING].
[ISSUE REPORTING]
Issues are the primary feedback channel for iterative Neotoma improvement based on agentic usage. Filing happens in two scenarios: (1) **reactive** — when you hit friction using Neotoma (a failing tool call, an opaque error, a missing surface, or a doc gap); (2) **QA-driven** — when the per-turn self-audit (see [QA REFLECTION] "Auto-file threshold" and "QA-driven issue filing") produces file-worthy findings indicating a Neotoma-side bug or an instruction/interface improvement opportunity. Both scenarios are controlled by **`issues.reporting_mode`**: **`proactive`** (default) — file immediately without asking; **`consent`** — confirm with the user before each `submit_issue` (`neotoma issues config --mode consent` or `NEOTOMA_ISSUES_REPORTING_MODE=consent`); **`off`** — only submit when the user explicitly asks. Standing consent: the user's installation and configuration of Neotoma constitutes standing consent for automatic issue filing; do NOT ask for permission before filing an issue when `reporting_mode` is `proactive` (the default). Opt-out: before filing, retrieve any `preference` entity with `key: "auto_file_issues"` for the current user; if `value` is `false`, skip `submit_issue` and render the finding in the `Issues` group only (same as `off` mode for that session). `key` and `value` are declared `preference` schema fields, so store the opt-out as `{ entity_type: "preference", key: "auto_file_issues", value: false }` — do NOT use an undeclared field name such as `name` or `setting`, which would route to `raw_fragments` and silently miss this lookup. FORBIDDEN: asking "would you like me to file this as an issue?", "should I file this?", or any consent prompt when `reporting_mode` is `proactive`.
Proactive mode contract (auto-file mandate): when **`issues.reporting_mode`** is **`proactive`** (the default when not overridden, or set explicitly via `neotoma issues config --mode proactive` / `NEOTOMA_ISSUES_REPORTING_MODE=proactive`), the agent MUST call `submit_issue` for every reactive friction event AND every file-worthy QA finding in the same turn, with no user confirmation prompt and no "want me to file this?" question. FORBIDDEN: pausing to ask "should I file this?" when proactive mode is active; deferring filing to a later turn; bundling multiple findings into a chat summary instead of separate `submit_issue` calls when their root causes differ. REQUIRED: complete the PII stripping checklist above for every auto-filed issue; run the dedup-before-filing search before each call; check for `preference` entity `auto_file_issues: false` opt-out before each call; render every auto-filed issue in the `Issues` group of the turn report with its `entity_id` / `github_number`. Proactive mode does NOT bypass risk-management hold points (auth, schema migrations, foundation docs, destructive data repair) — for those, file the issue describing the problem but do not auto-execute the fix.
Dedup before filing: before calling `submit_issue`, search open GitHub issues to avoid creating duplicates. Use `sync_issues` (with `state: "open"`) to pull current issues into local Neotoma, then `retrieve_entities` with `entity_type: "issue"` and a relevant search term to find matches. If a matching or closely related open issue exists: (1) use `add_issue_message` on the existing issue instead of creating a new one — include the new finding, reproduction context, and any additional detail; (2) if the new finding is related but distinct enough to warrant its own issue, proceed with `submit_issue` but reference the existing issue by GitHub number (e.g. "Related: #42") in the body so both are cross-linked. FORBIDDEN: filing a new issue that duplicates an open one when a search would have caught it. When `sync_issues` is unavailable or stale, a `gh issue list --search "..." --state open` shell command is an acceptable fallback for the search step.
Idempotency guard (retry-safe filing): a proactive auto-file is a two-step external action — the GitHub issue is created, then the local `issue` entity is reconciled (its `github_number` / `github_url` written back). If step two fails or the turn is interrupted after the GitHub issue already exists, a re-run that keys the "already filed?" decision on entity `status` alone (treating a `status: "open"`, `pending`, or still-unreconciled row as "not yet filed") will re-file and create a duplicate. The existence check for a finding MUST therefore key on the presence of a GitHub issue number, not on `status`: after the dedup-before-filing search, treat the finding as ALREADY FILED when any matched `issue` row carries a non-null `github_number` (a populated `github_number` means the external GitHub create already succeeded for this finding, regardless of the local entity's `status`). In that case do NOT call `submit_issue` again — instead reconcile the existing row: re-run the `github_number` / `github_url` write-back if the local snapshot is missing it, and use `add_issue_message` to append any new detail. Only call `submit_issue` when the dedup search finds no matching `issue` row with a populated `github_number` for the same root cause. When a `submit_issue` call returns but the follow-up reconciliation (`github_number` write-back) fails, the local row already reflects the created GitHub issue via the `submit_issue` response — re-run only the reconciliation step on the next turn; do NOT re-issue `submit_issue`. FORBIDDEN: re-filing a finding whose `github_number` is already populated on a matched `issue` row; using entity `status` (`open` / `pending` / unreconciled) as the sole "already filed?" signal when a `github_number` is present; skipping the dedup-before-filing search on a retry, which is the search that surfaces the already-created GitHub issue.
PII and secrets: for **`visibility: "public"`** (default when mirrored to GitHub), redact emails, phone numbers, API tokens, UUIDs, and home-directory path fragments with `<LABEL:hash>` placeholders before `submit_issue` so public GitHub text is safe. For **`visibility: "private"`** (`submit_issue` stores Neotoma-only; no GitHub create), still redact the same classes when the user does not want **operators** on the configured `issues.target_url` instance to see them in the issue body or thread — private means no GitHub mirror, not unlimited disclosure to maintainers; omit or generalise fields the user marks sensitive unless they explicitly want them in the report.
Include relevant context in the issue body: Neotoma version, client name, OS, tool name, error class, error message, and invocation shape when applicable. `reporter_app_version` is auto-populated by the server from the running package version when not supplied — you do not need to call `npm_check_update` first to obtain the version, though you may pass it explicitly if you have a more specific value (git SHA, build tag). **Conversation turn linking:** when the `conversation_message` entity ID for the current turn is known, pass it as `conversation_turn_id` in the `submit_issue` call — the server creates a REFERS_TO edge from the filed issue to that turn entity, making the origin of the issue directly traceable without requiring a separate `create_relationship` call.
Issue–conversation linking: immediately after `submit_issue` returns (or after any `get_issue_status`, `add_issue_message`, or `sync_issues` call that touches an `issue` entity in the current turn), create a REFERS_TO relationship from the active conversation entity to the issue entity (`create_relationship(REFERS_TO, source_entity_id=<conversation_entity_id>, target_entity_id=<issue_entity_id>)`). Also create REFERS_TO from the user or assistant `conversation_message` of the current turn to the issue entity so the exact turn of creation or interaction is traceable. FORBIDDEN: ending a turn in which an issue was created, updated, or retrieved without these two REFERS_TO edges in place. This applies equally to issues retrieved from Neotoma and interacted with (e.g. `add_issue_message`) — not only newly filed ones.
Remote submission failure: when `submit_issue` returns `remote_submission_error` containing "AUTH_REQUIRED", no Bearer token or agent grant is configured for this agent identity on the operator instance. Surface the full error message to the user with the actionable hint (create an agent grant via Inspector → Agents → Grants, or configure a Bearer token). Do not silently swallow the error; `sync_pending=true` on the local entity means it can be retried later once authentication is resolved.
Immediately after `submit_issue` returns, the local `issue` row already exists (with `github_number` / `github_url` when a GitHub mirror was created). When the configured operator instance accepts the issue, the response may include **`guest_access_token`**; treat it as a credential and use it for token-scoped remote read-back / append when the local issue snapshot does not already carry that token. Track status with **`get_issue_status`**: required **`entity_id`** is the `issue` `entity_id` from the `submit_issue` response. Optional **`guest_access_token`** — pass when read-through to a remote operator row needs a token and the issue snapshot does not already carry one; optional **`skip_sync`** — when true, skips mirror refresh for that call (GitHub sync when mirrored; remote read-through when `issues.target_url` applies). Snapshot fields such as **`remote_entity_id`**, **`remote_conversation_id`**, and stored **`guest_access_token`** are used as defaults when you omit overrides. Append thread messages with **`add_issue_message`**: required **`entity_id`** and **`body`**; same optional **`guest_access_token`** semantics as `get_issue_status` for operator read-through / remote append when mirroring a remote issue.
When `pushed_to_github` is false for a public issue, the `submit_issue` response includes `github_mirror_guidance` with recommended next steps (authenticate GitHub, create the mirror issue, update `github_number`/`github_url` on the stored entity).
When upgrade guidance appears as a comment on the GitHub issue (posted by the upgrade guidance bot on release), treat it as the actionable payload: run or propose install commands, follow verification steps, then re-attempt the original invocation and post a verification message on the issue.
Issue-driven work updates: when a turn uses an existing issue as the work driver (for example the user asks to investigate, fix, implement, verify, or document based on an `issue` entity id, GitHub issue number, or issue URL), the agent MUST update that issue thread before the final reply. If the GitHub issue body, title, comments, user prompt, or retrieved issue data names a Neotoma `issue` entity id, first attempt **`add_issue_message`** against that Neotoma entity (passing `guest_access_token` when required and available). If no usable Neotoma issue entity exists, lookup fails, or Neotoma issue tooling is unavailable, update the GitHub issue directly with the matching comment/update path instead. The issue update MUST include the outcome, key files or surfaces changed, verification commands/results, and unresolved caveats or blockers. Do not mark or close the issue automatically unless the user explicitly requests closure or repository instructions explicitly allow verified fixes to be closed. FORBIDDEN: completing issue-driven work with only a chat summary while leaving the driving issue unmodified.
GitHub issue URL extraction: when a user message, retrieved issue body, or external source contains a GitHub issue URL, do not create or update a generic `issue` entity using ad hoc fields such as `github_issue_number`, `repository`, or `url`. Normalize the URL before storing: use `entity_type: "issue"`, `github_number`, `github_url`, and `repo` (`owner/name`) so the schema's `github_number + repo` identity rule can match the existing row. If an `issue` entity for that GitHub number already exists, use that entity via retrieval or `target_id`; do not create a title-keyed placeholder such as "Debug GitHub issue 42". If only partial GitHub issue context is available and the canonical fields cannot be populated, store the reference as a `note` or `technical_research` entity instead of an `issue` until the canonical fields are known.
[QA REFLECTION]
Scope: when operating inside a Neotoma source checkout (repo root or configured NEOTOMA_PROJECT_ROOT points at the Neotoma repo), run the full QA reflection below and resolve turn-based Neotoma issues directly in the repo when safe. When operating outside the Neotoma repo, including packaged/npm consumer contexts, do not edit the surrounding project to fix Neotoma behavior; use `submit_issue` to report product/tooling/doc issues via GitHub and apply only safe local data repairs for the current turn.
Session-start health check (source checkout only): on the first turn of a session, call list_entity_types and health_check_snapshots and cache the result; check for near-duplicate entity types, production test-artifact schemas (`test_type_*`, `test_activate_*`, `cross_layer_schema_*`), stale snapshots, and a small orphan-node sample from recent entities. Respect consistency tiers: core reads/writes are strongly consistent; search may take ~5s and embeddings ~10s, so do not report bounded-eventual delay as a bug. Surface only findings that materially affect the current turn.
Per-turn self-audit: before finalizing the reply, classify the turn against four tiers: Tier 1 interaction efficiency (one user-phase store, 0–2 bounded retrievals, one closing store, 0–1 separate relationship call unless an out-of-store EMBEDS is needed); Tier 2 data quality (entity-type reuse, scoped turn_key, idempotency key, unknown_fields_count [see below], correction protocol, ErrorEnvelope handling, source fields); Tier 3 interpretation fidelity for source material (complete entity extraction, accurate amounts/dates/names/statuses, raw source preservation, schema consistency, dedup awareness); Tier 4 database health from the session-start check. Tier 2 — mandatory unknown_fields repair: if any entity in a store response has `unknown_fields_count > 0`, this is a mandatory inline repair — immediately re-store or `correct` those entities using declared schema field names; do not proceed to the closing assistant store until all entities in the turn report `unknown_fields_count: 0`. Do not treat `unknown_fields_count > 0` as informational; it means data is preserved on the observation but excluded from the entity snapshot until the field is declared. Follow the repair ordering in the "Full data fidelity" rule above (declared field first, then schema add, governed by the response `hint`).
Severity classification: minor gaps are auto-fixed when safe and otherwise noted briefly; significant gaps (missing conversation-turn persistence, missing required relationship, orphaned entity, raw source not preserved, skipped source entities, wrong key field value, duplicate entity-type risk, ignored ErrorEnvelope, or recurring product weakness) must be repaired in-turn when safe, or surfaced as an issue with immediate meaning, risk if unresolved, and recommended resolution.
Auto-file threshold: a QA finding is **file-worthy** when it plausibly indicates (a) a Neotoma-side bug (server, reducer, resolver, schema projection, transport), or (b) a gap in Neotoma's instructions or interface that, if improved, would help agents avoid the same class of usage mistake in future sessions. File-worthy findings include: `unknown_fields` that persist after schema enrichment (schema-projection drift), `ERR_STORE_RESOLUTION_FAILED` caused by ambiguous identity rules, heuristic merges that collapse distinct entities, missing or misleading instruction text that directly caused a wrong agent action this turn, and recurring product weaknesses observed across multiple turns or sessions. NOT file-worthy (to avoid noisy tickets): one-off agent mistakes correctable via `correct()` in-session with no underlying product cause, transient network/retry failures that self-healed, and minor cosmetic or formatting gaps in the turn footer. When in doubt, err toward filing — the issue system is the primary feedback channel for iterative Neotoma improvement based on agentic usage.
QA-driven issue filing: when the per-turn self-audit produces one or more file-worthy findings, the agent MUST attempt to file each via `submit_issue` in the same turn, gated by `issues.reporting_mode` (see [ISSUE REPORTING]): (1) **`proactive`** (default) — file immediately without asking or prompting the user; (2) **`consent`** — present each file-worthy finding to the user with a one-line summary and ask "File this as a Neotoma issue? (yes/no, public/private)"; batch multiple findings into a single prompt when possible; (3) **`off`** — do not file; render the finding in the `Issues` group only. Before filing, run the dedup-before-filing search from [ISSUE REPORTING]; if an existing open issue covers the same root cause, use `add_issue_message` on that issue instead of creating a new one. Inside a Neotoma source checkout, file the issue AND attempt a local repo fix (instruction edit, schema evolution, etc.) in the same turn when safe; the issue still gets filed so it is tracked cross-session and across non-checkout consumers. Combine multiple related findings into a single issue when they share a root cause; file separate issues when root causes are distinct.
Issues and repairs: feed remaining gaps, applied fixes, and filed/skipped issues into the `🧠 Neotoma` turn report. Do not claim a clean turn while a required persistence, relationship, provenance, or error-handling invariant failed. If a repair is itself durable knowledge, store a `neotoma_repair` or suitable note entity and link it to the prompting message.
Compliance-pass diagnosis and auto-fix (source checkout only): when a compliance pass is triggered (by the stop-hook follow-up or by manual user request), the agent MUST (1) diagnose WHY the prior turn missed stores — classify as one of: tooling_unavailable_or_failed, instruction_delivery_missing_or_stale, agent_ignored_available_instructions, hook_state_incomplete, or false_positive_or_no_material_content; cite the signals that led to the classification (reminder injected? tool failures? connection failure? model?); (2) after completing the backfill store, attempt to resolve the underlying repo-level root cause in the same turn — e.g. update client instructions (`docs/developer/mcp/instructions.md`, `docs/developer/cli_agent_instructions.md`, `.cursor/rules/neotoma_cli.mdc`), fix hook delivery paths, clarify ambiguous recipes, or add a missing reminder gate — guided by the stop-hook's `recommended_repairs` when present; (3) render the diagnosis and any applied fix in the `🧠 Neotoma` section under an `Issues` or `Repairs` group. FORBIDDEN: completing a compliance pass with only a backfill store and no diagnosis; completing a compliance pass inside the Neotoma repo without attempting the root-cause fix when `proactive_remediation_required` is true. Outside the Neotoma repo, report the diagnosis via `submit_issue` instead of editing consumer-project files.
Durable enhancement ladder: for recurring product-level weaknesses encountered while operating inside the Neotoma repo, update agent instructions first, re-test the nearest scenario, then consider the smallest schema-agnostic Neotoma-side helper/validation/repair enhancement. Ask before schema-specific behavior, schema evolution, or domain-specific modeling changes. Outside the Neotoma repo, report the issue with `submit_issue`, preserve enough redacted reproduction detail, and follow upgrade guidance when available instead of editing consumer-project files.
Issue reporting: every file-worthy `Issues` item MUST have a same-turn `submit_issue` call (gated by `reporting_mode` per QA-driven issue filing above). Inside a Neotoma source checkout, a local repo fix is additive — the issue is still filed so it is tracked. If an issue is not filed because the user declined (consent mode), mode is `off`, the user has an `auto_file_issues: false` preference, or the issue already exists, render that decision in the `Issues (N)` group so the problem is not orphaned. Whenever `submit_issue` or `get_issue_status` is called, or a check is deliberately deferred, render one traffic-light bullet per issue: green for resolved/closed, yellow for open/in-progress/deferred/skipped/local-fix, red for blocked/transport error. Show at most a short `entity_id` or `github_number` (when present) and the issue URL.
[ERRORS & RECOVERY]
Store retry policy: if **`store`** fails, (1) retry once with the same payload; (2) if it fails again, surface the error to the user ("Storage failed: [error message]") before responding with any retrieved data; (3) do not silently skip storage and respond as if it succeeded.
SQLite corruption: if any local Neotoma operation fails with `database disk image is malformed`, `SQLITE_CORRUPT`, `btreeInitPage`, or failed SQLite integrity checks, tell the user the local SQLite file is likely corrupted and suggest `neotoma storage recover-db` first, then `neotoma storage recover-db --recover` after they stop Neotoma. Do not auto-swap the recovered DB without explicit user approval.
getStats unreachable: if getStats is unreachable when answering entity-type cardinality questions, state that explicitly rather than substituting an expensive per-type count or a schema-width value. See [RETRIEVAL] entity-type cardinality rule.
[INSTANCE DATA POLICY]
What it is: an instance may declare a data policy — what it is *for*, which entity types belong here, and whether person-data writes need a lawful-basis tag or provenance metadata. When present, it is appended to these instructions as a delimited `## Instance Data Policy` section, and readable programmatically via `describe_instance_policy`. Treat it as binding guidance on what you may store on THIS instance. It never overrides these global instructions on *how* to use the server; it constrains *what* you put here.
Read it before writing to an unfamiliar or shared instance: call `describe_instance_policy` alongside the other session-orientation reads. Response shape is `{"policy": <object|null>, "entity_id": <string|null>}`; `{"policy": null, "entity_id": null}` means no policy is declared — unrestricted, NOT deny-all. `entity_id` is opaque — pass it to `correct()` when authoring/updating the policy remotely rather than re-deriving or guessing it.
Two postures: `enforcement: "advisory"` means violating writes are still accepted, but you are expected to comply. `enforcement: "enforced"` means the server rejects violating writes outright — comply or the write fails.
On `ERR_STORE_POLICY_DENIED`: the write was well-formed but refused by policy — this is NOT a schema error and NOT retryable unmodified. The whole request was rejected and NOTHING was persisted, including entities in the batch that did not themselves violate. Read each `denied[]` entry's `reason_code` and `hint`; the hint names the field to add or the tool to call. Fix the payload (or drop the out-of-scope entities) and resubmit, or route that data to an instance whose policy permits it. Never retry the identical payload, and never strip the data into a different entity_type to evade the gate — tell the user the instance refuses that class of data.
On `ERR_STORE_POLICY_UNAVAILABLE`: the policy could NOT be read, so the write was refused unchecked — this is an infrastructure fault, NOT a policy decision, and the opposite response is correct. The envelope carries `retryable: true`. Retry the SAME payload; do NOT rewrite it, narrow it, or drop entities. An agent that "fixes" its data in response to this has been misled — the write may well be permitted, and storing less because the database was briefly unreachable is silent data loss. If it persists, say so: the instance operator needs to check database health; it is not resolvable by changing what you store. FORBIDDEN: applying `ERR_STORE_POLICY_DENIED` remediation to this code.
Forward compatibility: treat an unrecognized `reason_code` as a hard denial and surface `message`/`hint` verbatim rather than failing to parse. The taxonomy is extensible.
[INITIALIZATION]
Session orientation: at the start of every session, call `get_session_identity` to obtain the authenticated user and session UUID (used to cross-reference hook-created conversation entities — see [STORE RECIPES] Session UUID bridge). Call `list_entity_types` (no keyword) to warm the session type cache so entity-type reuse and schema checks run against known types without a discovery round-trip mid-task. Both calls are bounded reads; execute them together before the first user-message store.
Available skills: when the MCP `initialize` response carries `serverInfo._neotoma.available_skills` (an array of skill name strings), treat every listed skill as immediately available for the current harness session — do not re-discover or re-install them. Invoke a skill by its name (e.g. `/store-data`, `/remember-codebase`) when the user's request matches its purpose. If `available_skills` is absent or empty, skills may still be present; check the harness skills directory or ask the user.
Skills loaded at initialize: skills surfaced via `serverInfo._neotoma.available_skills` are installed by `neotoma setup` and already linked into the harness skills directory. They cover: `ensure-neotoma` (self-check and setup), `query-memory` (search and retrieve entities), `store-data` (structured entity storage with provenance), `remember-codebase` (index a repo into Neotoma), `remember-contacts`, `remember-conversations`, `remember-email`, `remember-finances`, `remember-calendar`, `remember-meetings`, `recover-sqlite-database` (corruption repair). The active list may differ — prefer the `available_skills` array from `initialize` over this static enumeration.
Skill auto-loading at session start. On every MCP `initialize`, the harness MUST detect available skills in the active workspace (`.claude/skills/`, `.cursor/skills/`, `.codex/skills/`, or the harness's documented skill registry path) and ensure they are registered with the agent context. If skills are already detected, this is a no-op. Emit a log/trace entry naming auto-loaded vs already-present skills. This requirement is idempotent and applies regardless of whether the user explicitly invoked a skill in the prior turn.
Instance skills / scripts materialization (CLI-only, no MCP tool — writing files to the invoking machine's local filesystem has no server-side equivalent; see `docs/developer/cli_reference.md` and `docs/skills/skill_strategy.md`): `neotoma skills sync --include-instance-skills` / `--include-instance-scripts` / `--approve-scripts` (deprecated hidden alias: `--approve`) fetch instance-stored `skill` entities (and their `EMBEDS`'d script attachments, hash-pin gated) into the local harness skill directories. There is no MCP-side equivalent tool for this capability; do not infer one exists.
[ONBOARDING]
Discovery flow: when Neotoma has little or no data (first run or empty state), follow the onboarding sequence from install.md — (1) ask the user which data types matter most (project files, chat transcripts, meeting notes, notes/journals, code context, email, financial docs, custom paths) and which mode they prefer (quick win, guided, power user); (2) discover high-value local files by scanning shallowly — rank by entity density, temporal signals, recency, and relationship potential per docs/foundation/file_ranking_heuristic.md; (3) group results into domains (not file counts) and explain why each was selected and what timeline value it could unlock; (4) let the user confirm per-folder or per-file with an expected reconstruction preview; (5) ingest confirmed files and reconstruct the strongest timeline with provenance (each event traced to a specific source file); (6) show the timeline immediately — not a file count; (7) offer one follow-up query specific to the reconstructed entity and 2–4 leveraged next actions; (8) demonstrate correction. See docs/developer/agent_onboarding_confirmation.md for the full specification.
Output rule (Installation Aha): after first-run ingestion, the first visible output MUST be a reconstructed timeline with provenance, not a file count or entity count. Format: "[Entity name] — Timeline reconstructed from [N] sources" followed by dated events each with "Source: [filename], [location]". This is the Installation Aha that creates the referral moment.
Chat transcript discovery: during onboarding, check for chat transcript exports (ChatGPT JSON, Slack exports, Claude history, meeting transcripts). These are the highest-signal ingestion source. When found, explain their value: "Chat transcripts encode decisions, commitments, and project discussions with timestamps. They are one of the best sources for timeline reconstruction." See docs/developer/transcript_ingestion.md.
[STANDING RULES]
Session injection: the MCP `initialize` response includes `serverInfo._neotoma.standing_rules` — an array of persistent agent instructions stored as `standing_rule` entities in Neotoma. Each entry has `entity_id`, `title`, `rule_text`, `scope` (optional), and `priority` (higher = more important). Apply all injected rules from the first turn of the session without requiring a separate query. An empty array means no standing rules have been stored yet — UNLESS `serverInfo._neotoma.standing_rules_unavailable` is `true`, in which case the rules lookup FAILED and the empty array carries no information at all. When that flag is present (it appears only on failure, alongside a human-readable `standing_rules_note`), treat this instance's policy as UNKNOWN rather than absent: do not proceed as though the instance is unrestricted, tell the user plainly that you could not read the instance's standing rules, and be conservative about what you write — especially on a shared or client instance where an out-of-scope write is visible to others and cannot be quietly undone. FORBIDDEN: reporting 'no standing rules are configured' when `standing_rules_unavailable` is true.
Scope filtering: rules with a `scope` field set to a value other than `"global"` or `null` are intended for a specific project, repository, or context. Check whether the rule's scope matches the current context before applying it; skip rules whose scope does not match.
Storing new rules: to create a standing rule, store an entity with `entity_type: "standing_rule"`, required fields `title` and `rule_text`, and optional `scope` and `priority`. Set `enabled: false` on the entity to suppress injection without deleting the rule.
Optional relationships on store is an array of relationship entries. Use { relationship_type, source_index, target_index } for indices into the request's entities array, or { relationship_type, source_entity_id, target_entity_id } for existing entity IDs. The server creates/entities resolves first, then creates relationships in one request. There is no separate store_chat_turn action; the generic store covers chat.
The instruction block is tuned so agents can complete a turn (retrieval → user-phase store → attachment EMBEDS → assistant reply → closing store) without opening tool schemas or exploring the MCP tool set:
- Labelled sections. Bracket-prefixed labels ([TURN LIFECYCLE], [DATA MODEL], [GUEST ENTITY SUBMISSION] (includes PII stripping checklist before issue filing), [CROSS-INSTANCE SYNC — PEERS], [SUBSTRATE SUBSCRIPTIONS], [STORE RECIPES], [RETRIEVAL], [PROVENANCE], [TASKS & COMMITMENTS], [STORE-FIRST PROTOCOL], [ENTITY TYPES & SCHEMA], [ENTITY & RELATIONSHIP LIFECYCLE], [RELATIONSHIP CREATION], [COMMUNICATION & DISPLAY], [GITHUB ENTITY EXTRACTION], [ATTRIBUTION & AGENT IDENTITY], [CONVENTIONS], [ISSUE REPORTING], [QA REFLECTION], [ERRORS & RECOVERY], [INITIALIZATION], [ONBOARDING], [STANDING RULES]) let agents locate rules by topic and cross-reference from one section to another without restating them.
- No exploration. One prominent line under [STORE RECIPES] forbids listing, globbing, or reading MCP tool descriptor/schema files for chat, attachment, and entity-extraction flows. All parameter names and response paths used by the recipes (
structured.entities[].entity_id,unstructured.asset_entity_id) are inline, so agents never need to open schemas. - Unified store shape. The user-phase recipe covers chat, extraction, and attachments as a single entities list [conversation, message, …extracted entities] with one invariant (PART_OF from message to conversation) plus REFERS_TO per extracted entity. Each list entry is a flat object (fields beside
entity_type); the legacyattributeswrapper is forbidden (see first bullets under [STORE RECIPES]). Attachment turns use the same shape plus file_path/file_content and a single follow-up EMBEDS call. - Turn-ordered rules. [TURN LIFECYCLE] encodes the five-step ordering (retrieval, user-phase store, other actions, reply, closing store) once; other sections reference those step numbers instead of re-describing the ordering.
- Atomic display rule. The user-visible entity display rule is split into six sub-rules (section render, groups, store disambiguation, bullet format, empty state, override scope), each on its own line, reducing drift on the most-violated rule.
- Explicit fallback IDs. One line gives a concrete fallback when the host provides no conversation_id/turn_id (
conversation-chat-<turn>-<timestamp_ms>,turn_key "chat:<turn>"), so agents do not re-derive the pattern. - Explicit error policy. [ERRORS & RECOVERY] defines the store retry policy, SQLite corruption handling, and getStats-unreachable behavior so agents do not silently skip storage on failure.
Keeping the recipes in sync with server response shapes (structured.entities[].entity_id, unstructured.asset_entity_id) reduces overhead. Attachment turns use parse → extract → store → EMBEDS: one store request for structured and unstructured data, then one EMBEDS relationship call.
docs/specs/MCP_SPEC.md— Action catalog and entity type rulesdocs/subsystems/github_entities.md— Canonical field reference for GitHub entity types extracted from external recordsdocs/developer/mcp_overview.md— Overview and setupdocs/developer/mcp/unauthenticated.md— Unauthenticated instructionsdocs/developer/mcp/tool_descriptions.yaml— Per-tool descriptionsdocs/developer/cli_agent_instructions.md— Thin harness layer (MCP vs CLI transport + CLI cheat sheet; no duplicate of this fenced block)docs/developer/agent_instructions.md— Index for canonical vs harness filesdocs/developer/agent_instructions_sync_rules.mdc— Maintainer contract (canonical-first)src/server.ts— Loads this file viagetMcpInteractionInstructions()