usage_digest is a periodic, PII-safe aggregate telemetry entity. It captures a rollup of operational health and usage metrics for a Neotoma deployment over a bounded time window. It is complementary to — not a replacement for — the per-event harness_event and per-incident daemon_report entity types:
harness_event— one row per turn or operation (high-cardinality, event-level).daemon_report— one entity per incident or anomaly (exception-oriented).usage_digest— one entity per observer × period (aggregated rollup for trend queries).
A usage digest is a drill-down companion: retrieve_entities(entity_type: "usage_digest") gives the trend view; the related daemon_report rows for the same period give the incident drill-down.
Usage digests serve baseline observability goals:
- Trend analysis: is error rate going up or down over the last 30 days?
- Capacity: which entity types or tool calls are growing fastest?
- Quality feedback: are there recurring friction notes that signal UX gaps?
- Attribution: which agent (
aauth_sub) is driving the most operations?
The entity is designed to be safe to store in shared or multi-tenant Neotoma instances. Free-text fields (friction_notes, notes) are protected by defense in depth: emitters SHOULD redact client-side before submission, and the Neotoma store seam ALSO scans and redacts these fields on ingest for usage_digest entities as a server-side backstop (so a misconfigured guest emitter cannot silently store raw PII). See Redaction below.
The usage_digest entity type is registered in src/services/schema_definitions.ts.
| Field | Type | Notes |
|---|---|---|
schema_version |
string | Always "1.0" for digests built against this spec. |
period_start |
string (ISO-8601) | Start of the digest window, inclusive. Use RFC 3339 format (2026-06-01T00:00:00Z). Must be string, not date — lexicographic sort must match temporal order for sort_by: "snapshot.period_end" to work correctly. |
period_end |
string (ISO-8601) | End of the digest window, exclusive. Used as the time-series sort key. |
reporter_channel |
string | Observer slug, e.g. "lemonbrand-observer". Identifies the emitter instance. |
| Field | Type | Notes |
|---|---|---|
aauth_sub |
string | Agent attribution. Mirrors daemon_report convention. |
reporter_app_version |
string | Semver of the observer app, e.g. "1.4.2". |
reporter_git_sha |
string | Short SHA of the deployed build. |
operation_counts |
object | Opaque. See Object field shapes below. |
error_rate |
number | Fraction [0, 1] of operations that errored in the period. |
error_counts |
object | Opaque. See Object field shapes below. |
entity_type_usage |
object | Opaque. See Object field shapes below. |
tool_usage |
object | Opaque. See Object field shapes below. |
compliance_signals |
object | Opaque. Agent-instruction adherence metrics ("is Neotoma used as intended"). See Object field shapes and Compliance signals below. |
friction_notes |
array | Array of strings. PII SHOULD be redacted client-side; the store seam also redacts on ingest. See Redaction. |
effectiveness_signal |
string | Enum: excellent | good | fair | poor | unknown. Enforced by convention, not the registry. |
notes |
string | Free-text operational notes. PII SHOULD be redacted client-side; the store seam also redacts on ingest. |
redaction_salt |
string | Shared salt used for client-side PII redaction (see Redaction). |
The registry stores these fields as opaque JSONB objects. The schema does not validate their internal shape. This document IS the contract.
{
"total": 4821,
"by_operation": {
"store": 3100,
"retrieve_entities": 980,
"correct": 410,
"retrieve_entity_by_identifier": 331
}
}{
"total": 47,
"by_error_code": {
"ERR_STORE_VALIDATION_FAILED": 28,
"ERR_STORE_RESOLUTION_FAILED": 12,
"ERR_NOT_FOUND": 7
}
}{
"by_entity_type": {
"conversation": { "store_count": 214, "retrieve_count": 88 },
"task": { "store_count": 156, "retrieve_count": 304 },
"harness_event": { "store_count": 3021, "retrieve_count": 12 }
}
}{
"by_tool": {
"store": { "call_count": 3100, "error_count": 18 },
"retrieve_entities": { "call_count": 980, "error_count": 4 },
"correct": { "call_count": 410, "error_count": 7 }
}
}An array of redacted free-text strings. Each entry should be a brief description of a UX friction point observed during the period. Examples after redaction:
[
"retrieve_entities returned empty for entity_type=task despite known stored entities",
"store rejected with ERR_STORE_VALIDATION_FAILED on field <UUID:a1b2c3d4>"
]operation_counts / error_* answer "is Neotoma working?" They do not answer "is the agent using Neotoma as intended per the MCP/CLI instruction contract?" — an agent can post low error rates while systematically skipping the closing assistant store or orphaning entities. compliance_signals carries that second signal: each metric maps to a specific behavioral mandate in the agent instruction contract (docs/developer/mcp/compact_instructions.md, docs/specs/MCP_SPEC.md).
All rates are fractions in [0, 1] (1.0 = fully compliant); counts are non-negative integers (0 = no violations). Every field is optional — an emitter SHOULD only populate the metrics it can actually compute from the data it observes. Do not emit a metric you cannot compute (e.g. an observer that only sees CLI/MCP JSONL cannot reliably compute reply-text citation rates). Omission means "not measured," not "100%."
The emitter computes most of these from the observer JSONL (the chronological log of MCP/CLI calls + their request/response payloads). Confidence reflects how reliably a client-side observer can compute each:
- high — derivable directly from the JSONL call sequence + payloads (ordering, presence/absence of calls, field formats, response flags).
- medium — needs turn/transcript alignment or message-type detection the observer may only partially see.
- low (aspirational) — needs NLP over reply text or server-side graph data an external observer typically lacks. Document, but most emitters will omit these.
{
"compliance_signals": {
"turn_lifecycle": {
"retrieval_completeness_rate": 0.97, // high — retrieve_* precedes user-phase store on implied-entity turns
"user_phase_store_rate": 0.99, // high — each user turn has a unified store call
"closing_store_completion_rate": 0.98, // medium — assistant-reply turns have a paired :assistant store
"step_ordering_violation_count": 2, // high — non-Neotoma tools called before retrieval/store
"closing_store_skip_count": 1 // medium — reply produced, no closing store
},
"relationships": {
"closing_relationship_completion_rate": 0.96, // medium — closing store has PART_OF to conversation
"relationship_batch_efficiency_rate": 0.9, // high — relationships sent in store vs separate calls
"per_turn_entity_linkage_rate": 0.94 // medium — touched entities have REFERS_TO from the turn
// "reply_citation_completeness_rate" omitted — low/aspirational; needs NLP on reply text
},
"schema_fidelity": {
"unknown_fields_repair_rate": 1.0, // high — unknown_fields_count>0 followed by repair in same turn
"schema_check_hit_rate": 0.92, // high — known entity_type stores preceded by describe/retrieve
"flat_entity_shape_violation_count": 0, // high — no nested-attributes payloads
"data_fidelity_violation_count": 0 // medium — proxy: unknown_fields_count>0 left unrepaired
},
"turn_identity": {
"turn_key_format_compliance_rate": 1.0, // high — turn_key carries conversation_id prefix
"conversation_id_stability_rate": 1.0, // high — one conversation_id per visible thread
"idempotency_key_uniqueness_rate": 1.0, // high — distinct idempotency_key per store in a turn
"closing_message_key_conflict_count": 0 // high — :assistant suffix present, no collision
},
"retrieval_behavior": {
"retrieval_before_fallback_rate": 0.95, // high — Neotoma retrieval before native-tool fallback
"category_retrieval_compliance_rate": 0.9 // medium — list-intent queries call retrieve_entities
},
"coverage": {
"session_coverage_rate": 1.0, // high — sessions with >=1 store call
"rapid_fire_store_frequency_rate": 0.97 // medium — stores at least every 3–5 turns in rapid sessions
},
"security": {
"pii_stripping_compliance_rate": 1.0, // high — submit_issue bodies free of detectable PII
"issue_linking_completion_rate": 0.93 // high — submit_issue followed by REFERS_TO links
},
"summary": {
"overall_mandate_compliance_rate": 0.96, // emitter-defined weighted aggregate of populated rates
"critical_violations_count": 3 // total FORBIDDEN-pattern occurrences (skip-session, only-user-msg, no-closing-store)
}
}
}The groups above are the families an emitter SHOULD aim to cover; the exact key set is a documented convention (the registry stores the object opaquely). Add families as the instruction contract evolves — additive, no schema migration. Each metric's mandate, violation signature, and confidence tier are tracked in the design analysis attached to issue #1569.
The entity's canonical identity is the composite [reporter_channel, period_start, period_end]. This three-field composite prevents silent deduplication of replayed digests from different channels or different periods.
name_collision_policy: "reject" is set on this entity type. A second store with the same reporter_channel + period_start + period_end will return ERR_STORE_RESOLUTION_FAILED rather than silently merging or creating a duplicate. If you need to amend a digest, use correct instead.
The recommended idempotency key for store calls is:
usage-digest-<reporter_channel>-<period_end>
For example: usage-digest-lemonbrand-observer-2026-06-01T00:00:00Z.
{
"entity_type": "usage_digest",
"idempotency_key": "usage-digest-lemonbrand-observer-2026-06-01T00:00:00Z",
"schema_version": "1.0",
"period_start": "2026-05-25T00:00:00Z",
"period_end": "2026-06-01T00:00:00Z",
"reporter_channel": "lemonbrand-observer",
"reporter_app_version": "1.4.2",
"operation_counts": { "total": 4821, "by_operation": { "store": 3100 } },
"error_rate": 0.0097,
"error_counts": { "total": 47, "by_error_code": { "ERR_STORE_VALIDATION_FAILED": 28 } },
"entity_type_usage": { "by_entity_type": { "task": { "store_count": 156 } } },
"tool_usage": { "by_tool": { "store": { "call_count": 3100, "error_count": 18 } } },
"friction_notes": ["retrieve_entities returned empty for entity_type=task despite known data"],
"effectiveness_signal": "good"
}Two paths are supported:
- AAuth grant — issue a grant with
{ op: "store", entity_types: ["usage_digest"] }to the reporting agent'saauth_sub. This is the recommended path for automated observers. - Keyless guest — the
usage_digesttype ships withguest_access_policy: "submit_only"(seeguest_access_policy.md), so an external observer can submit digests via an unauthenticatedstorecall without registering an AAuth keypair.submit_onlymeans guests may write but not read digests back — appropriate for a telemetry sink (free-text fields are redacted server-side on ingest as a backstop; see Redaction). An operator can override per deployment via theNEOTOMA_ACCESS_POLICY_USAGE_DIGESTenv var or by changing the schema metadata.
{
"entity_type": "usage_digest",
"sort_by": "snapshot.period_end",
"sort_order": "desc",
"limit": 30
}sort_by: "snapshot.<field>" is a supported sort form in retrieve_entities. The period_end field is an ISO-8601 string, so lexicographic order matches temporal order (as required by the schema design decision). This makes descending period_end sort work correctly without a date-aware index.
To filter by channel:
{
"entity_type": "usage_digest",
"sort_by": "snapshot.period_end",
"sort_order": "desc",
"snapshot_filters": { "reporter_channel": { "op": "eq", "value": "lemonbrand-observer" } }
}A usage digest COVERS its period. The individual daemon_report entities emitted during that period provide the drill-down. Clients that build dashboards can:
- Retrieve the digest for the period to get aggregate error rate and counts.
- Retrieve related
daemon_reportentities (sorted bylast_observation_atdesc) to see the individual incidents.
The relationship from a digest to its daemon_report rows is expressed as:
{
"from_entity_id": "<usage_digest entity id>",
"to_entity_id": "<daemon_report entity id>",
"relationship_type": "REFERS_TO"
}This is an optional but recommended convention. Omitting it does not break anything; it only removes the drill-down link in the Inspector graph view.
Free-text fields (friction_notes, notes) are redacted with defense in depth:
- Client-side (SHOULD): emitters redact PII before submitting, using a shared redaction salt per digest. This keeps raw PII off the wire entirely and lets the emitter control placeholder correlation.
- Server-side (enforced): the Neotoma store seam scans
notesand eachfriction_noteselement on ingest for anyusage_digestentity and writes back the redacted form (reusing the samescanAndRedactguard as the issues path, in redact-and-store mode — a digest is never dropped, only cleaned). This is a backstop so a guest emitter that forgets client-side redaction cannot silently persist raw PII. The guard is strictly scoped tousage_digest; no other entity type is affected.
Both layers use the same patterns and the per-digest redaction_salt (reused if the client set one, generated otherwise).
Inspect each string for patterns matching:
| Pattern | Placeholder |
|---|---|
Email address (foo@bar.com) |
<EMAIL:sha256[:8]> |
| Bearer / API token (long hex or base64 strings) | <TOKEN:sha256[:8]> |
Filesystem path (/home/user/..., C:\Users\...) |
<PATH:sha256[:8]> |
UUID (xxxxxxxx-xxxx-...) |
<UUID:sha256[:8]> |
| Phone number | <PHONE:sha256[:8]> |
The hash uses sha256(value + redaction_salt), truncated to 8 hex characters. This lets a single operator correlate the same redacted value across digests (using the same salt) without re-exposing PII.
Store the redaction_salt on the entity so the server-side record is self-describing. The salt is not a secret; its purpose is correlation, not encryption.
The following is a reference snippet for an observer building and submitting a weekly digest. Include this in your observer's reporter module.
import crypto from "node:crypto";
/**
* Build and submit a usage digest for the given period.
*
* @param neotomaClient - Neotoma MCP store client (wraps the `store` operation).
* @param channel - Reporter channel slug, e.g. "lemonbrand-observer".
* @param periodStart - ISO-8601 string, start of the period (inclusive).
* @param periodEnd - ISO-8601 string, end of the period (exclusive).
* @param metrics - Aggregated metrics for the period.
*/
export async function emitUsageDigest(
neotomaClient: { store: (payload: Record<string, unknown>) => Promise<unknown> },
channel: string,
periodStart: string,
periodEnd: string,
metrics: {
operationCounts: Record<string, unknown>;
errorRate: number;
errorCounts: Record<string, unknown>;
entityTypeUsage: Record<string, unknown>;
toolUsage: Record<string, unknown>;
frictionNotes: string[];
effectivenessSignal: "excellent" | "good" | "fair" | "poor" | "unknown";
notes?: string;
reporterAppVersion?: string;
reporterGitSha?: string;
aauthSub?: string;
}
): Promise<void> {
// Generate a per-digest redaction salt for PII correlation.
const redactionSalt = crypto.randomBytes(16).toString("hex");
// Redact PII from free-text fields before submission.
const redactedFrictionNotes = metrics.frictionNotes.map((note) => redactPii(note, redactionSalt));
const redactedNotes = metrics.notes ? redactPii(metrics.notes, redactionSalt) : undefined;
const idempotencyKey = `usage-digest-${channel}-${periodEnd}`;
await neotomaClient.store({
entity_type: "usage_digest",
idempotency_key: idempotencyKey,
schema_version: "1.0",
period_start: periodStart,
period_end: periodEnd,
reporter_channel: channel,
...(metrics.reporterAppVersion && { reporter_app_version: metrics.reporterAppVersion }),
...(metrics.reporterGitSha && { reporter_git_sha: metrics.reporterGitSha }),
...(metrics.aauthSub && { aauth_sub: metrics.aauthSub }),
operation_counts: metrics.operationCounts,
error_rate: metrics.errorRate,
error_counts: metrics.errorCounts,
entity_type_usage: metrics.entityTypeUsage,
tool_usage: metrics.toolUsage,
friction_notes: redactedFrictionNotes,
effectiveness_signal: metrics.effectivenessSignal,
...(redactedNotes && { notes: redactedNotes }),
redaction_salt: redactionSalt,
});
}
// ─── PII redaction ────────────────────────────────────────────────────────────
const PII_PATTERNS: Array<{ re: RegExp; label: string }> = [
{ re: /[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g, label: "EMAIL" },
{ re: /\b[0-9a-fA-F]{32,}\b/g, label: "TOKEN" },
{ re: /(\/[a-zA-Z0-9_.-]+){3,}/g, label: "PATH" },
{ re: /[C-Z]:\\(?:[^\\/:*?"<>|\r\n]+\\)+/g, label: "PATH" },
{
re: /[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/gi,
label: "UUID",
},
{ re: /\+?[\d\s\-().]{7,15}\d/g, label: "PHONE" },
];
function hashFragment(value: string, salt: string): string {
return crypto
.createHash("sha256")
.update(value + salt)
.digest("hex")
.slice(0, 8);
}
function redactPii(text: string, salt: string): string {
let result = text;
for (const { re, label } of PII_PATTERNS) {
result = result.replace(re, (match) => `<${label}:${hashFragment(match, salt)}>`);
}
return result;
}await emitUsageDigest(
neotomaClient,
"lemonbrand-observer",
"2026-05-25T00:00:00Z",
"2026-06-01T00:00:00Z",
{
operationCounts: { total: 4821, by_operation: { store: 3100, retrieve_entities: 980 } },
errorRate: 0.0097,
errorCounts: { total: 47, by_error_code: { ERR_STORE_VALIDATION_FAILED: 28 } },
entityTypeUsage: { by_entity_type: { task: { store_count: 156, retrieve_count: 304 } } },
toolUsage: { by_tool: { store: { call_count: 3100, error_count: 18 } } },
frictionNotes: ["retrieve_entities returned empty for entity_type=task despite known data"],
effectivenessSignal: "good",
reporterAppVersion: "1.4.2",
}
);GET /usage is a separate, complementary surface that exposes pre-aggregated dashboard metrics computed on the fly from the live database. It is not the same as querying usage_digest entities: the endpoint returns ephemeral per-request aggregates scoped to the authenticated user, while usage_digest entities are durable, periodic telemetry records submitted by external observers.
The endpoint powers the Neotoma Inspector dashboard (the summary tiles showing total entities, recent ingestion, and schema coverage). It can also be called directly by operators or agents that want a quick snapshot of their local deployment's current state without parsing raw entity records.
GET /usage
operationId: getUsage — declared in openapi.yaml and mapped in src/shared/contract_mappings.ts with adapter: "cli" (CLI-accessible via neotoma request --operation getUsage).
Authentication is required. All figures are scoped to the authenticated user.
Supported auth methods (same as all other authenticated endpoints):
| Method | How to supply |
|---|---|
| Bearer token | Authorization: Bearer <NEOTOMA_BEARER_TOKEN> |
| AAuth grant | Agent identity admitted via X-AAuth-* headers |
| MCP OAuth token | Bearer token issued through MCP OAuth flow |
Guest principals cannot call this endpoint. A missing or invalid token returns 401 AUTH_REQUIRED.
Optional query parameter: user_id (UUID string). When supplied, the authenticated user's ID must match — useful for CLI tooling that passes its local user ID explicitly. AAuth-admitted agents cannot override user_id to a different user.
200 OK — application/json — UsageStats object:
| Field | Type | Description |
|---|---|---|
entities_by_type |
Record<string, number> |
Count of active (non-merged) entities per entity_type, sorted by count descending. |
total_entities |
number |
Total active entities across all types. |
observations_by_source |
Record<string, number> |
Count of observations grouped by observation_source value, sorted by count descending. Sources not present on any observation appear as "unclassified". |
total_observations |
number |
Total observation records for this user. |
entities_created_last_7_days |
number |
Entities whose created_at timestamp falls within the last 7 days. |
entities_created_last_30_days |
number |
Entities whose created_at timestamp falls within the last 30 days. |
entity_types_with_schema |
number |
Count of distinct entity_type values present in the entities table that also have an active entry in schema_registry. |
entity_types_total |
number |
Total distinct entity_type values present across all active entities for this user. |
last_updated |
string (ISO-8601) |
ISO timestamp generated at the moment the aggregates were computed. Reflects when the response was assembled, not a database watermark. |
All figures are computed fresh on each request — there is no server-side cache. The last_updated timestamp reflects the time of the request, not a cache expiry.
Request:
GET /usage HTTP/1.1
Authorization: Bearer <NEOTOMA_BEARER_TOKEN>Response:
{
"entities_by_type": {
"conversation": 214,
"task": 156,
"contact": 88,
"harness_event": 42
},
"total_entities": 500,
"observations_by_source": {
"human": 310,
"llm_summary": 140,
"sensor": 50
},
"total_observations": 500,
"entities_created_last_7_days": 34,
"entities_created_last_30_days": 112,
"entity_types_with_schema": 6,
"entity_types_total": 10,
"last_updated": "2026-06-19T09:00:00.000Z"
}curl example:
curl -s \
-H "Authorization: Bearer $NEOTOMA_BEARER_TOKEN" \
http://localhost:3080/usage | jq .| Status | error_code |
Cause |
|---|---|---|
401 |
AUTH_REQUIRED |
Missing or invalid Bearer token; guest principal attempted. |
403 |
FORBIDDEN |
Supplied user_id query param does not match the authenticated user. |
500 |
DB_QUERY_FAILED |
Database query failed; check server logs for APIError:usage_stats. |
The Neotoma Inspector dashboard polls GET /usage on page load via React Query (queryKey: ["usage"]). The response drives the four summary tiles:
- Total entities —
total_entitieswithentity_types_totalentity types as a subtitle. - Total observations —
total_observations. - New (7 days) —
entities_created_last_7_dayswithentities_created_last_30_dayslast 30 days as a subtitle. - Schema coverage — derived from
entity_types_with_schema/entity_types_total.
The dashboard does not poll on an interval; it refetches on manual navigation or page reload. There is no server-sent staleness header.
GET /usage |
usage_digest entities |
|
|---|---|---|
| Durability | Ephemeral — computed on each request, not stored. | Durable — persisted entities queryable over time. |
| Scope | Current live state for the authenticated user. | Historical rollup for a bounded time window from an external observer. |
| Writer | Neotoma server, on each HTTP call. | External agent / observer daemon via store MCP operation. |
| Auth | Requires authenticated user (Bearer / AAuth / OAuth). | Can accept guest store writes via submit_only guest policy. |
| PII risk | No free-text fields; counts only. | Contains friction_notes / notes — redacted on ingest. |
| Primary consumer | Inspector dashboard, CLI operators. | Telemetry dashboards, cross-deployment trend analysis. |
The full schema for UsageStats and the getUsage operation are declared in openapi.yaml. The generated TypeScript types live in src/shared/openapi_types.ts under components["schemas"]["UsageStats"] and operations["getUsage"].
daemon_report— per-incident anomaly reports; the drill-down companion to usage digests.harness_event— per-turn/per-operation event records; the raw data that usage digests aggregate.guest_access_policy.md— configuring keyless write access for telemetry sinks.aauth.md— issuing agent grants for authenticated digest submission.subscriptions.md— subscribing toUsageDigestClosedtimeline events for downstream alerting.docs/api/rest_api.md— top-level REST API reference including the/usageendpoint entry.