[Product Gap] Implement evidence-based email, media, and document resolution pipeline

[seonghobae]

Buyer problem

Naruon has broad workspace surfaces, but the buyer-visible trust gap remains: users must know why a mail/thread/document/image was connected, what source evidence supports an AI judgment, what confidence/boundary applies, and what action a control will perform before provider/workspace state changes.

This issue is the umbrella implementation contract for the evidence-based workspace direction introduced by #1349. None of the active slices below is shipped until it reaches protected develop; PR-head evidence must not be described as protected-branch product truth.

Current implementation truth — revalidated 2026-08-16

Protected develop was observed at bc98789521d21271e84789888413c182aa111b4d. Current bounded lanes include:

• Attachment admission — feat(attachments): recognize HWP and HWPX parser boundaries #1353 (8beb01c260497100c75cf020d3ebc3b11b438f2f): deterministic HWP/HWPX family recognition, retained-byte provenance, bounded HWPX package metadata, and two-signal HWP admission.
• HWPX ordered text recognition — draft feat(hwpx): recognize ordered section text with provenance #1373, observed stacked exactly on feat(attachments): recognize HWP and HWPX parser boundaries #1353 (8beb01... -> b66d9334677a4184c3793ee66ce20eb7696a97c2): deterministic local HWPX section/spine text extraction with source hashing and content-graph provenance. It remains dependency-sensitive until its parent is integrated/retargeted; stacked/predecessor evidence is non-passing.
• Async action-state accessibility — fix(a11y): expose async button busy states #1352 (537bcdbdff77741cd76bb118db283eb1ea7b856e): separates a shared document-action lock from the initiating action's aria-busy state.
• Reusable design-token review bridge — feat(ui): add Storybook design-token contract #1354 (3cc9716232a53dca07b94f610e14816070f1f842): tested Storybook-readable token adapter with explicit candidate-vs-production authority; no Storybook runtime dependency is added in that slice.
• Email source/dedupe provenance — feat(email): deterministic dedupe provenance — gate strong fingerprints on genuine Date (naruon#1086) #1195 remains a separate large implementation lane for genuine Date/Message-ID provenance and deterministic review disposition; it is not considered shipped here.

All PR heads/base tips/checks/reviews/threads/workflows above are observations, not durable passing evidence; refetch them before action.

Live merge-contract correction — 2026-08-17

Repository ruleset 15586698 by itself requires zero approving reviews, but the effective default-branch contract is the union of all active applicable rulesets. Repository ruleset 17214772 (Lock default branch) and inherited organization ruleset 18156473 (CWL Central required workflows) are also active and each requires one approving review, dismisses stale reviews on push, requires approval after the last push, and requires review-thread resolution. No active ruleset currently requires CODEOWNER review. The effective live contract therefore still requires a qualifying independent post-last-push approval, exact-head status checks, and required workflows. Issue #1371 remains open because the current collaborator inventory exposes only seonghobae, so the independent human approval route is not presently verifiable from this repository writer's authority. This does not authorize merging over a current valid formal CHANGES_REQUESTED finding or any failed/pending/absent/stale required evidence; each finding must still be verified against source and each live gate must pass on the unchanged head. Refetch every active ruleset and exact-head review/check/workflow state before action.

Required implementation slices

Slice 1 — canonical email identity

• Add or validate email_raw, email_message, email_instance, and duplicate-review state.
• Use RFC headers, raw MIME hash, body similarity/hash, embeddings, attachment-manifest similarity, participant overlap, and sent-time delta as resolver features without treating collection time as source evidence.
• Distinguish same_message, review_required, related_message, and distinct_message.
• Test forwarded copies, missing/reused Message-ID across tenants, provider duplicate instances, and genuine-vs-synthetic date provenance.

Slice 2 — thread graph resolver

• Store reply/provider/quote/participant/semantic/manual edges with evidence and tenant/context isolation.
• Return thread confidence and evidence rather than an unexplained thread identifier.
• Support merge/split/manual-override UX with auditability.
• Test multi-parent, forwarded, Korean subject-prefix, equal-date ordering, and provider-thread mismatch cases.

Slice 3 — media and image pipeline

• Resolve cid: images through MIME multipart/related before OCR/vision.
• Classify tracking pixels, logos/signatures, screenshots, charts, document scans, table images, and unsupported media.
• Normalize only supported images into bounded LLM-safe artifacts; remote images remain no-fetch unless a separately authorized egress policy permits them.
• Preserve source/member position and hash provenance for base64/CID/repeated images.

Slice 4 — HWP/HWPX documents

• Admit HWPX/HWP only after bounded deterministic family validation (feat(attachments): recognize HWP and HWPX parser boundaries #1353 lane).
• Parse HWPX XML package sections in manifest/spine order with source provenance (feat(hwpx): recognize ordered section text with provenance #1373 lane) before adding table/image/layout reconstruction.
• Treat HWP as conversion-first and sandboxed; the cheap FileHeader marker is not full CFB trust.
• Track text/table/image/layout/provenance confidence separately and expose partial/quarantine states rather than synthetic success.

Slice 5 — AI synthesis and UX action contract

• Bind every AI judgment/action/calendar candidate to authorized source artifacts and evidence IDs.
• Store prompt/model/artifact versions or privacy-safe cryptographic provenance where raw prompts/results must not be retained.
• Require confirmation/authorization for send, calendar writeback, external share, policy change, and destructive provider/file action.
• Update UI action states and audit events according to docs/ui-ux/button-action-contract.md; shared disabling must not falsely mark unrelated actions busy.
• Keep deterministic safety/authorization gates independent of model judgment.

Standards and doctoring

Use and keep code-current:

• docs/planning/naruon-evidence-workspace-task-definition.md
• docs/architecture/evidence-pipeline-architecture.md
• docs/ui-ux/button-action-contract.md
• docs/doctoring/evidence-workspace-technical-sources.md

Doctoring must preserve primary/authoritative references and APA 7 citations for applicable RFC mail/MIME standards, record-linkage research, current HWP/HWPX/OWPML standards/docs, supported image/vision boundaries, accessibility requirements, and any material security/privacy standard used by code.

Quality gates

• Database objects use descriptive two-or-more-word snake_case names and persistent relational state remains 3NF unless a documented bounded exception is justified.
• Tests use realistic mail/document/image examples, including malicious/ambiguous package structures and source-provenance edge cases.
• Hosted-model tests use the approved NVIDIA NIM path; COPILOT_GITHUB_TOKEN is not a development/test credential.
• Do not blanket-mask business PII; enforce purpose-bound authorization, tenant/context isolation, least privilege, retention, encryption, access/export logging, and auditable external-sharing contracts.
• Current exact-head CI/security/coverage/dependency/container/package/provenance evidence and review-thread resolution are mandatory; predecessor/stacked/pending/skipped/neutral/model-only evidence is non-passing.
• Protected integration must satisfy the live ruleset, required workflows, and any still-valid formal review finding on the unchanged head; never substitute a partial ruleset snapshot or model-only status for current evidence.

Done

Close only when the protected product lets a buyer open a relevant Naruon screen and determine, from real persisted/derived evidence:

• what source evidence backs a result and which source bytes/artifacts it came from;
• how confident/bounded Naruon is and what remains unknown/partial;
• what action each control performs and whether it changes provider/workspace state;
• how authorization, confirmation, and audit evidence apply to that action; and
• how to correct a wrong duplicate, thread, document, image, or AI judgment without losing provenance.

[seonghobae]

Governance correction after a later same-day ruleset change: inherited organization ruleset 18156473 was updated at 2026-08-16T23:45:11.508+09:00 and now again requires one approving review, stale-review dismissal, approval after the last push, and review-thread resolution, alongside required central workflows. This supersedes the earlier zero-approval snapshot in the issue body. Repository ruleset 15586698 remains zero-approval but the organization rule is additive. #1371 is reopened for the independent-human approval path. Exact-head CI/security/workflow evidence remains mandatory and model-only evidence does not substitute for the required independent human review.

[seonghobae]

Correction to the current merge-contract note: repository ruleset 15586698 does require zero approvals, but it is not the complete protected-branch contract. Fresh 2026-08-17 ruleset enumeration shows inherited organization ruleset 18156473 also applies to the default branch and requires one approving review, dismissal on push, approval after the last push, and resolved review threads, in addition to the central required workflows. With only seonghobae in the current collaborator inventory and all CODEOWNERS paths assigned to @seonghobae, the independent approval route is still unsatisfied; #1371 remains open. Treat the paragraph saying #1371 was closed / independent approval was superseded as stale evidence until the issue body is reconciled.

[seonghobae]

Orgmetra integration canary found a current Naruon calendar-writeback contract defect at exact naruon develop SHA ddd05c5aaf3e170aa2bdc4412647b43b95d5a6b9 (backend/api/calendar.py). This is Naruon-owned; Orgmetra will not patch or work around Naruon source.

Exact reproduction / first causal boundary

1. Call POST /api/calendar/writeback-intent with an authorized customer-owned CalDAV source, action="create", and execute_provider=true.
2. create_writeback_intent() sets requires_if_match = request.action == "update", therefore a create intent correctly gets requires_if_match=false and if_match=None.
3. The same function then unconditionally rejects provider execution when intent.if_match is None, returning HTTP 409 before runner_manager.dispatch_command(...).
4. Result: provider-executed create is impossible even though WritebackIntentRequest.action explicitly admits "create" and the runner command supports write_caldav.

Smallest owner-side remedy
Require If-Match only for update execution. Preserve the existing fail-closed ETag requirement for action="update"; do not weaken source authorization, writeback capability checks, customer-owned protocol checks, or runner dispatch controls.

RED acceptance tests

• authorized create + execute_provider=true dispatches exactly one write_caldav command with no If-Match precondition and returns the typed dispatch result;
• create + execute_provider=false remains intent-only and performs no provider write;
• update + execute_provider=true without a source ETag remains HTTP 409;
• update + execute_provider=true with a source ETag dispatches with that exact If-Match;
• unauthorized/cross-organization source selection remains 403 and no dispatch occurs.

GREEN proof required before downstream reliance
Focused calendar API tests plus the normal Naruon exact-head CI/security/coverage suite on the repaired head, followed by protected-branch integration evidence. Orgmetra will consume only the published API contract and will keep provider execution fail-closed until this owner-side behavior is proven.