Skip to content

capture-format.md

Latest commit

 

History

History
477 lines (396 loc) · 23.3 KB

capture-format.md

File metadata and controls

477 lines (396 loc) · 23.3 KB
Raw

Capture Markdown format

This is the authoritative spec for the Markdown files Transcripted saves under the capture library (<capture-library>/meetings/, <capture-library>/dictations/, and <capture-library>/writing/). Agents and standalone tools parse these files, so treat this document as the contract. Writers live in Sources/TranscriptedCore/Storage/TranscriptFormatter.swift, Sources/Meeting/MeetingTranscriptStyler.swift, Sources/Dictation/DictationTranscriptWriter.swift, and the main app's Writing day-file writer (never the keyboard). Readers include the app's Home scanner (TranscriptFrontmatter), Tools/TranscriptedCaptureKit (used by the CLI and MCP tools), and Tools/TranscriptedQA. If you change the written format, update this doc, the CaptureKit parsers, and their tests in the same change.

Versioning

Two flat frontmatter keys carry the contract:

  • format_version: 1 — the capture-format version. Absent means the file predates versioning; parse it exactly like version 1. The grammar below has not changed since before the key existed — the key was added so future breaking changes have somewhere to land. Bump the number only for a breaking change; additive changes stay within the current version.
  • transcript_style: raw | styled — which meeting body grammar the file currently carries (meetings only; see the lifecycle below). Absent means the file predates the key: sniff the body heading instead (## Full Transcript → raw, ## Transcript → styled).

Dictation and writing day files carry format_version but no transcript_style (they have a single body grammar).

Stability rules

  1. Flat key: value frontmatter only. The app's own scanner (TranscriptFrontmatter.values(from:)) skips indented lines, so nested YAML is invisible to it (see the Issue #500 comment in TranscriptFormatter.swift). New keys must be top-level. The only nested blocks are the pre-existing gap_events, speakers, tags, aliases, and cssclasses; parsers that need them re-read the raw frontmatter lines.
  2. Parsers must ignore unknown keys. Older files may contain summary metadata from features that no longer write new fields. Future app versions may add more keys too.
  3. Additive changes only within a version. Never rename or remove a key, or change an existing key's value format, without bumping format_version.
  4. Frontmatter values may be quoted or bare; parsers strip surrounding double quotes. Escaping for quoted scalars follows TranscriptSaver.escapeYAML.
  5. Key order is not guaranteed. Match by key, not position.

Meeting file lifecycle

A meeting transcript is written once and then rewritten in place up to two times. Any stage after the first can be skipped (the restyle fails closed on bodies it cannot parse; the app can also quit first), so all intermediate shapes exist in the wild:

  1. Initial save — TranscriptFormatter.formatTranscriptMarkdown writes the raw form to Call_<YYYY-MM-dd_HH-mm-ss>.md (collision suffix _<n>), with format_version: 1 and transcript_style: raw.
  2. Restyle + rename (async) — MeetingTranscriptStyler.restyleTranscript rewrites the body into the styled form, re-emits title: first, preserves every other frontmatter line (including format_version), rewrites transcript_style to styled (replacing the raw marker — never duplicated; legacy files without the key gain it), and renames the file to <YYYY-MM-dd> <title>.md (collision suffix <n>). Retained audio directories follow the rename (MeetingArtifactRenamer). Unknown trailing Markdown sections from older files are preserved.

The Home title editor may later rewrite title: and rename the file again. It also preserves unknown frontmatter and trailing sections.

Meeting frontmatter keys

Written at initial save (all flat unless noted):

Key Example Notes
capture_id "5E9A…" UUID, quoted. Same value as transcript_id.
capture_type meeting Discriminator; dictation day files use dictation_day, writing day files writing_day.
format_version 1 See Versioning. Absent = pre-versioning.
transcript_style raw / styled See Versioning and lifecycle.
transcript_id "5E9A…" UUID, quoted.
date 2026-04-07 yyyy-MM-dd, local time.
time 09:14:00 HH:mm:ss, local time.
duration "12:30" m:ss (minutes may exceed 59), quoted.
processing_time "41.3s" Seconds with one decimal + s, quoted.
transcription_engine parakeet_local Engine identifier.
transcription_language "auto" / "fi" Optional immutable requested language for this job. Absent legacy files retry as Automatic.
transcription_language_resolution detected Optional explicit, detected, automaticUncertain, multilingual, or unsupported. Detection is bounded evidence, not proof of a recording's only language.
transcription_language_resolved "fi" Optional language passed to the recognizer; absent when Automatic remains uncertain, multilingual, or unsupported by that engine. Exception: Apple Speech (apple_speech_local) can't detect languages, so its Automatic uses the Mac's language and writes automaticUncertain together with that code.
diarization_engine nemotron_offline The diarizer that actually ran for this meeting. See below.
voiceprint_model "redimnet2-b4" Optional, quoted. The model that embedded each speaker's voiceprint. Absent in files saved before 2026-09-29 and when no diarizer ran.
sources [mic, system_audio] Inline list of captured channels.
mic_utterances 12
system_utterances 34
mic_speakers 1
system_speakers 2
total_word_count 1204
title "Weekly Sync" Optional at save (imported audio, detected meetings); the restyle always writes one.
imported_at 2026-09-24T19:34:04Z Imported files only: when the file was transcribed, as a UTC ISO 8601 instant. date/time stay the original recording time; the app lists an import by this key so it shows up where it was just made. Kept when the same file is re-transcribed.

The transcription_engine value identifies the concrete model used, not a later picker selection. Current identifiers are additive within format version 1:

Identifier Model Raw transcript footer name
parakeet_local Parakeet TDT v3 (default) Parakeet (preserved for compatibility)
parakeet_v2_local Parakeet TDT v2 (English only) Parakeet V2
parakeet_ultra_local Parakeet Ultra (experimental, installed by scripts/models/parakeet-ultra) Parakeet Ultra
whisper_large_v3_turbo_local Whisper Large v3 Turbo Whisper Large V3 Turbo
whisper_large_v3_local Whisper Large v3 Whisper Large V3
apple_speech_local Apple Speech (macOS SpeechTranscriber) Apple Speech

Readers should preserve unknown identifiers rather than assuming every local capture uses Parakeet v3.

diarization_engine records the backend that diarized the meeting, read when the diarizer ran, not the one the app asked for. Values are additive within format version 1:

Value Meaning
nemotron_offline NVIDIA Nemotron 3 Diarization (the app default since 2026-09-28).
pyannote_offline pyannote community-1. Also what a Nemotron load failure falls back to.
none No diarizer ran (for example a mic-only meeting without local speaker split).

Files saved before 2026-09-29 always say pyannote_offline, even when Nemotron diarized them, so treat that value in older files as "unknown backend". voiceprint_model uses the embedder's identifier: redimnet2-b4 (default), eres2net, wespeaker (pyannote's native offline WeSpeaker), or wespeaker-fluid-online. Missing means unknown; readers keep unknown values. The raw footer names the diarizer too (Parakeet + Nemotron (local), or just Parakeet (local) when none ran); like the STT name there, it's descriptive. The footer is descriptive; use the frontmatter key for model identity.

The selected language survives saved-audio retranscription and failed-job retry. Automatic detection is resolved once per pipeline invocation from at most three non-overlapping, speech-candidate windows of up to ten seconds spread across the recording. Its result is shared by both channels and all speaker segments. A retry can resolve Automatic again using its current engine; the previous resolved language is descriptive metadata, not a permanent forced override.

Recording-health keys (optional, only when health info exists):

Key Example Notes
capture_quality excellent excellent / good / fair / degraded.
audio_gaps 0
device_switches 0
system_audio_signal_verified true Optional flat boolean: finite nonzero system PCM was observed during this live recording. false means unverified, not permission denied; ordinary silence after observed signal remains true. Omitted for imports and legacy captures, for "Record Just My Mic", and when no system audio was heard but macOS confirmed System Audio Recording access (a quiet Mac). Independent of capture_quality.
gap_events — Nested list of quoted strings; invisible to flat parsers. Omitted when empty.
audio_health mic_attenuated_by_call_app Flat; omitted for healthy meetings.
mic_boost_prompt "declined" Flat; only alongside audio_health.
microphone_audio_unusable true Flat; microphone audio was missing or unusable and excluded from a system-audio-only partial transcript.
system_audio_missing true Flat; no system track survived to save, so the transcript holds only the mic side and capture_quality is degraded.
mic_only true Flat; the user picked "Record Just My Mic", so only their side was recorded on purpose and the other side of any call is not in the transcript. Not a capture problem: the grade is unaffected and system_audio_missing is never set with it, and sources lists only the mic. A retry or Home re-transcribe keeps the key but writes no capture_quality/audio_gaps/device_switches, since nothing measured the capture. Omitted for two-sided meetings.

Speaker metadata (optional, nested speakers: block — flat parsers skip it; CaptureKit and the app re-read the raw lines):

speakers:
  - id: "0"
    channel: mic          # mic | system; older files may omit (treat as system)
    db_id: "1111…"        # optional persistent speaker DB UUID
    name: "Justin"
    confidence: high      # high | medium | manual | unknown
    source: db_scan

Obsidian metadata (legacy, read-only): older transcripts written while the retired enableObsidianFormat flag existed may carry nested tags:, aliases:, and cssclasses: blocks, with named body speakers wrapped in [[wiki links]]. New transcripts never include these, but parsers and speaker-rename rewrites still tolerate and preserve them.

Older files may contain auto_summary_*, local_summary_*, or other summary frontmatter and trailing sections. These are legacy data: current meeting writers leave them untouched, and current Home surfaces do not create new AI summary content.

Meeting body: raw form (transcript_style: raw)

Written at initial save. Filename Call_<YYYY-MM-dd_HH-mm-ss>.md.

# Meeting Recording - Apr 7, 2026 at 9:14 AM

**Duration:** 12:30 | **Words:** 42 | **Utterances:** 2

---

---

## Channel & Speaker Analytics

### Microphone (You)
- **Utterances:** 1
- **Words:** ~20
- **Speaking Time:** 22s

### Meeting Audio (Remote Participants)
- **Utterances:** 1
- **Words:** ~22
- **Speaking Time:** 31s
- **Speakers Detected:** 1

#### Remote Speaker Breakdown

- **Sarah:** 1 utterances, ~22 words, 31s

---

## Full Transcript

[00:00] [Mic/You] Thanks for making time today.

[00:04] [System/Sarah] Happy to help. Let's get started.

---

*Generated by Transcripted with Parakeet + PyAnnote (local) | Duration: 12:30 | 42 words | 2 speakers*

Transcript grammar: one entry per line under ## Full Transcript, blank line between entries:

[<MM:SS or HH:MM:SS>] [<Mic|System>/<speaker label>] <text>

Elapsed timestamps use MM:SS below one hour and HH:MM:SS from one hour onward, truncating fractional seconds: 59:59, 01:00:00, 100:00:00. Hours have at least two digits and do not wrap at 24. Readers also accept older total-minute timestamps such as 60:00 and 6000:00, and unpadded hours such as 1:00:00. Speaker renaming preserves the original timestamp spelling. These already-supported clock forms remain within format version 1; duration metadata and speaking-time summaries keep their existing formats.

Mic labels default to You (or a named speaker with local-speaker review); system labels default to Speaker <n> until named. In legacy Obsidian-formatted files, named labels are wiki-linked: [System/[[Sarah]]] (read-side tolerance only; new files never write this). Multi-speaker mic captures add a ### Microphone (People in the Room) heading and a #### Local Speaker Breakdown section. When mic capture is disabled the microphone section is omitted and sources reflects it.

Meeting body: styled form (transcript_style: styled)

Produced by the async restyle. Filename <YYYY-MM-dd> <title>.md.

---
title: "Meeting with Sarah"
capture_id: "5E9A…"
capture_type: meeting
format_version: 1
transcript_id: "5E9A…"
date: 2026-04-07
time: 09:14:00
…all other preserved keys…
transcript_style: styled
---

# Meeting with Sarah

Recorded Apr 7, 2026 at 9:14 AM  •  12 min, 30 sec  •  42 words  •  2 turns

## Transcript

**00:00**  [Mic/You]
Thanks for making time today.

**00:04**  [System/Sarah]
Happy to help. Let's get started.

Details:

  • The detail line joins parts with • (two spaces around a bullet): Recorded <medium date, short time>, humanized duration, word count and turn count when nonzero.
  • Transcript entries are blank-line-separated blocks: a header line **<MM:SS or HH:MM:SS>** [<Mic|System>/<label>] (two spaces after the bold timestamp, with the same elapsed-time and legacy-read rules as raw transcripts) followed by the utterance text on the next line(s).
  • An empty transcript renders as _No transcript captured._.
  • Unknown trailing Markdown sections, including old generated summary blocks, are preserved after the transcript.
  • The restyle fails closed: if the body carries transcript text the entry parser cannot understand, the file is left byte-for-byte untouched (still raw-form, still transcript_style: raw if it had it).

Dictation day files

Sources/Dictation/DictationTranscriptWriter.swift appends completed dictations into one file per local day: <capture-library>/dictations/Dictations_<YYYY-MM-dd>.md.

---
title: "Dictations for April 7, 2026"
date: 2026-04-07
capture_type: dictation_day
format_version: 1
---

# Dictations for April 7, 2026

## 9:15 AM - first note from the morning

Entry ID: `dictation-20260407-091500-123-8c…`
Captured: 2026-04-07T09:15:00.000Z
Source app: Slack
Bundle ID: `com.tinyspeck.slackmacgap`
Delivery: pasted
Words: 5
Characters: 27

first note from the morning

## 4:45 PM - second note from the afternoon

…

Details:

  • The day header is written once, when the file is created; later saves append sections only. Header keys are flat; format_version follows the same convention as meetings.
  • Two edits rewrite a day file in place: Delete removes one section, and Transcribe again replaces one section's heading title, Words:, Characters:, and body. Entry ID:, Captured:, the source app, Delivery:, and Audio: never change, so readers can key on the Entry ID.
  • Section heading: ## <h:mm a> - <title> where the title is the first ~7 words of the text (or Dictation <MMM d> at <h:mm a> for very short text).
  • Metadata lines, in order: Entry ID: (backticked), Captured: (ISO 8601 with fractional seconds), Source app:, optional Bundle ID: (backticked, omitted when unknown), Delivery:, Words:, Characters:, optional Audio: (backticked). Older files may carry Timestamp: instead of Captured:.
  • Audio: is present only when the take's audio was kept (Settings → Storage → Keep dictation audio). Its value is a path relative to the dictations folder, audio/<session-uuid>.m4a. The file may be the .wav sibling while compression is pending, and it may be gone entirely once it ages out of the keep window or the user switched to Don't keep: readers must treat a missing file as normal and never resolve the path outside dictations/audio/. Additive within format_version: 1; older readers skip the line.
  • Delivery values: pasted, copied, failed, saved_without_paste.
  • The dictated text follows after a blank line and runs to the next ## heading or end of file.

Writing day files

The main Transcripted app (never the keyboard) appends settled typed text into one file per local day: <capture-library>/writing/Writing_<YYYY-MM-dd>.md, in a writing/ folder beside meetings/ and dictations/. The shape extends dictation day files and follows the same stability rules.

---
title: "Writing for September 25, 2026"
date: 2026-09-25
capture_type: writing_day
format_version: 1
---

# Writing for September 25, 2026

## 10:42 AM - Pushing the launch to Thursday so QA

Entry ID: `writing-20260925-104211-387-4f2a9c1e`
Captured: 2026-09-25T15:42:11.387Z
Source app: Slack
Bundle ID: `com.tinyspeck.slackmacgap`
Words: 12
Characters: 65
Accepted words: 3

Pushing the launch to Thursday so QA can finish the AirPods pass.

## 11:05 AM - …

Details:

  • The day header is written once, when the file is created; later saves append sections only, same as dictations. Header keys are flat.
  • Section heading: ## <h:mm a> - <title> where the title is the first ~7 words of the text (or Writing <MMM d> at <h:mm a> for very short text).
  • Metadata lines, in order:
    • Entry ID: (backticked, writing-<yyyyMMdd>-<HHmmss>-<SSS>-<8 hex>)
    • Captured: (ISO 8601 UTC with milliseconds; the entry's first keystroke)
    • Source app: (display name)
    • optional Bundle ID: (backticked, omitted when unknown)
    • Words:
    • Characters:
    • Accepted words: (words that came from accepted suggestions)
  • There is no Delivery: line; nothing is pasted.
  • The text follows after one blank line and runs to the next ## heading or end of file. It is the settled text: characters removed with plain Backspace inside the keyboard's own buffer aren't saved. Option- or Cmd-Backspace breaks the segment instead, so the word it deleted stays in the saved text. Pastes, mouse edits, and the host app's autocorrect aren't seen (a known v1 limit).
  • A text line that starts with ## is written as \## so it can't open a new section. Readers strip that one backslash (CaptureMarkdownParser.unescapeWritingBody). So a line that literally starts with \## doesn't round-trip: it reads back as ## . Whole-day reads (MCP read_writing without entry_id when it returns the file unpaged, and read-writing without --entry-id) return the raw Markdown with the escape still in it. Entry reads, pages, and parsed entries are unescaped.
  • An entry is one app's continuous writing. A new entry starts on an app switch, after 2 minutes idle, or on a caret jump or segment break the keyboard reports. A caret the host reports late, or never moves (some terminals report it at 0), isn't a jump: a word typed in one go is saved as that word. A scrap under 3 words (a quick "sounds good") doesn't end at a segment break when the same app's next segment starts within a minute: it folds into that entry, each segment on its own line. Box fragments (one character, or up to 5 characters with no letters, like a card or code box) don't count toward the 3 words, so a card or code typed over several boxes stays one entry. Entries under 2 characters after trimming aren't saved.
  • Nothing is written for secure input, excluded apps (password managers always), apps outside the user's scope, or while Save my writing is off.
  • Secrets are scrubbed before an entry is written: a password typed at a terminal prompt (sudo, ssh, passwd, read -s, …), a line that is only a password-shaped token, one-time codes and PINs, card numbers (also split across boxes, with the expiry and CVV after them), values after labels like password:, API_KEY=, --token or Bearer, credentials in URLs, and known token formats. Each becomes ⟨redacted:<kind>⟩, where kind is password, code, card, secret, api-key, jwt, pem, iban or ssn. Emails and phone numbers are kept. An entry that was nothing but a secret is saved as just its token (a sudo password becomes a ⟨redacted:password⟩ section), so the command before it never runs into the line after it on disk. Words: and Characters: count the scrubbed text, and the heading title comes from it.
  • Day files written before the scrubber (or under older rules) are rescrubbed once when Writing starts: changed sections get their title and counts redone (Accepted words: never more than Words:), sections that were only a secret keep their heading with just the ⟨redacted:…⟩ token as their text (so the command before them never runs into the line after, and a second pass changes nothing), and files with nothing to scrub aren't touched. A file that changes while it's being rescrubbed, or that Writing stopped before reaching, is left for the next launch. One that can't be read as a day file at all (a symlink, not UTF-8) is skipped and doesn't hold the version back. The rules version it ran with is remembered per Mac (WritingDayFilesScrubbedRulesVersion), so day files that arrive later from another Mac aren't rescrubbed until the next rules bump, and nothing runs while Writing is off.
  • Files are created 0600 and the writing/ folder 0700.

Parser guidance

  • Detect meetings via capture_type: meeting; fall back to the transcript headings for pre-capture_type files. Detect dictation day files via capture_type: dictation_day or the Dictations_ filename prefix.
  • Detect writing day files via capture_type: writing_day or the Writing_ filename prefix. Check for them before falling back to "any other Markdown file with frontmatter is a meeting", or they get read as meetings.
  • Choose the meeting body grammar by transcript_style when present, else by heading: ## Full Transcript (raw) vs ## Transcript (styled). Robust parsers accept both grammars in either file (see Tools/TranscriptedCaptureKit/Sources/TranscriptedCaptureKit/CaptureMarkdownParser.swift).
  • Treat every key as optional. Missing format_version = version 1 semantics.
  • Tools/TranscriptedCaptureKit exposes format_version / transcript_style as optional formatVersion / transcriptStyle fields on ParsedMeetingCapture and formatVersion on ParsedDictationDayCapture and ParsedWritingDayCapture (CaptureMarkdownParser.parseWritingDay). A dictation entry's Audio: line is audioRelativePath on ParsedDictationDayCapture.Entry (nil when absent). Detect writing day files via capture_type: writing_day or the Writing_ prefix, and check for them before the "frontmatter means meeting" fallback.