Repository navigation
feat(observability): add SQLite + JSONL observability foundation - #38
Merged
Merged
Conversation
Adds src/db/observability.py — a lightweight, local-first observability
+ notes foundation for IntelliFile. Satisfies the five required
behaviors (system self-profile, structured run logging, artifact
registry, manual notes, improvement queue) plus privacy-aware defaults.
Components:
- ObservabilityStore: SQLite-backed store with thread-safe writes
and JSONL mirrors (runs.jsonl, notes.jsonl) for tailing.
- RunContext: context manager that auto-finishes runs (success on
clean exit, failed with error_summary on exception).
- redact(): regex-based redaction of sensitive field names (token,
password, api_key, api_id, api_hash, session, auth, credential,
private_key) — case-insensitive. Bypassed only when debug=True.
- Repeated failures and warn-or-worse notes auto-bump the
improvement_queue table (matched by source+title to keep the queue
small).
Storage layout:
$XDG_DATA_HOME/intellifile/observability/
observability.db (source of truth)
runs.jsonl (append-only mirror)
notes.jsonl (append-only mirror)
12 unit tests in tests/test_observability.py cover all five behaviors
+ privacy defaults + JSONL mirror. All tests pass.
Also adds docs/OBSERVABILITY.md explaining the API, what it is NOT
(no telemetry, no remote upload, no full issue tracker), and how Omni
could adopt the same shape in a separate PR.
UI surfacing (Activity / Notes / Diagnostics panels) intentionally
deferred to a separate PR per the user's instruction: 'If UI exposure
is too large for one PR, first implement the backend/storage structure
and minimal surfacing, then iterate.'
Reference: ECOSYSTEM_STATE.md in repo-sync-toolkit (PR #1).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
src/db/observability.py(689 lines) —ObservabilityStore+RunContext+redact()helper. SQLite-backed, thread-safe writes, JSONL mirrors for tailing.tests/test_observability.py(185 lines) — 12 unit tests covering all five behaviors + privacy defaults + JSONL mirror.docs/OBSERVABILITY.md(137 lines) — API reference, what it is NOT (no telemetry, no remote upload, no full issue tracker), how Omni can adopt the same shape later.Why it matters
Per ecosystem policy (Phase 3), the active product repos must have a practical local observability + notes foundation. This PR implements it for IntelliFile (Omni can adopt the same pattern in a follow-up PR).
Satisfies the five required behaviors:
Plus privacy-aware defaults:
<REDACTED>.debug=True.Reference:
ECOSYSTEM_STATE.mdinrepo-sync-toolkit(PR #1).Risks / intentionally not changed
src/core/smart_tagger.py(medical scope tracked separately).mobile/(quarantined in PR chore(deps): bump actions/checkout from 4 to 7 #3).Verification performed
python -m pytest tests/test_observability.py -v— 12/12 PASS.api_key,api_id,api_hash,session,password,authall redacted in stored input_summary; non-sensitive keys preserved; debug=True bypasses.runs.jsonlappended correctly, last line parses as JSON.Blockers / manual follow-up
None. Future PRs should:
RunContextinto existing components (search, tagger, extractor, etc.) one component at a time.omni-medical-suite/src/observability/in a separate PR.Next PR
PR #7 — repo-sync-toolkit-safety-hardening.