Recommendation and Choice Engine

Author: Saurabh Singh

CLAUDE.md

@@ -43,7 +43,7 @@ ruff check inkwell/
 ruff format inkwell/
 ```
 
-Ruff config: line length 100, target `py311` (see [pyproject.toml](pyproject.toml)).
+Ruff config: line length 100, target `py311` (see [pyproject.toml](pyproject.toml)). Tests use `pytest-asyncio` in `auto` mode (`asyncio_mode = "auto"`), so async route tests in [tests/test_routes/](tests/test_routes/) need no `@pytest.mark.asyncio` decorator; they drive the FastAPI app via `httpx`.
 
 ## Architecture
 
@@ -60,15 +60,18 @@ Key points future instances must know:
 - **`config.py` is the single source of truth for settings.** It loads `.env` (secrets, `LLM_MODEL`, etc.), exposes path constants (`ROOT_DIR`, `CONFIG_DIR`, `DATA_DIR`), and provides `load_subreddits / load_personality / load_filters`. Each loader has a **backward-compat fallback**: if a YAML file is missing from `config/`, it looks in `ROOT_DIR`. Preserve this when changing loaders.
 - **Scanners, exporters, and filters are Protocol-based** (structural typing, not inheritance). A scanner is anything with `name: str` and `scan(targets, max_age_hours) → list[RawSignal]`. An exporter has `name` + `export(rows, config)`. Add new platforms/outputs by creating a class that matches the protocol — no base class required.
 - **Scanners self-register via a lazy registry.** New scanners must (1) call `registry.register(MyScanner())` at module level, and (2) be imported inside [registry.py](inkwell/scanners/registry.py)'s `_ensure_loaded()` so they load on first `get_scanner()` call.
+- **Ratings feed back into scoring (the learned-preference loop).** The `/rate` endpoint writes `{signal_id, rating}` to `data/feedback/ratings.json`. [storage/feedback.py](inkwell/storage/feedback.py)'s `compute_preference_weights()` joins those ratings back to stored signals and produces mean-centered, clamped per-subreddit and per-keyword weights. The scan loop computes this **once per run** and threads it through `analyze_signal(..., weights=...)` → [analyzers/rules.py](inkwell/analyzers/rules.py)'s `_learned_bonus`, which adds a `learned_bonus` term (±`LEARNED_BONUS_SCALE`, default 2.0) to `score_breakdown`. It's a free nudge — no LLM. Cold start is a no-op: below `MIN_RATINGS_TO_LEARN` (5) usable ratings, weights are empty and scoring is unchanged. `score_breakdown`/`engage`/`analyze_rules` all take an optional `weights` arg so the scorer stays pure and testable.
 - **Filtering happens before scoring.** [filters/rule_filter.py](inkwell/filters/rule_filter.py) applies keywords/score/flair/post-type rules first; [analyzers/rules.py](inkwell/analyzers/rules.py) then scores what survives. `ai_preferences.prefer_topics` is a soft boost to the engage score; `ai_preferences.avoid_topics` is a hard "No". Neither spends tokens.
 - **The analyzer is split: rules (free) + voice (BYOK, on demand).** [analyzers/rules.py](inkwell/analyzers/rules.py) is pure heuristics and produces `summary`, `coolest_comment`, `engage` (Yes/Maybe/No), and `why`. [analyzers/pipeline.py](inkwell/analyzers/pipeline.py) is a thin wrapper calling `analyze_rules` — **the LLM is never invoked during scan**. [analyzers/voice.py](inkwell/analyzers/voice.py) is the only code path that spends tokens; it takes `model` and `api_key` as arguments (not env) so the web UI can BYOK from browser `localStorage` via the `X-LLM-Key` header, and the server never persists the key. Native JSON mode is still OpenAI-only (see `_supports_native_json_mode` in [llm_client.py](inkwell/analyzers/llm_client.py)). The old single-call pipeline (summary + engage + voice in one LLM request) was removed when scans became free.
 - **Voice drafts are NOT produced at scan time.** CSV/Sheets exports show em-dashes for `Suggested reply to cool comment` and `Suggested post comment`. Users generate drafts on demand via the UI's *Draft* button (`POST /api/signals/{id}/draft`) or `inkwell draft <signal_id>` on the CLI. Drafts are cached back into the daily signal JSON (`drafts` field) so refresh/second-visit doesn't re-bill.
 - **Web UI is plain HTML/CSS/JS — no framework.** Jinja2 templates in [inkwell/templates/](inkwell/templates/), shared CSS at [inkwell/static/app.css](inkwell/static/app.css), one JS module per page in [inkwell/static/js/](inkwell/static/js/). Routes: [routes/pages.py](inkwell/routes/pages.py) renders HTML; [routes/api_profile.py](inkwell/routes/api_profile.py), [routes/api_settings.py](inkwell/routes/api_settings.py), [routes/api_scan.py](inkwell/routes/api_scan.py), [routes/api_signals.py](inkwell/routes/api_signals.py) handle JSON. Server binds `127.0.0.1` — there is no auth, by design (single-user localhost tool).
 - **Only `engage == "Yes"` rows populate the `Coolest comment` column** in CSV/Sheets exports; `Maybe`/`No` rows display em-dashes (`\u2014`). This is intentional noise reduction — don't "fix" it.
-- **Storage is JSON files on disk, not a database.** Everything under `data/` (`signals/`, `campaigns/`, `feedback/`, `scan_history/`, `progress.json`) is human-readable JSON. `ensure_data_dirs()` creates the subdirectories — call it before any storage I/O.
+- **Storage is JSON files on disk, not a database.** Everything under `data/` (`signals/`, `feedback/`, `scan_history/`, `progress.json`) is human-readable JSON. `ensure_data_dirs()` creates the subdirectories — call it before any storage I/O. (There was a `storage/campaigns.py` CRUD module; it was deleted as unwired dead code — don't resurrect it without a real feature behind it.)
 - **Resume-by-default is core to the scan loop.** [storage/progress.py](inkwell/storage/progress.py) tracks `completed_subs` and `processed_ids`; re-running the same day skips what's already done, and progress resets automatically on a new UTC date. Signals are also deduped by `id` on save. Any change to the scan loop must preserve both levels of dedup and the `KeyboardInterrupt` → flush-then-save-progress path in `__main__.py`.
 - **Google Sheets export has a fallback.** On write failure, [exporters/google_sheets.py](inkwell/exporters/google_sheets.py) writes to `fallback_rows.json`. It also creates one tab per day (`YYYY-MM-DD`) and appends via `INSERT_ROWS`. `token.json` stores OAuth credentials — deleting it forces re-auth.
+- **`scheduler/` is a Phase 1 placeholder.** [scheduler/scheduler.py](inkwell/scheduler/scheduler.py)'s `init_scheduler()` only logs and does nothing; `apscheduler` is a declared dependency but not yet wired in. Don't assume scheduled scans exist.
 - **Reddit scanner uses the public JSON API** (no PRAW, no auth). Rate-limit handling is hand-rolled: 2s sleep between calls (`REDDIT_SLEEP`), exponential backoff on 429 (5/10/15s), skip on 403, 3 retries on network errors. Post `status` (`active`/`archived`/`inactive`/`blocked`) is derived from Reddit flags and flows through to filters and exports.
+- **A Hacker News scanner exists** ([scanners/hackernews.py](inkwell/scanners/hackernews.py)) — registered, tested, uses the public Algolia HN Search API (no auth). Its `targets` are search queries/topics, not subreddits; it sets `metadata["subreddit"]` to `hn:<query>` so existing heuristics/exports keep working. It mirrors RedditScanner's two-phase shape (`fetch_comments=False` + `hydrate_comments`). **Caveat:** it's not yet wired into the CLI `scan` loop — `cmd_scan` in [__main__.py](inkwell/__main__.py) is still Reddit-specific (per-subreddit Sheets flush, Reddit row/URL construction, progress keyed on subreddits). Generalizing the loop to multiple platforms is a deliberate follow-up that must preserve the resume/dedup invariants below.
 
 ## Adding things