[FEATURE] Configure HTTP User-Agent

[reubano]

Summary

Closes #12.

Stacked on #84. #84 establishes riko/<version> as the default HTTP User-Agent; this PR adds an optional per-pipe user_agent override while preserving that default when unset.

This PR:

• adds shared user_agent configuration to URL-fetching source pipes and exchangerate;
• resolves explicit values through the same default-UA policy established by [FIX] Send default User-Agent on HTTP requests #84;
• forwards the option through sync Requests/urllib and async HTTPX paths;
• carries the override through feed discovery and the subsequent feed fetch;
• keeps arbitrary request headers and authentication out of the legacy fetch surface;
• adds coverage for default/custom async resolution plus sync/async source propagation.

Dependency

This PR targets fix/default-user-agent and should merge after #84. Once #84 lands in features, this PR can be retargeted to features so only the #12 delta remains.

Commits

1. [TEST] Cover configurable HTTP user agent
2. [FEATURE] Configure HTTP user agent

[Copilot]

🔵 Needs a closer look

It’s a cross-cutting change affecting multiple sync/async HTTP entry points and generated config layers, and it also depends on stacked behavior from #84.

Pull request overview

Adds an optional per-pipe user_agent override to Riko’s HTTP-capable pipes while preserving the default riko/<version> policy (as established by the stacked #84), and propagates that value through both sync (requests/urllib) and async (httpx/anyio) fetch paths.

Changes:

• Introduces user_agent as shared configuration across URL-fetching source pipes (and exchangerate) and wires it through their sync/async parsers.
• Extends the async HTTP utilities (async_get, async_url_open, async_url_read, and RSS discovery) to accept and apply the resolved User-Agent.
• Adds internal tests covering async User-Agent resolution and propagation through fetchpage, plus sync Fetch(...) header forwarding.

File summaries

File	Description
tests/internal/test_user_agent.py	Adds contract tests for custom/default User-Agent resolution and pipe forwarding.
riko/types/modules.py	Adds shared user_agent fields to relevant TypedDict conf contracts via _Http* bases.
riko/types/_configs.py	Regenerates parse-time Objconf types to include user_agent for affected modules.
riko/modules/xpathfetchpage.py	Forwards user_agent into sync/async URL open paths and documents the option.
riko/modules/fetchtext.py	Forwards user_agent into sync/async URL open paths and documents the option.
riko/modules/fetchtable.py	Forwards user_agent into sync/async URL open paths and documents the option.
riko/modules/fetchsitefeed.py	Propagates user_agent through RSS autodiscovery and subsequent feed fetch.
riko/modules/fetchpage.py	Forwards user_agent into sync Fetch and async async_url_read, updates docs.
riko/modules/fetchdata.py	Forwards user_agent into sync Fetch and async URL open paths, updates docs.
riko/modules/fetch.py	Propagates user_agent into feed fetching/parsing for sync and async paths.
riko/modules/feedautodiscovery.py	Propagates user_agent into RSS autodiscovery (sync + async).
riko/modules/exchangerate.py	Applies user_agent to both httpx (async_get) and sync Fetch request paths.
riko/modules/csv.py	Forwards user_agent into sync/async URL open paths and documents the option.
riko/bado/io.py	Adds user_agent plumbing to async URL helpers so httpx requests use the resolved UA.
riko/bado/_util.py	Adds user_agent to async_get and applies it via request headers (httpx).
riko/autorss.py	Adds user_agent to RSS discovery helpers and forwards into underlying fetch.
riko/_io.py	Adds shared resolve_user_agent() and uses it in sync HTTP opener headers.

Review details

• Files reviewed: 17/17 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.