直接运行已有零售渠道爬虫，或把新渠道网址交给 Agent 诊断并接入。
Run built-in retail scrapers—or give an agent a retailer URL to assess and onboard a new channel.
| 你的目标 / Your goal | 从这里开始 / Start here |
|---|---|
| 抓取已支持的零售商 / Scrape a supported retailer | 安装后运行 retail-scrape catalog 或 retail-scrape prices / Install, then run catalog or prices |
| 接入新零售商或市场 / Add a retailer or market | 运行 retail-scrape assess <url>，再使用随附的 Agent Skill / Run assess, then use the bundled agent skill |
| 修复失效的 adapter / Repair a broken adapter | 把渠道、错误和证据交给 Agent Skill / Give the channel, error, and evidence to the agent skill |
Agent 只参与一次性的渠道接入或修复。验证通过的 adapter 可以像普通 Python 任务一样在本地、GitHub Actions 或你自己的调度器中运行，日常抓取不需要调用模型。
The agent is only involved in one-time onboarding or repair. Once validated, adapters run as ordinary Python jobs locally, in GitHub Actions, or in your own scheduler—daily scraping does not require a model call.
项目尚未正式发布到 PyPI。当前 alpha 版本请从 GitHub 安装：
The project is not yet published on PyPI. Install the current alpha from GitHub:
git clone https://github.com/Mrjie7205/retail-scrapers.git
cd retail-scrapers
python -m pip install -e .正式发布后，推荐的安装方式会简化为 pipx install retail-scrapers。在 PyPI 页面可用之前，请不要依赖这条命令。
After the first PyPI release, the recommended path will become pipx install retail-scrapers. Do not rely on that command until the PyPI page is live.
让工具检查运行环境并准备可选的浏览器依赖：
Let the tool check the environment and prepare optional browser dependencies:
retail-scrape setup如果只想检查、不希望下载缺失的 Chromium，请使用 retail-scrape setup --check-only。
To check without downloading a missing Chromium installation, use retail-scrape setup --check-only.
然后运行完全离线的演示。它不访问零售网站，默认在 retail-scrapers-demo/ 生成 catalog.jsonl 和 prices.jsonl：
Then run the fully offline demo. It does not contact a retailer and writes catalog.jsonl and prices.jsonl to retail-scrapers-demo/ by default:
retail-scrape demo需要其他路径或 CSV 时，可使用 --output-dir PATH 和 --format csv。
Use --output-dir PATH and --format csv when you need another directory or CSV output.
先查看当前版本支持的渠道：
See the channels supported by the installed version:
retail-scrape channels再抓取少量 catalog 数据。下面是跨平台单行命令，可直接在 PowerShell、Command Prompt、Bash 或 zsh 中使用：
Then fetch a few catalog rows. This single-line command works in PowerShell, Command Prompt, Bash, and zsh:
retail-scrape catalog --channel elkjop-no --max-items 3 --no-strict --output output/elkjop.jsonl真实网站会变化。离线 demo 通过只说明本地安装正常；当前的小规模真实抓取才是渠道健康状态的最终检查。
Live sites change. A passing offline demo proves the local installation works; a current small live scrape remains the final channel health check.
所有渠道都输出统一的 JSONL 或 CSV 字段，同时保留零售商的原始币种：
Every channel emits the same JSONL or CSV contract while preserving the retailer's native currency:
{"channel":"elkjop-no","country":"NO","sku":"demo-1001","brand":"Example","title":"Example 55 inch TV","url":"https://www.example.com/product/demo-1001","price":7990.0,"currency":"NOK","availability":"in_stock"}查看完整字段说明：
Inspect the complete contract:
retail-scrape schema catalog --format markdown价格任务接受包含 id,url 两列的 CSV。仓库中的 examples/products.template.csv 只是输入模板，不能直接运行；其中的 example.com 和 replace-me 是故意保留的占位内容。
Price jobs accept a CSV with id,url columns. The included examples/products.template.csv is an input template, not a runnable example; its example.com and replace-me values are intentional placeholders.
即使只安装了 PyPI 包、没有克隆仓库，也可以按下面两行创建自己的输入文件：
If you installed only the PyPI package and did not clone the repository, create your own input file with these two lines:
id,url
your-product-id,https://www.example.com/product/replace-me请先复制模板，并把每个 URL 换成所选渠道的真实公开商品页。确认没有占位值后再运行：
Copy the template first and replace every URL with a real public product page from the selected channel. Run the command only after all placeholders are gone:
retail-scrape prices --channel currys-gb --input path/to/your-products.csv --output output/prices.jsonl严格模式默认开启；可用的完整性检查或价格成功率低于阈值时，命令会失败。--no-strict 只适合探索和小规模 smoke test。
Strict mode is enabled by default; the command fails when available completeness checks or price success rates fall below their thresholds. Use --no-strict only for exploration and small smoke tests.
先做安全的本地诊断；默认情况下，这条命令不会访问目标网站：
Start with a safe local assessment; by default, this command does not contact the target site:
retail-scrape assess https://www.example.com --format markdown确定性诊断会返回 exact_adapter、market_config 或 new_adapter。只有 Agent 验证了稳定 ID、分页、价格含义、币种和网络行为后，才能把 new_adapter 进一步判断为 pattern_reuse。
The deterministic assessment returns exact_adapter, market_config, or new_adapter. Only an agent that has verified stable IDs, pagination, price meaning, currency, and network behavior may refine new_adapter to pattern_reuse.
-
exact_adapter：直接运行已有 adapter。exact_adapter: run an existing adapter. -
market_config：给已有零售商引擎增加市场配置。market_config: add market settings to an existing retailer engine. -
pattern_reuse：经 Agent 证据验证后，复用已证明可行的采集模式。pattern_reuse: reuse a proven acquisition pattern after agent evidence review. -
new_adapter：在证据支持复用之前，保持渠道实现相互隔离。new_adapter: keep the retailer isolated until evidence supports reuse.
然后让你的 coding agent 阅读 skills/retail-scrapers/SKILL.md，并发送类似任务：
Then point your coding agent at skills/retail-scrapers/SKILL.md and send a task like this:
Codex 用户可以直接安装 wheel 随附的 Skill；其他 Agent 可以通过 --target 指定自己的 Skill 父目录：
Codex users can install the skill bundled in the wheel directly; use --target to choose a skill parent directory for another agent:
retail-scrape skill install --agent codex也可以让诊断命令直接生成一段可复制给 Agent 的任务：
The assessment command can also generate a copy-ready agent task:
retail-scrape assess https://retailer.example --format agent使用 $retail-scrapers 为 https://retailer.example 接入 catalog 和当前价格；保留来源币种，如果公开访问不足以完成任务，请输出阻塞报告。
Use $retail-scrapers to add https://retailer.example for catalog and current prices. Keep the source currency and stop with a blocker report if public access is insufficient.
Skill 会引导 Agent 完成诊断、公开证据采集、架构选择、脱敏 fixture、离线测试和有界 live smoke。它不会承诺所有网站都一定可以抓取。
The skill guides the agent through assessment, public evidence collection, architecture selection, sanitized fixtures, offline tests, and a bounded live smoke check. It does not promise that every website is scrapeable.
| Channel ID | 市场 / Market | Catalog | Price | 主要策略 / Main strategy |
|---|---|---|---|---|
amazon-de |
德国 / Germany | Yes | Yes | Amazon 共用引擎，原生 EUR 校验 / Shared engine, native EUR guard |
amazon-gb |
英国 / United Kingdom | Yes | Yes | Amazon 共用引擎，原生 GBP 校验 / Shared engine, native GBP guard |
amazon-it |
意大利 / Italy | Yes | Yes | Amazon 共用引擎，原生 EUR 校验 / Shared engine, native EUR guard |
amazon-es |
西班牙 / Spain | Yes | Yes | Amazon 共用引擎，原生 EUR 校验 / Shared engine, native EUR guard |
boulanger-fr |
法国 / France | Yes | Yes | 品牌 facet、HTML、Schema.org fallback / Brand facets, HTML, Schema.org fallback |
currys-gb |
英国 / United Kingdom | Yes | Yes | 隔离 session、Schema.org fallback / Isolated sessions, Schema.org fallback |
elkjop-no |
挪威 / Norway | Yes | Yes | Algolia catalog、tRPC price、页面 fallback / Algolia catalog, tRPC prices, page fallback |
运行 retail-scrape health --format markdown 可以查看维护说明，但仍应使用小规模 live smoke 确认网站当前状态。
Run retail-scrape health --format markdown for maintenance notes, but use a small live smoke run to confirm the site's current state.
把 examples/github-actions/scheduled-scrape.yml 复制到你的 fork，即可运行手动 catalog/price 任务和定时 smoke 任务。输出会作为 GitHub Actions artifact 上传。
Copy examples/github-actions/scheduled-scrape.yml into your fork to run manual catalog/price jobs and a scheduled smoke job. Outputs are uploaded as GitHub Actions artifacts.
配置步骤见自动化说明。
See GitHub Actions setup.
本仓库只负责数据提取，不包含数据库、长期价格历史、汇率换算、私有产品匹配、提醒、看板、账号、Cookie 或真实抓取数据。这些下游选择由你的应用负责。
This repository handles extraction only. It does not include a database, long-term price history, currency conversion, private product matching, alerts, dashboards, accounts, cookies, or real scraped datasets. Your application owns those downstream choices.
不同零售商的代码保留在相互隔离的 adapter 中；共享数据结构、校验、运行参数和 CLI 行为则保持统一。项目不会假设所有网站都有相同的反爬或数据访问结构。
Retailer-specific code stays in isolated adapters; shared records, validation, runtime controls, and CLI behavior remain consistent. The project does not pretend that every website has the same anti-bot or data-access shape.
安装开发依赖并运行本地检查：
Install the development dependencies and run the local checks:
python -m pip install -e ".[dev]"
ruff check .
pytest
mypy src/retail_scrapers提交 Issue 或 Pull Request 前，请阅读中文贡献指南。
Read CONTRIBUTING.md before opening an issue or pull request.
只使用无需登录的公开信息。请检查目标网站条款、robots 规则和适用法律，保持合理请求频率，不要收集个人信息或绕过访问控制。
Use public, non-login information only. Review the target site's terms, robots rules, and applicable law; keep request rates reasonable; and never collect personal information or bypass access controls.
MIT