Fix XLSX fidelity and checkpoint overhead

Author: rastagan-git

.agents/skills/curate-aroma-data/SKILL.md

@@ -26,6 +26,7 @@ If the console launcher is unavailable, replace `aromanexus ...` with the equiva
    - State the input, selected worksheet for XLSX, new output path, selected provider, expected columns, skip patterns, whether odor annotations are requested, any existing-CAS confirmation column, approximate request count, cache behavior, and material access caveats.
    - Write a sibling output by default. Never reuse the input path as the output path; `--force` is only for a separate existing destination.
    - Keep XLSX input and output when worksheet formulas, formatting, or other workbook content must survive; CSV/TSV output is a flat export.
+   - Choose checkpoint cadence deliberately. Each `.partial.xlsx` remains a complete, openable workbook; shorter intervals provide a more recent recovery file but still perform more writes, even with optimized preservation.
 4. Run one focused command.
    - Identity and odor metadata: `aromanexus pubchem INPUT --identifier-column "CAS Number"`
    - Name lookup with dataset-specific structural rows: `aromanexus pubchem INPUT --identifier-column "Name" --skip-pattern '^C\d+$'`
@@ -39,13 +40,14 @@ If the console launcher is unavailable, replace `aromanexus ...` with the equiva
    - For XLSX only, append `--sheet "SHEET"` to any table command when the target is not the first worksheet. Never pass `--sheet` for CSV or TSV.
 5. Verify the result.
    - Re-run the inspection script on the same worksheet for XLSX, or without `--sheet` for CSV/TSV.
-   - Confirm identical row order and row count, expected new fields, typed status counts, source URL, retrieval time, version, and license/access fields. Treat a blank retrieval time as correct when no provider or cached representation was obtained, including an explicit pre-request skip.
+   - Confirm identical row order and row count, expected new fields, typed status counts, source URL, retrieval time, provider-interface or snapshot label, and license/access fields. Treat a blank retrieval time as correct when no provider or cached representation was obtained, including an explicit pre-request skip.
+   - For PubChem, treat `PubChem Version` as interfaces attempted, not proof that PUG-View contributed odor data. Read [references/output-schema.md](references/output-schema.md) before interpreting `PUG REST` versus `PUG REST + PUG-View`.
    - For XLSX output, also compare worksheet order and names, per-sheet content digests, workbook properties, non-target-sheet content, untargeted source formulas and cached values, styles, dimensions, and reported workbook features.
    - Treat `PubChem Status` as provider execution state, not proof of a uniquely resolved CAS. Check `PubChem CAS Resolution`, candidate count, and `Resolved CAS` separately.
    - Leave `multiple`, `missing`, `input_cas_conflict`, and `input_cas_invalid` CAS resolutions unresolved; retain all candidates and route only the affected rows to a targeted fallback source or manual review.
    - Treat `http_error`, `network_error`, `parse_error`, `missing_data`, `data_error`, `partial`, `blocked`, and `skipped` separately from `not_found`.
    - Consult [references/output-schema.md](references/output-schema.md) when reconciling columns or statuses.
-6. Report the output path, provider versions, status counts, partial failures, and any source terms the user must still review.
+6. Report the output path, provider-interface or snapshot labels, status counts, partial failures, and any source terms the user must still review.
 
 ## Guardrails
 

.agents/skills/curate-aroma-data/references/output-schema.md

@@ -56,10 +56,12 @@ PubChem odor enrichment is enabled by default. With `--no-odor`, skip PUG-View r
 
 ## Provenance
 
-Keep provider status, source URL, retrieval time, cache-hit flag, pinned version or snapshot, license URL, and message columns. For PubChem odor text, also keep contributor source names, URLs, and license URLs.
+Keep provider status, source URL, retrieval time, cache-hit flag, provider-interface or pinned-snapshot label, license URL, and message columns. For PubChem odor text, also keep contributor source names, URLs, and license URLs.
 
 `Retrieved At` is the timestamp of an actual provider or cached representation. Leave it empty for outcomes decided before any representation was obtained, including explicit skips, local input validation failures, and transport failures before a response. A received HTTP or parse-error response retains its retrieval timestamp.
 
+Interpret `PubChem Version` as the interfaces attempted for that row. Use `PUG REST` when odor lookup is disabled or the lookup ends before the PUG-View odor endpoint is attempted. Once that endpoint request begins, use `PUG REST + PUG-View`, even if the request produces no odor annotation or the row becomes `partial`. This label does not prove that PUG-View contributed data; inspect `PubChem Status`, `PubChem Message`, and the odor fields. Keep the version blank for a row skipped before calling the PubChem client.
+
 ## Workbook QA
 
 After every run, confirm:

README-CN.md

@@ -134,12 +134,14 @@ aromanexus --cache-dir .cache/aromanexus --timeout 30 pubchem compounds.xlsx
 
 [Openpyxl 无法保留所有 OOXML 功能](https://openpyxl.readthedocs.io/en/3.1/tutorial.html)。因此，AromaNexus 会先在内存中试写一遍；若检测到绘图形状、批注、ActiveX/OLE 控件、切片器、线程批注、VML、数字签名等已知高风险内容，或任何会被试写丢弃的 OOXML 包部件，就会在调用数据源前停止。Excel 的可选计算链可能会被移除，以便表格软件重新生成。若显式输出 CSV/TSV，结果只是扁平表格，无法保留 Excel 专属内容。
 
-默认来源记录包括数据源状态、来源 URL、获取时间、是否命中缓存、固定版本、许可 URL 与诊断信息。`Retrieved At` 只表示真正取得数据源响应或缓存表示的时间；显式 `skipped`、请求前发现的无效输入、尚未收到响应的网络失败等本地结果会留空。只有在确实需要旧版形状时才使用 `--no-provenance`。
+默认来源记录包括数据源状态、来源 URL、获取时间、是否命中缓存、数据源接口或固定快照标签、许可 URL 与诊断信息。`Retrieved At` 只表示真正取得数据源响应或缓存表示的时间；显式 `skipped`、请求前发现的无效输入、尚未收到响应的网络失败等本地结果会留空。只有在确实需要旧版形状时才使用 `--no-provenance`。
 
 PubChem 会单独报告 CAS 解析状态。仅当查询本身是已确认的 CAS、名称查询只剩一个校验有效的候选，或 `--existing-cas-column` 中的有效 CAS 确实出现在返回候选里时，才填入 `Resolved CAS`。已有 CAS 与候选冲突或本身无效时保持未解析；空单元格会回退到原有的 `unique`、`multiple` 或 `missing` 规则。对于 `partial` 数据源结果，只接受查询或已有 CAS 的正向确认；依赖完整候选集合的判断会保持 `not_evaluated`。原始标识符列和已有 CAS 列都不会被改写；若已有 CAS 列名与当前输出列重叠，CLI 会在请求前拒绝运行。
 
 PubChem 气味扩充默认开启。`--no-odor` 会跳过 PUG-View 请求，并且不新增或更新 `PubChem Odor`、`PubChem Odor Annotations`、`PubChem Odor Sources`、`PubChem Odor Source URLs`、`PubChem Odor License URLs`。如果输入中原本就有这些列，它们会原样保留。
 
+`PubChem Version` 是逐行记录的接口尝试标签。使用 `--no-odor`，或查询在尝试气味端点前已经结束时，该字段为 `PUG REST`；一旦开始请求 PUG-View，则为 `PUG REST + PUG-View`，即使该行随后报告为 `partial`。这个标签不代表 PUG-View 一定返回或贡献了气味注释；还应检查 `PubChem Status`、`PubChem Message` 与气味字段。在调用 PubChem 客户端前就被跳过的行，其版本字段保持为空。
+
 ```bash
 # 明确指定输出位置
 aromanexus pubchem compounds.xlsx --output results/compounds_enriched.xlsx
@@ -151,7 +153,7 @@ aromanexus pubchem compounds.xlsx --checkpoint-every 10
 aromanexus pubchem compounds.xlsx --output compounds_pubchem.xlsx --force
 ```
 
-检查点形如 `compounds_pubchem.partial.xlsx`：需要使用时会在访问数据源前验证，运行期间定期刷新，中断后保留，最终文件写入成功后删除。AromaNexus 只会删除本次运行自己创建且未被外部替换的检查点；无关的 `.partial` 文件不会被碰。若目标文件或本次必需的检查点路径已存在，命令会停止，除非显式传入 `--force`。即使使用 `--force`，输入路径或同一文件的别名也不能作为输出或检查点路径。
+检查点形如 `compounds_pubchem.partial.xlsx`：需要使用时会在访问数据源前验证，运行期间定期刷新，中断后保留，最终文件写入成功后删除。每次保存的检查点仍是完整、可直接打开的工作簿，而不是进度日志。保真写入路径会尽量减少重复的整本工作簿序列化开销，但更短的间隔仍意味着更多次写盘，是运行速度与恢复文件新鲜度之间的取舍。AromaNexus 只会删除本次运行自己创建且未被外部替换的检查点；无关的 `.partial` 文件不会被碰。若目标文件或本次必需的检查点路径已存在，命令会停止，除非显式传入 `--force`。即使使用 `--force`，输入路径或同一文件的别名也不能作为输出或检查点路径。
 
 成功的 HTTP 响应与下载快照默认缓存到 `~/.cache/aromanexus`。如需更改位置，可设置 `AROMANEXUS_CACHE_DIR`，或在子命令之前传入 `--cache-dir`；更名前的缓存环境变量仍可兼容使用。
 

README.md

@@ -134,12 +134,14 @@ For XLSX-to-XLSX runs, AromaNexus starts from an immutable copy of the source pa
 
 [Openpyxl cannot preserve every OOXML feature](https://openpyxl.readthedocs.io/en/3.1/tutorial.html). AromaNexus therefore performs an in-memory trial round trip and stops before provider calls when it detects known unsafe content—such as drawing shapes, comments, ActiveX/OLE controls, slicers, threaded comments, VML, or digital signatures—or any package part that the trial would discard. Excel's optional calculation chain may be removed so spreadsheet software can rebuild it. An explicit CSV/TSV output is a flat export and cannot retain Excel-only content.
 
-By default, provenance columns include provider status, source URL, retrieval timestamp, cache hit, pinned version, license URL, and a diagnostic message. `Retrieved At` records when a provider or cached representation was actually obtained. It stays empty for local pre-request outcomes such as an explicit `skipped` row, invalid input, or a network failure before any response. Use `--no-provenance` only for legacy-shaped output.
+By default, provenance columns include provider status, source URL, retrieval timestamp, cache hit, a provider-interface or pinned-snapshot label, license URL, and a diagnostic message. `Retrieved At` records when a provider or cached representation was actually obtained. It stays empty for local pre-request outcomes such as an explicit `skipped` row, invalid input, or a network failure before any response. Use `--no-provenance` only for legacy-shaped output.
 
 PubChem reports CAS resolution separately and populates `Resolved CAS` only when the query itself is a confirmed CAS, a name lookup has exactly one checksum-valid candidate, or `--existing-cas-column` supplies a valid CAS that appears among the returned candidates. A conflicting or invalid existing CAS keeps the result unresolved; a blank cell falls back to the normal `unique`, `multiple`, or `missing` decision. For a `partial` provider result, only positive query or existing-CAS confirmation can resolve the row; decisions that depend on a complete candidate set remain `not_evaluated`. The original identifier and existing-CAS columns are never rewritten, and the CLI rejects an existing-CAS column name that overlaps an active output column.
 
 PubChem odor enrichment is enabled by default. `--no-odor` skips PUG-View requests and does not add or update `PubChem Odor`, `PubChem Odor Annotations`, `PubChem Odor Sources`, `PubChem Odor Source URLs`, or `PubChem Odor License URLs`. If those columns already exist in the input, they are preserved unchanged.
 
+`PubChem Version` is a per-row interface-attempt label. It is `PUG REST` when `--no-odor` is used or the lookup ends before the odor endpoint is attempted. Once a PUG-View request begins, it is `PUG REST + PUG-View`, including when the row later reports `partial`. This label does not prove that PUG-View returned or contributed an odor annotation; inspect `PubChem Status`, `PubChem Message`, and the odor fields. A row skipped before the PubChem client is called keeps a blank version.
+
 ```bash
 # Choose an output explicitly
 aromanexus pubchem compounds.xlsx --output results/compounds_enriched.xlsx
@@ -151,7 +153,7 @@ aromanexus pubchem compounds.xlsx --checkpoint-every 10
 aromanexus pubchem compounds.xlsx --output compounds_pubchem.xlsx --force
 ```
 
-Checkpoints are named like `compounds_pubchem.partial.xlsx`. A required checkpoint path is validated before provider access, refreshed during the run, preserved if processing is interrupted, and removed after the final output succeeds. AromaNexus deletes only a checkpoint created by the current run; an unrelated or externally replaced `.partial` file is left alone. Existing destinations and required checkpoint paths cause the command to stop unless `--force` is supplied. The input path—or an alias of the same file—can never be used as an output or checkpoint path, including with `--force`.
+Checkpoints are named like `compounds_pubchem.partial.xlsx`. A required checkpoint path is validated before provider access, refreshed during the run, preserved if processing is interrupted, and removed after the final output succeeds. Every saved checkpoint remains a complete, openable workbook rather than a progress journal. Its preservation path reduces the overhead of repeated full-workbook serialization, but shorter intervals still perform more writes and trade speed for a more recent recovery file. AromaNexus deletes only a checkpoint created by the current run; an unrelated or externally replaced `.partial` file is left alone. Existing destinations and required checkpoint paths cause the command to stop unless `--force` is supplied. The input path—or an alias of the same file—can never be used as an output or checkpoint path, including with `--force`.
 
 Successful HTTP responses and downloaded snapshots are cached under `~/.cache/aromanexus` by default. Set `AROMANEXUS_CACHE_DIR` or pass `--cache-dir` before the subcommand to use another location; the pre-rename cache environment variables remain accepted for compatibility.
 

aromanexus/__init__.py

@@ -3,4 +3,4 @@
 from aromanexus.models import LookupResult
 
 __all__ = ["LookupResult"]
-__version__ = "0.3.1"
+__version__ = "0.3.2"