Skip to content

feat: LLM token ergonomics — list_paragraphs default cap, compact SearchResult, context() helper (ISSUES.md #43) - #60

Merged
pablospe merged 3 commits into
mainfrom
task/a5a11571-LLM-token-ergonomics--list_par
Jul 22, 2026
Merged

pablospe merged 3 commits into
mainfrom
task/a5a11571-LLM-token-ergonomics--list_par

Conversation

@pablospe

@pablospe pablospe commented Jul 17, 2026 •

Copy link
Copy Markdown
Owner

Summary

Implements ISSUES.md #43 (LLM token ergonomics). Note: that number is a repo-local ISSUES.md item, not a GitHub issue — intentionally no Closes keyword.

  • list_paragraphs default cap: a bare call now returns at most 200 paragraphs (previously unbounded). Whenever paragraphs remain beyond the returned window — default or explicit limit — the last entry is a truncation notice, e.g. "... 50 more paragraphs; use start=201 or limit=None", telling the caller the next start. Notice lines always begin with ... and never match the P{i}#{hash} ref shape, so ref-consuming code can filter them with entry.startswith("..."). limit=None restores the full listing.
  • list_paragraphs_structured: same 200-record default cap, but silent (no notice record) to keep the result homogeneously typed — truncation is detected by comparing len(result) with paragraph_count().
  • SearchResult ergonomics: new paragraph_index field (1-based, the same integer embedded in paragraph_ref — no more string-parsing refs) and a compact one-line repr/str (SearchResult(P3#a7b2 occ=1 '30 days'), with a trailing spans_rev marker) so printing a whole find_all() list stays cheap.
  • Document.context(ref, window=2): returns the paragraphs around a ref as ParagraphInfo records, clamped at document edges — the "show me the section around this match" helper. Shares parse/bounds/hash validation with get_paragraph_location via the extracted _resolve_validated_ref().
  • Call-site and docs sweep: internal callers that consume entries as refs (benchmarks, conftest.find_ref, a docstring example) now pass limit=None; README, quickstart, SKILL.md, and api.md rewritten to the notice-based pagination idiom; the docx-session eval help example uses limit=None so filtered output cannot silently miss matches past P200.

Testing

  • Full suite: 1158 passed, 2 skipped (uv run pytest)
  • New coverage: TestListParagraphsDefaultCap (8 tests, 250-paragraph fixture), TestSearchResultErgonomics (5), TestDocumentContext (10, incl. stale-hash, out-of-range, closed-doc)
  • ruff check / ruff format --check clean; ty check exit 0 with baseline-identical diagnostics

Summary by CodeRabbit

  • New Features

    • Added paragraph pagination with a default limit of 200 and continuation notices for larger documents.
    • Added surrounding-context retrieval for referenced paragraphs, with configurable window sizes.
    • Search results now include the paragraph’s 1-based document index and a compact display format.
  • Documentation

    • Updated API, quick-start, and usage guidance with pagination, truncation handling, paragraph context, and search-result details.
  • Bug Fixes

    • Improved paragraph reference validation and clearer errors for invalid, outdated, or out-of-range references.

…rchResult, context() helper (ISSUES.md #43)

- list_paragraphs: bare calls now return at most 200 paragraphs; when more
  remain, a trailing "... N more paragraphs; use start=… or limit=None"
  notice gives the next start. Notices always begin with "..." and never
  match the P{i}#{hash} ref shape. limit=None restores the full listing.
- list_paragraphs_structured: same 200-record default cap, silent (no notice
  record) to keep the result homogeneously typed; detect truncation by
  comparing len(result) with paragraph_count().
- SearchResult: new paragraph_index field (1-based, same integer as in
  paragraph_ref) and a compact one-line repr/str
  (SearchResult(P3#a7b2 occ=1 '30 days'), trailing spans_rev marker).
- Document.context(ref, window=2): the paragraphs around a ref as
  ParagraphInfo records, clamped at document edges; shares ref validation
  with get_paragraph_location via the extracted _resolve_validated_ref().
- Internal callers that consume entries as refs (benchmarks, conftest
  find_ref, docstring example) now pass limit=None; README/quickstart/
  SKILL.md/api.md rewritten to the notice-based pagination idiom; the
  docx-session eval help example uses limit=None so filtered output cannot
  silently miss matches past P200.
@coderabbitai

coderabbitai Bot commented Jul 17, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@pablospe, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 22 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: bf460ef8-3bef-40d4-bb52-0a1da73fee5d

📥 Commits

Reviewing files that changed from the base of the PR and between f59c1e8 and 5d602bd.

📒 Files selected for processing (8)
  • README.md
  • docs/api.md
  • docx_editor/document.py
  • docx_editor/session.py
  • docx_editor/track_changes.py
  • skills/docx/SKILL.md
  • tests/test_find_all.py
  • tests/test_paragraph_hash.py
📝 Walkthrough

Walkthrough

Paragraph listing now defaults to 200 entries with truncation notices, while structured listings remain silent when capped. Documents gain validated surrounding-context lookup, and search results expose paragraph indexes with compact representations. Tests, benchmarks, examples, and documentation reflect these contracts.

Changes

Paragraph API updates

Layer / File(s) Summary
Capped paragraph listing behavior
docx_editor/document.py, tests/test_paragraph_hash.py, benchmarks/*, docs/*, README.md, skills/docx/SKILL.md
Paragraph listings default to 200 entries; unstructured results append continuation notices, structured results do not, and full-enumeration callers pass limit=None.
Validated paragraph context
docx_editor/document.py, tests/test_document.py, skills/docx/SKILL.md
Document.context() validates references and returns bounded surrounding ParagraphInfo records, with coverage for boundaries and validation errors.
Search result metadata and representation
docx_editor/track_changes.py, docx_editor/document.py, tests/test_find_all.py, docs/api.md
SearchResult gains paragraph_index; find APIs populate it and representations use compact one-line formatting with conditional revision markers.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SearchResult
  participant Document
  participant ParagraphInfo
  SearchResult->>Document: provide paragraph_ref
  Document->>Document: validate index and paragraph hash
  Document->>ParagraphInfo: return surrounding paragraph records
Loading

Possibly related PRs

Poem

I hopped through pages, two hundred at a time,
With notices neatly marking the line.
Context now cuddles each paragraph near,
Search results carry their index clear.
A compact little repr makes rabbits cheer!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 9.09% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes: list_paragraphs pagination, compact SearchResult, and the new context() helper.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch task/a5a11571-LLM-token-ergonomics--list_par

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docx_editor/document.py`:
- Around line 570-599: Update _resolve_validated_ref to return a named immutable
ResolvedRef dataclass containing the validated index, paragraph element, and
fetched paragraphs list. Modify get_paragraph_location and other callers to use
these fields, reusing ResolvedRef.paragraphs for context slicing instead of
calling getElementsByTagName("w:p") again, while preserving existing validation
behavior.

In `@skills/docx/SKILL.md`:
- Line 636: Remove the trailing space from the inline code span in the
paragraph-reference documentation, changing the displayed token to represent the
pipe without whitespace while preserving the surrounding explanation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1992c355-f711-4e0b-943f-085935f5be9e

📥 Commits

Reviewing files that changed from the base of the PR and between 8235f24 and f59c1e8.

📒 Files selected for processing (13)
  • README.md
  • benchmarks/corpus/corpus_harness.py
  • benchmarks/hash_anchored_vs_plain.py
  • docs/api.md
  • docs/quickstart.md
  • docx_editor/document.py
  • docx_editor/session.py
  • docx_editor/track_changes.py
  • skills/docx/SKILL.md
  • tests/conftest.py
  • tests/test_document.py
  • tests/test_find_all.py
  • tests/test_paragraph_hash.py

Comment thread docx_editor/document.py Outdated
Comment thread skills/docx/SKILL.md
pablospe added 2 commits July 17, 2026 04:07
… detection, occ=0 examples, singular notice, no redundant DOM query

- document.py: '.. versionchanged::' reST directives replaced with Google-style
  'Note:' sections (mkdocstrings' google parser renders the directive as
  literal text); list_paragraphs_structured docstring example now demonstrates
  truncation detection against a bounded call via the last record's index
  (the limit=None example could never trigger the check); notice noun
  pluralizes ('1 more paragraph'); _resolve_validated_ref returns the
  validated paragraph list so get_paragraph_location no longer re-queries
  the DOM.
- docs/api.md: bold 'Changed in 0.6.1:' label (file convention), occ=0 in the
  repr example (paragraph_occurrence is 0-based), index-based truncation rule.
- SKILL.md: occ=0 repr example, index-based truncation rule.
…, complete limit=None sweep

- track_changes.py: repr elides matched text past 60 chars (display only;
  the text attribute keeps the full match) — sentence-length search anchors
  otherwise blow up the exact token budget the compact repr exists for.
- SKILL.md refs_only example and docx-session exec help example now pass
  limit=None like their siblings, so no entry can be a truncation notice.
- README: merge the duplicated bare list_paragraphs() intro line into the
  pagination example.
- docs/api.md: document the repr elision.
@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.7%. Comparing base (8235f24) to head (5d602bd).

Additional details and impacted files
@@          Coverage Diff          @@
##            main     #60   +/-   ##
=====================================
  Coverage   95.7%   95.7%           
=====================================
  Files         11      11           
  Lines       3740    3764   +24     
  Branches     744     746    +2     
=====================================
+ Hits        3581    3605   +24     
  Misses        73      73           
  Partials      86      86           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pablospe
pablospe merged commit 3446ef5 into main Jul 22, 2026
10 checks passed
pablospe added a commit that referenced this pull request Jul 30, 2026
…structure, workspace reset (ISSUES.md #52 + #60)

Six code items plus two documentation items from the round-3/4 dogfood papercut
list (ISSUES.md #52 and #60 — repo-local item numbers, not GitHub issues).

- SearchResult stands in for (text, paragraph, occurrence) on replace, delete,
  insert_after, insert_before, add_comment and the four EditOperation
  constructors. Passing paragraph=/occurrence= alongside one raises rather than
  picking a winner.
- ParagraphInfo gains in_table / style / outline_level, so a table of contents
  needs one list_paragraphs_structured() call instead of a second
  list_paragraph_locations() pass.
- EditValidationResult.current_ref gives dry-run stale-hash rows the ref for the
  paragraph's current content. It clears the hash check only; the rebuilt
  operation is re-validated like any other.
- Document.discard_workspace() replaces the deep Workspace import for resetting
  after a crashed script.
- ExecResult/EvalResult gain started, distinguishing a timeout of your own
  running code from one that never left the kernel queue.
- replace() trims whitespace-only spans character-wise, so a spacing fix is a
  pure insertion or deletion instead of an invisible del+ins pair.
- Docs: pagination overshoot documented as a valid empty answer (no behaviour
  change), get_paragraph()/context() marked O(document).

Two fidelity fixes found in review, both beyond the original scope:

- w:delText never carried xml:space="preserve", so a conforming consumer trimmed
  the edges and rejecting a whitespace deletion restored the wrong spacing. Every
  tracked deletion whose text has a whitespace edge now carries the attribute.
- ipykernel discards requests queued behind a command that raises. exec_code
  reported those as status="ok" (exit 0, no output) having run nothing; started
  now detects it, the exec CLI warns loudly, and eval names the discard instead
  of blaming its transport. Exit codes unchanged.

word/styles.xml is memoized per Document and parsed without the line-tracking
parser, keeping get_paragraph() at its previous per-call cost now that it needs
the style maps; the location APIs get an ~11x win on repeat calls.

Reviewed by three independent reviewers plus codex, red-teamed: 11 findings
confirmed and fixed, 3 killed as false positives, 6 downgraded; a second pass
caught and fixed an overstatement in the first round's own fix.

1409 passed, 2 skipped. CI green on Python 3.10-3.14.
@pablospe
pablospe deleted the task/a5a11571-LLM-token-ergonomics--list_par branch August 30, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant