Repository navigation
Conversation
`Tree.cache` and `TreeModifier` were announced as removed in an old changelog, but remain exported and supported. Their documentation also suggested sorting alone would make edits persistent, leaving their status and practical limits unclear. Document that cache edits affect one instance and do not update its object ID, stored tree, parents, index, or commits. Correct the historical removal claim and recommend `IndexFile.write_tree()` for staged path changes. Serialization already delegates construction to Git or Gix; no deprecation or replacement implementation is needed. Git's `Documentation/git-mktree.adoc` confirms that `mktree` normalizes entry order without pre-sorted input. Add a regression covering local subtree edits, serialization into a new object, and unchanged parent references. The focused tree tests passed with both CLI and Gix backends (3 tests each), using Git 2.54.0 and GixPython 0.3.0. The HTML build rendered the updated documentation; its strict check reported one existing field-list warning in `git.objects.submodule.util.is_relative_url`.
Sphinx omits underscore-prefixed methods by default, hiding documented extension points such as `RemoteProgress._parse_progress_line()` and `LockFile._obtain_lock()`. Traversal documentation used a separate allowlist, and `Tree.traverse()` linked to its implementation using an ordinary URL. Mark supported subclass interfaces with Sphinx's built-in `:meta public:` docstring metadata. Remove the traversal-specific `:private-members:` list, use a method reference for traversal documentation, and explain this convention in `CONTRIBUTING.md`. Private implementation helpers stay hidden. Repair an existing malformed `is_relative_url()` return field so strict HTML validation can complete. Before this change, the HTML reference lacked the progress and lock extension-point targets. The strict Sphinx HTML build now passes. Assertions against its output verify all five selected interface targets, the traversal link, and omission of `LockFile._lock_file_path()`. Runtime logic is unchanged.
Aider-style staged-file reads fail with `UnicodeDecodeError` when a raw diff contains a non-UTF-8 filename. Index entries and blob streams already preserve the legacy bytes, but `Diff._handle_diff_line()` decoded the entire NUL-delimited output before separating metadata from paths. Keep raw diff fields as bytes, decode only the ASCII mode/OID/status metadata, and pass original path bytes into `Diff`. This also preserves rename and copy destinations without requiring callers to reencode them. The existing text path properties retain their display behavior. Git 2.54.0's `Documentation/diff-options.adoc` specifies that `--raw -z` returns unmodified pathnames separated by NULs. All six new regressions failed before the fix: five cover raw add/delete/ modify/rename/copy records, and one stages Windows-1252 filename and content bytes in a real index and reads its diff and blobs. All diff and index tests now pass with CLI and Gix backends: 150 passed, 2 skipped, and 9 subtests on each. Ruff and ShellCheck pass. Windows execution remains for CI validation; no conversion of the legacy files is needed.
Packed-reference enumeration now delegates to Git/Gix and no longer has the original UTF-8 file-reader failure. However, tag paths retain undecodable bytes as surrogate escapes, and passing those strings back into GixPython raises `UnicodeEncodeError` before validation or reference lookup. Make `safe_encode()` the byte-preserving inverse of `safe_decode()`. Use it at the supported Gix reference validation, lookup, revision, reflog and edit boundaries, and when serializing Git reference transactions. GixPython 0.3.0 accepts bytes in these APIs, so this fixes adapter conversion without adding a reference-storage implementation or requiring an upstream workaround. Add a scripted packed-ref fixture with lightweight and annotated non-UTF-8 tag names for both SHA-1 and SHA-256. Git 2.54.0's `for-each-ref` supplies the reference inventory; `git-for-each-ref.adoc` documents the `refname` and `objectname` projections used for the baseline. The regression checks byte round trips, tag/commit resolution, CLI parity and absence of CLI fallback during native resolution. Packed records are fixture inputs because APFS rejects those bytes in loose reference filenames. Validation: 42 reference tests and 24 subtests passed with CLI; 144 reference and backend tests and 24 subtests passed with Gix. Ruff, ShellCheck and mypy passed. Windows behavior will be validated by CI.
Reading `repo.head.commit` can open persistent object readers or retain a native repository handle. Deleting the repository while its handles remain open is not a supported sequence on Windows. Git's read-only object files can independently prevent plain `shutil.rmtree()` from removing a clone. `Repo.close()` and its context-manager support already release the retained handle and terminate persistent Git readers synchronously. Document that contract in the API reference, tutorial and README, replacing the old advice to invoke destructors manually. Demonstrate closing the repository first, then using the existing strict `git.util.rmtree()` helper for read-only files. All instances and separately opened readers must be closed by their owners. Add a packed-repository fixture and a tutorial regression that reads HEAD and blob data, verifies reader processes have exited and caches are released, and removes the directory without suppressing deletion errors. The test passes with both CLI and Gix backends on macOS with Git 2.54.0. Sphinx's strict HTML build, Ruff and ShellCheck pass. Windows execution is left to CI; no automatic cleanup of a still-live repository is introduced.
The dependency `test (gitdb, 3.9)` and project `install/import (ubuntu-latest, 3.9)` checks fail before installing code. `ubuntu-latest` now selects Ubuntu 26.04, and `actions/setup-python` reports that no Python 3.9 x64 build exists for that image. Run the dependency matrix and the Linux legacy-runtime import checks on Ubuntu 24.04. Preserve the import-check matrix names and the existing macOS/Windows runners. The main modern-Python suite continues to cover the current Ubuntu image. Validated both workflow files as YAML and checked the upstream `actions/python-versions` manifest for available Ubuntu 24.04 x64 builds of Python 3.8, 3.9 and 3.10. This is a CI environment repair discovered while validating the five issue-specific commits, not a runtime code change.
The Windows Python 3.15 free-threaded CI job enumerated the non-UTF-8 packed tags correctly, then failed both object-format variants when the new regression tried to resolve their surrogate-escaped names through the CLI. Git 2.56.0.windows.2 returned exit 128 from `git rev-parse --verify --end-of-options` with `fatal: Needed a single revision`. Git's Windows `wmain()` converts UTF-16 arguments to UTF-8, so those arguments cannot carry the original invalid UTF-8 reference bytes. Separate packed-tag enumeration from reference resolution. Keep the reported enumeration regression on both backends and every platform; skip only the Windows CLI resolution variant. Gix resolution still runs on every platform with `Git.execute()` forbidden, using the supported byte-capable native APIs. Document the demonstrated CLI limitation and the coverage boundary in `doc/gix-backend.md`. Validation: all 44 reference tests and 24 subtests pass locally on each backend; Ruff check/format and `git diff --check` pass. Windows execution remains for CI.
The Windows Gix job also failed both packed-tag resolution regressions: enumeration preserved the names, but `reference_info()` selected its native read diagnostic fallback before the no-CLI guard caught a Git launch. The run used GixPython 0.3.0, Git 2.56.0.windows.2 and CPython 3.12.10. The pinned `gix-ref` 0.69.1 lookup attempts loose-reference path conversion before consulting packed refs, which cannot represent invalid UTF-8 bytes on Windows. The CI log demonstrates the fallback, but does not contain the underlying Gix diagnostic text. Keep byte-name resolution coverage on POSIX and check the existing native fallback explicitly on Windows. Packed-tag enumeration remains covered on both backends on every platform, resolving the reported enumeration problem without claiming unsupported native lookup coverage. Record the open GIX-32 capability gap and required upstream lookup support in `doc/gix-backend.md`. Git's Windows CLI also cannot address those original bytes through UTF-16 arguments, so this evidence is not classified as a demonstrated Git compatibility mismatch. No library behavior changes. Validation: all four focused packed-tag regressions pass on each local backend; Ruff check/format and `git diff --check` pass. The Windows fallback assertion remains for CI.
Review item #2274 (comment) reports that "reading them raises `AttributeError`" when native decoding has no tagger, and requests preserving initialized fields through a CLI fallback. Four SHA-1/SHA-256 regressions reproduce the native failure. The two cases with a message also expose CLI loss of the first message line, so falling back alone would not restore complete metadata. Adapt `Tag.tagger()` returning `None` to the existing empty `Actor` and zero date/offset defaults through `parse_actor_and_date("")`. Gix already exposes the absence correctly; no new Git semantics or fallback is required. Make the CLI parser recognize the optional tagger header and preserve the entire message. Git 2.54.0's `tag.c::parse_tag_buffer()` accepts an absent tagger and sets its date to zero. The existing GIX-28 strict writer gap remains open. All four regressions now pass on both backends; the native cases forbid `Git.execute()` during metadata access. Reference and CLI object/index tests pass on CLI (105 tests, 24 subtests) and Gix (105 tests, 24 subtests). Ruff, ShellCheck and mypy pass.
Review item #2274 (comment) reports that "subsequent iteration or lookup silently accepts that truncated tree" after an entry fails validation. Stored trees containing a `.git` entry reproduce this on both backends: the first iteration rejects the name, but the second accepts an empty tree or the valid prefix. Build the entries in a local list and attach `_cache` only after every record passes parsing and validation. Failed reads leave the lazy cache unset, so later accesses retry and raise the same error instead of exposing partial data. No backend behavior or tree validation policy changes. Git 2.54.0's `mktree` and `ls-tree` accept and emit the fixture's `.git` entry; GitPython applies its existing metadata-path guard while consuming that output. Four regressions cover invalid entries first and last in SHA-1/SHA-256 trees. All fail before the fix and pass afterwards on CLI and native Gix `ls_tree`. All 97 tree tests pass on each backend. Ruff, ShellCheck and mypy pass.
Review item #2274 (comment) reports that "the returned head points to `refs/heads/topic`, which may be missing or identify a different branch" after renaming to the short branch name `refs/heads/topic`. A repository with an existing `topic` at commit A and a source branch at commit B reproduces the incorrect returned ref on both backends. Compute the destination by prepending `refs/heads/` to the literal name passed to `git branch`. Validate that same destination and retain it only after Git successfully renames the branch. Git 2.54.0's `builtin/branch.c::copy_or_rename_branch()` uses `refs.c::check_branch_ref()`, which always prepends that namespace. The general `to_full_path()` helper continues to normalize already-qualified references for its other callers. The SHA-1/SHA-256 regressions fail before this fix and pass afterwards. They check the returned path, name and commit, current `HEAD`, and that the original `topic` still points to commit A. All 50 reference tests and 24 subtests pass on each backend, including ordinary and forced renames. Ruff, ShellCheck, mypy and basedpyright pass.
Byron
changed the title
The documentation described temporary development regressions as restored public behavior, including a `TreeModifier` deprecation that never shipped. Audit current guidance and the changelog against `3.2.1` and the relevant historical release tags so dependants see the net changes from releases. Describe tree editing, configuration assignment, revision paths, literal paths and compatibility imports directly. Clarify that `GitDB` selects the managed backend, and scope the version lookup improvement to source checkouts. Remove the historical tree-removal entry: its announcement shipped in `1.0.2`, but the API was present in both `1.0.1` and `1.0.2`. Keep genuine released deprecations and fixes. Identify the Gix integration as unreleased and its measurements as development validation. Remove internal restoration and commit-rewrite narration while preserving compatibility bugs, reproduction evidence and measurement caveats. The strict Sphinx HTML build, Ruff lint/format checks and `git diff --check` pass. Python AST comparison confirms that source edits affect only docstrings and comments; runtime tests were not rerun for this documentation-only change.
Retained native repositories required file and environment fingerprints, conditional reloads and a per-Repo refresh lock, while included configuration still needed unconditional refresh. Open a new `gix.Repository` through the shared accessor for each operation and remove `_gix_state`, the cached handle, the refresh lock and CLI-triggered invalidation. Keep execution-only configuration overrides out of configuration queries and preserve storage compatibility fallbacks. Update pickling, cleanup assertions and documentation for operation-local native handles. Reopening repeats native initialization; the historical retained-handle timings do not measure this policy. The ten focused Gix lifetime/discovery cases pass. Cleanup, fixture lifetime and the documented close-before-removal example pass on both backends (27 tests and one skip each), with localhost access for the Git daemon cases. Ruff lint and format checks pass. Benchmark timings were not rerun.
Diff raw paths preserved bytes, but their text properties decoded with replacement characters. Renaming `legacy-\xff` to `legacy-\xfe` collapsed both names to the same string, hid the rename and gave diff blobs unusable paths. Patch rename fields also retained Git's quoted spelling. Use the existing lossless decoder for text paths and submodule matching, compare raw rename names, and unquote patch rename fields through the existing path decoder. Document the intentional change from replacement characters to surrogate escapes in public text paths. The new raw/patch rename regressions fail before this change and pass with both `core.quotePath` settings on CLI and Gix. Git 2.54.0 supplies the fixture renames; native operation counts confirm the raw Gix path. All 49 Diff tests pass on each backend. Ruff, ShellCheck and whitespace checks pass.
Staging a symbolic link decoded its target through `os.readlink()` and then strictly re-encoded it. Targets containing undecodable filesystem bytes raised `UnicodeEncodeError` instead of being stored as the link blob. Use `os.fsencode()` to recover the original target bytes. Git's `object-file.c` stores the bytes read from a symlink directly; the scripted regression stages `../target-\xff` and verifies those exact object bytes. The fixture executes at its final location because native fixture copying follows symlinks. The regression fails before the fix and passes on both backends. Connected index tests pass on CLI and Gix; the combined Gix Diff/tree/index run passes 253 tests and nine subtests, with two platform skips. Ruff and ShellCheck pass.
`TreeModifier.set_done()` sorted decoded Unicode names. A name ending in an undecodable byte could sort before a valid UTF-8 name even when Git stores them in the opposite order, changing iteration order after cache editing. Recover each name's original bytes in the existing sort key and keep its directory terminator. This matches the byte comparison in Git's `tree.c::base_name_compare()` without changing tree construction, which still belongs to Git or Gix. The regression compares cache iteration with `git ls-tree -z` for byte and emoji names. It fails before the fix and passes on both backends. Connected tree tests pass on CLI and Gix; Ruff, ShellCheck and whitespace checks pass.
Git accepts configuration subsection names and values containing bytes that are not valid in the filesystem encoding. Strict decoding made configuration enumeration, remote names and URLs fail with `UnicodeDecodeError`; byte-valued writes also decoded strictly before reaching Git. Use the shared lossless decoder for Git's NUL-framed records and incoming byte values. Keep existing key validation, mutation framing and Git-managed locking. File and BytesIO regressions compare against `git config --null`, exercise remote names/URLs and verify exact bytes after writes. Four regressions fail before the fix. All 66 configuration tests pass on each backend with Git 2.54.0 and GixPython 0.3.0. Ruff, mypy and whitespace checks pass. Windows byte-valued argument writes remain excluded because Windows command-line arguments cannot preserve arbitrary original bytes.
The dedicated Gix submodule parser returns path, URL and branch values as bytes. Strict decoding forced valid non-UTF-8 values into the CLI fallback. Decode those values with the shared lossless helper while preserving existing checks for duplicate sections, implicit values and raw branch spelling. Record the remaining native API gap: non-UTF-8 subsection names require CLI because generic config keys require text, and the byte-capable branch getter interprets empty/implicit values instead of exposing their raw spelling. No Python configuration semantics or fabricated native success are introduced. The native regression fails before the fix and passes with Git.execute blocked. Scripted tests also verify byte subsection names through fallback and cached field reads against Git 2.54.0. Connected tests pass: nine Gix cases, two CLI cases and seven native-only skips. Ruff, mypy and whitespace checks pass.
`Commit.stats.files` exposed Git's C-quoted output as dictionary keys. With `core.quotePath=true`, byte names and many Unicode names were quoted; tabs, newlines, quotes and backslashes could also produce keys that were not paths. After parsing the numeric records, unquote filename keys with the existing diff path decoder and lossless byte/text helpers. Unquoting after framing keeps tabs and newlines inside filenames from becoming record delimiters. Native statistics and the existing Gix filename-quoting fallback are unchanged. The scripted Git 2.54.0 regression covers both quotePath settings, byte and Unicode names, literal quoting/backslashes, root/update/delete commits and binary counts. All eight subtests pass with both backends after failing before the fix; the connected CLI stats/rename selection passes five tests and eight subtests. Ruff, mypy and whitespace checks pass.
Git separates reference and remote URL records with LF, but Unicode `splitlines()` also split valid names at U+0085, U+2028 and U+2029. Enumeration could return a truncated, nonexistent reference. Fetch parsing and progress handling also stripped valid Unicode whitespace from the ends of names. Split these outputs only at LF, trim fetch column padding as ASCII spaces, and remove only CR/LF framing from progress records. Preserve empty remote-URL output behavior. Reference storage and native lookup remain owned by Git/Gix. Scripted SHA-1/SHA-256 repositories contain leading, middle and trailing separators in branch, tag and remote-ref names. Tests compare Git's inventory, resolve the names, feed bytes/text through the fetch progress handler, check local stale refs and retain Unicode separators in URLs. All 26 focused cases pass on each backend after reproducing the failures. Existing progress tests, Ruff, mypy and whitespace checks pass.
Status and untracked-file filtering passed surrogateescaped strings into GixPython. Converting those strings to Rust text raised `UnicodeEncodeError`, while equivalent Git CLI queries correctly matched the original byte names. Encode patterns with the shared lossless helper at the two native boundaries. `StatusPlatform.into_iter()` and `Repository.dirwalk_iter()` already accept byte patterns, so no new API, fallback or path matching implementation is needed. The scripted SHA-1/SHA-256 regressions fail before the fix and pass on both backends. They check matching deleted paths, nonmatching paths and a byte-bearing character class matching an untracked file. Gix checks prohibit CLI execution. Mypy and Ruff pass. Windows operand tests remain excluded because equivalent arbitrary-byte CLI arguments cannot be represented there.
Strict text decoding could make fetch/push output readers fail after Git successfully transferred refs, leaving empty result lists. Clone diagnostics, progress messages, failed-process stderr and timeout messages could likewise raise Unicode errors when names contained arbitrary bytes. Use the shared lossless helpers for output pumps, progress parsing and error re-encoding. Default subprocess text streams to surrogateescape while respecting an explicit errors policy. Keep binary streams binary and retain existing newline, timeout and process-finalization behavior. Nine regressions demonstrate the former failures and pass after the fix. They cover synchronous/streamed output, binary pumps, saved stderr, timeout and clone diagnostics, and actual SHA-1/SHA-256 fetch/push results. Reftable fixtures carry byte refs without invalid loose filenames or byte-valued refspec arguments. The connected Gix command/remote run passes 295 tests and four subtests; its remaining remote test passes with the project runner's isolated Git config instead of inherited fetch.prune=true. Ruff and mypy pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tasks
This section is for Byron only. Models continuing this PR must not add, remove, check, uncheck, rename, or reorder checkboxes here.
Everything below this line was generated by
Codex GPT-6.Created by Codex on behalf of Byron. Byron will review before this is ready to merge.
Preserve legacy filename bytes in raw diffs and native reference inputs, clarify supported APIs, and fix three concrete failures from the follow-up review of #2274. Missing tagger fields are initialized without losing tag messages, failed tree reads cannot expose a partial cache, and renamed heads retain the actual destination Git created.
The five open issues of type Task or Bug are addressed in separate commits, with Tasks first:
Tree.cacheandTreeModifierare supported in-memory editors. Correct the historical removal claim and test that serialization creates a new object without implicitly updating parents. Construction already delegates to Git/Gix.:meta public:docstring metadata for documented subclass interfaces, remove the traversal allowlist, and repair the traversal link.Repocontext manager andclose()contract, followed by strictgit.util.rmtree()deletion. Test release of object readers and native state before removing a packed repository. All instances and separately opened readers must be closed; Windows read-only files are a separate reason plainshutil.rmtree()can fail.Fixes #1771
Fixes #1848
Fixes #2030
Fixes #2064
Fixes #2101
Fixes #2274
The three review items are addressed in separate follow-up commits:
7c77a2f6: project native absence to the existing empty actor and zero date/offset. Correct CLI optional-header parsing so it retains the first message line. Gix reads remain native.88958c9d: attach the cache only after every entry has parsed and validated. Repeated reads of an invalid tree raise the same error.265a9517: validate and retain the namespace-prefixed destination corresponding to the literal short name passed to Git. The returned head points to the renamed branch, even whentopicalready exists.All three review reports reproduce with real repository fixtures; none was dismissed as a nit or hypothetical concern. Tag and tree regressions cover SHA-1/SHA-256 and empty/nonempty messages or invalid first/last entries. The rename regression preserves a different existing
topicto catch selecting the wrong branch.Validation on CPython 3.12.14/macOS, Git 2.54.0 and GixPython 0.3.0:
Git.execute()calls. Tree reads use nativels_tree; branch rename retains its documented CLI path.PASSED, with none skipped. Native tag tests retain their no-CLI guard on Windows.265a9517b20f03bafa691b1f4c7bc0c527b2a673, including Linux, macOS, Windows, Gix, legacy import checks, type/lint/docs checks and backend comparison. The earlier issue-fix head also passed all 64 checks. No new PR comments, reviews or review threads were present after final CI completed.fetch.prune=truesetting; the isolated complete runs both pass.Three separate CI commits pin legacy Python jobs to Ubuntu 24.04, separate packed-tag enumeration from byte-name resolution, and record the Windows native lookup gap (GIX-32). Windows enumeration remains covered on both backends. Successful byte-name resolution remains POSIX-only because Windows CLI arguments cannot preserve the original non-UTF-8 bytes and native Windows lookup currently falls back.
Native operations use supported GixPython APIs. Existing upstream capability gaps and compatibility bugs remain documented and open in
doc/gix-backend.md, including GIX-28's strict tag-writing gap and GIX-32's Windows byte-reference lookup gap.