Skip to content

Diff custom objects by attributes, matching DeepDiff's object rules - #95

Open
ksco92 wants to merge 5 commits into
mainfrom
feature/66-custom-objects
Open

Diff custom objects by attributes, matching DeepDiff's object rules#95
ksco92 wants to merge 5 commits into
mainfrom
feature/66-custom-objects

Conversation

@ksco92

@ksco92 ksco92 commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Closes #66. Follow-up for the object-view gaps: #99.

Custom objects were the last value type the README's value-types list excluded (onix raised TypeError). They now diff by their attributes, matching DeepDiff's _diff_objattribute_added/attribute_removed with root.attr paths, type_changes between two classes that are not the same — inside lists, dicts, and under ignore_order.

DeepDiff 9.1.0 source, verified

DeepDiff uses three attribute views of one object: helper.detailed__dict__ (instance __dict__ plus dir()-derived properties and class attributes, via getattr) for diffing; deephash._prep_obj (raw __dict__/slots, class-tagged) for ignore_order; and serialization.json_convertor_default (public properties, else public __dict__, else TypeError) for to_json(). Its _get_item_length counts an object's __dict__ keys, not its values. This PR implements the diffing view; the hashing and serialization views and to_dict()'s original-instance return are the split follow-up (#99, see Skipped).

Accept-list gating

A fallback arm that reshapes whatever reaches it is gated by an explicit accept-list checked against DeepDiff's own _diff dispatch ladder, so no type is silently reshaped. Cross-checked against the ladder three ways:

  • Same predicates, concrete not ABC: numbers are refused by DeepDiff's concrete tuple (int, float, complex, Decimal, Fraction) — not the numbers.Number ABC — so a class registered with or subclassing numbers.Number reaches attribute diffing (matching DeepDiff), while complex/Decimal/Fraction are refused. Iterables use the collections.abc.Iterable ABC, which is exactly what DeepDiff's ladder tests.
  • Same order: the Enum acceptance sits after the number and iterable refusals, matching the ladder (Iterable precedes the Enum elif), so an iterable-mixin type is not mis-accepted as an Enum.
  • Same domain: every _diff_obj enumeration strategy is accepted — __dict__, __slots__, and the getmembers strategy for a C type with neither (e.g. re.Pattern, diffed by root.pattern). Only an empty extraction (a bare object()) is refused, the {}-for-unequal hazard.

Accepted → attribute diff: a user-defined class instance, and an Enum member (matches _diff_enum via name/value).
Refused → typed, path-naming TypeError: complex/Decimal/Fraction, any iterable (bytes/bytearray/memoryview/range/generators/__iter__ types, and a custom non-dict Mapping — a deliberate over-refusal, documented), uuid, ipaddress, class objects, modules, a bare object(). Invariant (rule + tests): nothing that raised before reports {} for two Python-unequal values; an empty extraction and a self-referential object each end-to-end (the latter MaxDepthError).

Class identity

DeepDiff's rule is type(t1) is not type(t2) — a comparison of the type objects. onix keys identity on format!("{module}\0{qualname}\0{id(type)}"): the type object's address (id) is the discriminator, so two classes created under one qualified name (a class defined in a function body, type("E", (), {}) twice, make_dataclass) are distinct and report type_changes; both type objects are alive for the whole diff (each instance holds a reference), making the address sound within a run. Module and qualname are folded in, NUL-joined (impossible in either), which also removes the earlier dot-separator collision. The rendered old_type/new_type stays the bare __name__. Routed through one same_class shared by the diff dispatch and Value equality; a dict subclass versus a same-named object is a type_changes too (name plus kind).

Robustness

__dict__ read via .copy() at every dict and object level, so a @property getter that inserts into a dict being converted (even one levels up) cannot panic pyo3's live-dict iterator — it works over a snapshot, as DeepDiff's _diff_dict does over copied key sets. A getter raising anything but AttributeError (a ValueError, or a BaseException) propagates at the attribute's path; an AttributeError skips that attribute. A __dict__ that is not a mapping, and a class object (mappingproxy), get the typed path-naming error.

Goldens, fuzz, mutation

  • New $object golden tag (both readers); 13 hand-designed cases from live DeepDiff, byte-identical, plus test_conversions.py cases (slots-only, dict+slots, dataclass with default_factory, name-mangled _Cls__x, property+class-attribute, Enum, dict-subclass-vs-object, cross-module, two-type("E"), local classes, re.Pattern, registered-number, property raising ValueError/AttributeError, dict-mutating property at one and two levels) asserted against live DeepDiff.
  • Differential fuzz over generated plain classes (bare-scalar attributes), 300 seeds × {ordered, ignore_order} = 600 diffs, zero divergences. (Property/class-attribute fuzz widening waits on Match DeepDiff's three object views: storage-view hashing, public-view to_json, original instances from to_dict #99, since such objects diverge on exactly the hashing/serialization views Match DeepDiff's three object views: storage-view hashing, public-view to_json, original instances from to_dict #99 covers.)
  • cargo mutants scoped to the changed onix-core functions: all caught, zero survivors. The mutation run does not cover onix-py; its enumeration strategies and the accept-list are pinned by the test_conversions.py cases above.

Benchmarks

Release, final head vs 0.11.1, hyperfine (the CLI path is JSON-only, so it measures the dict/scalar path the object arm shares; the object-arm hash writes never execute on these fixtures). No row regresses beyond noise:

Fixture 0.11.1 this branch
flat_dict_100k 55.1 ms ± 1.1 55.9 ms ± 1.0
api_payloads 433.9 ms ± 4.0 437.1 ms ± 4.0
nested_uniform 413.4 ms ± 5.6 412.2 ms ± 4.9
ignore_order_10k 74.0 ms ± 3.3 72.6 ms ± 3.3

ignore_order_10k (all-numeric) is unchanged within noise; the object-arm changes do not touch it.

Divergence triage

  • Real semantics matched: attribute enumeration/order, paths, categories, type_changes by type-object identity and kind, Enum, re.Pattern, the accept-list refusals.
  • Nuance (documented): an AttributeError-raising property skips that attribute vs DeepDiff's whole-object unprocessed; a custom non-dict Mapping, and an object with an unsupported-typed attribute (a direct numbers.Number/ABCMeta subclass's _abc_impl), are over-refused.
  • Architectural divergence (documented): a recursive object raises MaxDepthError where DeepDiff's identity-based cycle detection returns {} — onix holds a value model, not object identity; deterministic, not a crash.

Skipped — tracked in #99

Three behaviors need attribute views onix does not hold, split to #99 (owner decision), documented in README Known limitations and tests/golden/README.md (which carries the full trigger list):

  • ignore_order hashing (semantic): [Prop(1), Prop(2)] vs [Prop(2), Prop(3)] → DeepDiff pairs by the storage view (iterable_item_added/removed); onix by the detailed view (values_changed root[0]._x). Needs per-attribute storage-vs-computed provenance.
  • to_json() whole-object value (semantic): type_changes(WithProp, WithProp2) → DeepDiff {'p': 10} (public props); onix {'kls': 'k', 'p': 10, 'x': 1} (full view). Needs a public-view render.
  • to_dict() (the issue's requirement): DeepDiff returns the original instances; onix returns attribute dicts (it holds no PyObject after conversion). Needs a per-diff original-instance side table.

The to_dict() fuzz comparison stays scoped off the object batch for this reason; #99 re-enables it.

Version

0.12.0 (adds a capability), rebased onto 0.11.1.

Custom objects now diff by their attributes instead of raising TypeError,
matching DeepDiff's _diff_obj: attribute_added/attribute_removed with
root.attr paths, type_changes between two different classes, and the same
enumeration DeepDiff uses (instance __dict__ plus non-callable, non-dunder
names from dir() up the MRO, or slots for a slots-only class). Objects work
inside lists, dicts, and under ignore_order, where they hash and pair by a
class-tagged content key so a custom object never matches a plain dict or an
instance of another class.

Reuses the existing Object value with an ObjectKind marker, a new
Attribute path segment, and attribute_added/attribute_removed report
categories. Adds an $object golden tag (both readers), 13 hand-designed
goldens, a differential-fuzz batch over generated classes, and depth-guard
and recursive-object tests.

Bumps to 0.12.0.
…on and mappingproxy fixes

Gate the object fallback by an accept-list derived from DeepDiff's _diff
dispatch ladder: only a genuine user-defined class (and an Enum member, which
matches _diff_enum via name/value) is diffed by attributes. Every type DeepDiff
routes to a dedicated handler onix lacks -- a number (complex/Decimal/Fraction),
any iterable (bytes/bytearray/memoryview/range/generator/__iter__), uuid,
ipaddress, a class object, a module, or a bare attribute-less object -- is
refused with a typed, path-naming TypeError rather than reshaped into an object
that would silently report {} for unequal values.

Class identity is now the qualified __module__+__qualname__ plus kind, not the
bare __name__: two same-named classes from different modules, and a dict
subclass versus a same-named object, are type_changes. Routed through one
same_class definition shared by dispatch and Value equality.

Read __dict__ via .copy() (handles a snapshot against property mutation);
propagate any non-AttributeError from a property getter at its path; refuse
class objects before their mappingproxy is read. Drop the unused _path
parameter; build the fuzz classes from one shared init.
@ksco92
ksco92 force-pushed the feature/66-custom-objects branch from c421e35 to 34e71a4 Compare September 6, 2026 16:03
Enum members are diffed by their name/value (matching DeepDiff), not refused,
so drop Enum from the raise-list and state the exception. to_dict() returns an
attribute dict for every custom object, not only those with a property or class
attribute, so state it unconditionally.
Drop the .claude rules-path citation from is_diffable_object's doc and from
tests/golden/README.md's divergence bullet (the accept-list stands on 'derived
from DeepDiff's _diff ladder'), and replace the '#66 item N' ordinals in two
test section comments with a plain '#66' reference.
…y, dict snapshot

Gate the object fallback by the concrete predicates and order of DeepDiff's
_diff ladder (concrete number tuple, not the numbers.Number ABC; the Iterable
ABC; Enum after both; the getmembers strategy for C types like re.Pattern;
refuse only an empty extraction). Key class identity on the type object's
address (id(type)) plus module/qualname NUL-joined, so classes sharing a
qualified name but not a type object are type_changes, matching
'type(t1) is not type(t2)'. Iterate a dict snapshot so a property getter that
mutates a dict under conversion cannot panic pyo3's iterator. A non-mapping
__dict__ gets the typed path error. Share the getattr helper, inline the
distance wrapper, delegate ccustom to ccustom_id, and correct the docs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Diff custom objects by attributes, matching DeepDiff's object rules

1 participant