Weights silently lost on pandas methods that aren't hand-overridden (_metadata/_constructor not wired)

[vahid-ahmadi]

MicroDataFrame/MicroSeries never declare _metadata = ["weights"], _constructor, or _constructor_sliced. Weight survival across a pandas method is therefore accidental — it depends on whether the internal path happens to route through our __init__ — rather than guaranteed by the pandas subclassing contract. Methods that were explicitly overridden (drop, merge, reset_index, copy, astype, loc/iloc, __getitem__) are fine; everything else is a coin flip, and when it loses, the user gets an unweighted number with no warning.

Reproduction (true weighted sum(x) is 1*1 + 2*2 + 3*3 = 14), pandas 3.0.5:

import pandas as pd
from microdf import MicroDataFrame

d = MicroDataFrame(pd.DataFrame({"x": [1, 2, 3]}), weights=[1, 2, 3])

d.sort_values("x", ascending=True)    # MicroDataFrame, sum_x = 14.0   (no-op reorder, survives)
d.sort_values("x", ascending=False)   # DataFrame,      sum_x = 6      WRONG, silent
d.fillna(0)                           # DataFrame,      sum_x = 6      WRONG, silent
d.sample(frac=1)                      # DataFrame,      sum_x = 6      WRONG, silent
pd.concat([d, d])                     # DataFrame                      weights gone

The ascending=True vs ascending=False split is the clearest evidence that this is not a per-method gap but a missing contract: the same method call preserves or destroys weights depending on whether pandas actually had to reorder the block.

Pickle is the same root cause. weights is a plain instance attribute and is not part of the pickle state:

import pickle
from microdf import MicroSeries
s = pickle.loads(pickle.dumps(MicroSeries([1, 2], weights=[3, 4])))
s.sum()   # AttributeError: 'MicroSeries' object has no attribute 'weights'

This also breaks to_pickle/read_pickle round-trips and anything that ships a MicroSeries across a process boundary (joblib, multiprocessing, Dask).

Suggested fix

• Declare _metadata = ["weights"] on both classes and propagate through __finalize__ so pandas carries weights across every operation that returns a new object.
• Define _constructor / _constructor_sliced so intermediate objects come back as MicroDataFrame/MicroSeries.
• Add __reduce__/__setstate__ (or rely on _metadata, which pandas includes in pickle state) so weights survive serialisation.
• Where weights genuinely cannot be propagated (row-count-changing ops), warn rather than silently degrade — consistent with the existing .values/.to_numpy()/.cov() warnings.

Note this is a broader restatement of the closed #242; the _metadata framing plus the sort_values(ascending=False)/pickle repros are new.

Found while reviewing the repo against pandas 3.0.5 / Python 3.13.

[vahid-ahmadi]

PR #313 covers the pickle half of this: declaring _metadata = ["weights"] makes weights survive pickle/to_pickle/deepcopy.

Two notes for whoever takes the rest:

1. _metadata alone does not fix the method returns. sort_values(ascending=False), fillna and sample come back as plain DataFrame/Series, so __finalize__ never runs on them. That needs _constructor/_constructor_sliced.
2. There's a design obstacle in the way. MicroDataFrame's weighted aggregations are installed as per-instance closures by override_df_functions, called from __init__. Declaring _constructor means every pandas-internal construction runs __init__ and re-installs ~30 closures, so it needs a performance pass — and it's what made the pickle bug two bugs deep (weights restored, aggregations not; sum() silently returned the unweighted 6 instead of 14, which Preserve weights through pickling and deepcopy #313 fixes with __setstate__). Moving those overrides to real methods on the class would remove both problems.

Also worth deciding how weights should be reindexed for row-changing operations before wiring __finalize__ — reindex-and-fill-1.0 is one choice, but dropping to a warning may be safer for ops that add rows.