_prepare_returns cache key collision changes Series/DataFrame type and column metadata
Description
utils._prepare_returns() can return a different pandas container type from the input because its cache key does not distinguish a Series from an equivalent single-column DataFrame.
The result depends on call order:
- If the DataFrame is prepared first, preparing the equivalent Series returns a DataFrame.
- If the Series is prepared first, preparing the equivalent DataFrame returns a Series.
DataFrame column labels are also missing from the key, so two DataFrames with identical values but different column names share a cache entry and the second result receives the first DataFrame's columns.
This appears to be a possible root cause of the downstream HTML report error reported in #383:
Index(...) must be called with a collection of some kind, 'Strategy' was passed
Minimal reproduction
import pandas as pd
import quantstats as qs
index = pd.date_range("2024-01-01", periods=3)
series = pd.Series([0.01, -0.02, 0.03], index=index, name="Strategy")
frame = series.to_frame()
qs.utils._PREPARE_RETURNS_CACHE.clear()
prepared_frame = qs.utils._prepare_returns(frame)
prepared_series = qs.utils._prepare_returns(series)
print(type(prepared_frame).__name__)
print(type(prepared_series).__name__)
print(
qs.utils._generate_cache_key(frame, 0.0, None)
== qs.utils._generate_cache_key(series, 0.0, None)
)
Actual output
Reversing the two _prepare_returns() calls produces:
Column metadata is affected as well:
frame_a = pd.DataFrame({"A": [0.01, -0.02, 0.03]}, index=index)
frame_b = pd.DataFrame({"B": [0.01, -0.02, 0.03]}, index=index)
qs.utils._PREPARE_RETURNS_CACHE.clear()
qs.utils._prepare_returns(frame_a)
print(qs.utils._prepare_returns(frame_b).columns.tolist())
Actual output:
Expected output:
Expected behavior
_prepare_returns() should preserve the input container type and metadata independently of earlier calls:
- Series input returns a Series.
- DataFrame input returns a DataFrame.
- DataFrame column labels are preserved.
Root cause
_generate_cache_key() uses the same expression for Series and DataFrame inputs:
data_hash = pd.util.hash_pandas_object(data).sum()
For a Series and an equivalent single-column DataFrame, this produces the same value. The generated key contains only data_hash, rf, and nperiods; it omits the pandas container type and column/name metadata.
Suggested fix
Include the following in the cache key:
- Container type.
- Series name for Series inputs.
- Column labels for DataFrame inputs.
- Potentially dtype metadata if it affects the prepared result.
Regression tests should call _prepare_returns() in both orders and verify that type and column/name metadata are preserved.
Environment
- QuantStats: 0.0.81
- pandas: 2.3.3
- Python: 3.12.12
_prepare_returnscache key collision changes Series/DataFrame type and column metadataDescription
utils._prepare_returns()can return a different pandas container type from the input because its cache key does not distinguish aSeriesfrom an equivalent single-columnDataFrame.The result depends on call order:
DataFrame column labels are also missing from the key, so two DataFrames with identical values but different column names share a cache entry and the second result receives the first DataFrame's columns.
This appears to be a possible root cause of the downstream HTML report error reported in #383:
Minimal reproduction
Actual output
Reversing the two
_prepare_returns()calls produces:Column metadata is affected as well:
Actual output:
Expected output:
Expected behavior
_prepare_returns()should preserve the input container type and metadata independently of earlier calls:Root cause
_generate_cache_key()uses the same expression for Series and DataFrame inputs:For a Series and an equivalent single-column DataFrame, this produces the same value. The generated key contains only
data_hash,rf, andnperiods; it omits the pandas container type and column/name metadata.Suggested fix
Include the following in the cache key:
Regression tests should call
_prepare_returns()in both orders and verify that type and column/name metadata are preserved.Environment