[DATA] `Statement.to_dataframe(matrix=True)` treats `standard_concept` as the period and returns a blank equity matrix

[synfonia-llc]

Data Quality Issue Details

Issue Type:

• Incorrect financial values (wrong numbers)
• Missing financial data (expected data not present)
• Calculation errors (formulas producing wrong results)
• Data inconsistency (different values for same metric)
• Historical data problems (changes over time)

Environment

EdgarTools Import Version: 5.55.0
Distribution Metadata: 5.54.0 (stale editable-install metadata)
Python Version: 3.14.4
Operating System: Microsoft Windows
Source Revision: origin/main at a0ff18089c844ddfe81bd21a1c7c56f4ed629e55; executed from audit worktree HEAD 70c31469dc7e6d4b386a6f8fe4ce1276d48fca51, whose edgar/** tree is identical to origin/main

Financial Data Details

Company/Ticker: Apple Inc. (AAPL)
Form Type: 10-K
Filing Date/Period: Fiscal year ended September 30, 2023; tested comparative period fiscal year ended September 24, 2022
Statement Type: Statement of Equity

Specific Metric/Concept:

• Financial line item: Equity-component matrix values, including stockholders' equity for common stock and net income for retained earnings
• XBRL concept name: us-gaap:StockholdersEquity, us-gaap:NetIncomeLoss, and us-gaap:StatementEquityComponentsAxis

Data Issue

Expected Value:

• Amount: The ordinary flat DataFrame contains 14 rows on StatementEquityComponentsAxis, all 14 with non-null filed values in 2022-09-24 (FY). Positive controls include 57,365,000,000 for us-gaap:StockholdersEquity / common stock and 99,803,000,000 for us-gaap:NetIncomeLoss / retained earnings.
• Source: The checked-in AAPL SEC XBRL fixture and the same Statement object's ordinary flat DataFrame
• Page/section reference: Consolidated Statements of Shareholders' Equity; comparative fiscal year ended September 24, 2022

Actual Value from EdgarTools:

• Amount: The matrix contains 33 equity-component cells, and all 33 are null.
• How obtained: The reproduction materializes the ordinary and matrix DataFrames from the same Statement, confirms the filed values in the flat view, and counts non-null component cells in the matrix view.

Code to reproduce:

from pathlib import Path

from edgar.xbrl import XBRL

root = Path("data/xbrl/datafiles/aapl")
xbrl = XBRL.from_files(
    instance_file=root / "aapl-20230930_htm.xml",
    schema_file=root / "aapl-20230930.xsd",
    presentation_file=root / "aapl-20230930_pre.xml",
    calculation_file=root / "aapl-20230930_cal.xml",
    definition_file=root / "aapl-20230930_def.xml",
    label_file=root / "aapl-20230930_lab.xml",
)

equity = xbrl.statements.statement_of_equity()
assert equity is not None

flat = equity.to_dataframe()
matrix = equity.to_dataframe(matrix=True)
period = "2022-09-24 (FY)"

dimensioned = flat[
    flat["dimension_axis"].str.contains(
        "StatementEquityComponentsAxis",
        na=False,
    )
]
component_columns = [
    column
    for column in matrix.columns
    if column not in {"concept", "label", "level", "abstract"}
]

print("dimensioned flat rows:", len(dimensioned))
print(
    f"non-null {period} values:",
    int(dimensioned[period].notna().sum()),
)
print(
    "non-null dimensioned standard_concept values:",
    int(dimensioned["standard_concept"].notna().sum()),
)
print(
    "matrix component cells:",
    int(matrix[component_columns].size),
)
print(
    "non-null matrix component cells:",
    int(matrix[component_columns].notna().sum().sum()),
)

for concept, member_text in (
    ("us-gaap_StockholdersEquity", "common stock"),
    ("us-gaap_NetIncomeLoss", "retained earnings"),
):
    row = dimensioned[
        (dimensioned["concept"] == concept)
        & dimensioned["dimension_member_label"].str.contains(
            member_text,
            case=False,
            na=False,
        )
    ].iloc[0]
    print(concept, member_text, row[period])

Observed output:

No exception is raised.

dimensioned flat rows: 14
non-null 2022-09-24 (FY) values: 14
non-null dimensioned standard_concept values: 0
matrix component cells: 33
non-null matrix component cells: 0
us-gaap_StockholdersEquity common stock 57365000000.0
us-gaap_NetIncomeLoss retained earnings 99803000000.0

Cross-Verification

Have you verified this issue with:

• Multiple time periods for same company
• Multiple companies with same issue
• Direct SEC filing comparison
• Other financial data sources

Affects multiple periods/companies?

• Companies tested: Apple Inc. (AAPL)
• Time periods tested: Comparative fiscal year ended September 24, 2022, in the fiscal-2023 10-K fixture
• Pattern observed: The flat view retains 14 filed values for the selected dimensional period, while the matrix view built from that same data selects a null metadata column as its period and blanks every component cell

Expected Behavior

What should happen:

Statement.to_dataframe(matrix=True) should populate an equity matrix from the same dimensional values exposed by the statement's ordinary flat DataFrame. The positive-control values shown above should appear in the appropriate component cells.

Data validation rules:

• Period detection must exclude every metadata column, including standard_concept.
• The primary period must be an actual filing period column with the expected instant or duration semantics.
• For a given concept, period, axis, and member, the matrix value must agree with the corresponding non-null flat-statement value.
• A missing standardization mapping is valid metadata and must not erase the filed value.

Additional Context

The two displayed positive controls were also traced to raw filed facts. us-gaap:StockholdersEquity lexical value 57365000000 uses context c-27, instant 2021-09-25, and StatementEquityComponentsAxis=CommonStockIncludingAdditionalPaidInCapitalMember; it is the beginning-balance value shown in the 2022-09-24 (FY) column. us-gaap:NetIncomeLoss lexical value 99803000000 uses context c-37, duration 2021-09-26 through 2022-09-24, and StatementEquityComponentsAxis=RetainedEarningsMember. This independently establishes that the blank matrix cells have filed support.

The first divergent boundary is period-column detection in Statement._pivot_to_matrix(). Its metadata_cols set does not include standard_concept, so the list comprehension treats that column as a period.

The method then selects period_cols[0] as primary_period and reads each component from member_row[primary_period]. In this reproduction, that means reading member_row["standard_concept"] instead of member_row["2022-09-24 (FY)"].

This is a matrix projection bug, not evidence that the filed facts or their standardization mappings are wrong. The ordinary DataFrame retains the filed values, and a missing standard_concept is valid metadata that should not participate in period selection.

A live search of the open and closed GitHub issue tracker should be performed immediately before submission to confirm that the behavior has not already been reported or fixed.

Impact Assessment:

• Minor (affects specific edge case)
• Moderate (affects common use cases)
• Major (affects core financial calculations)
• Critical (produces completely wrong results)

---

Data quality issues are high priority and will be verified against official SEC filings. Accuracy is fundamental to EdgarTools.