Skip to content

Remove unused functionality not used by PolicyEngine - #256

Merged
MaxGhenis merged 6 commits into
masterfrom
remove-unused-functionality
Jul 22, 2025
Merged

Remove unused functionality not used by PolicyEngine#256
MaxGhenis merged 6 commits into
masterfrom
remove-unused-functionality

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Jul 22, 2025

Copy link
Copy Markdown
Collaborator

Summary

This PR removes all functionality not actively used by PolicyEngine repositories, significantly simplifying the package to focus on its core purpose.

What's kept

Based on analysis of PolicyEngine repositories, only the following are actively used:

  • MicroDataFrame class - for handling weighted microdata
  • MicroSeries class - for weighted calculations and inequality metrics (including .gini() method)
  • MicroSeriesGroupBy and MicroDataFrameGroupBy classes - used internally

What's removed

All standalone modules and their associated tests:

  • agg.py - aggregation functions
  • concat.py - concatenation utilities
  • constants.py - various constants
  • custom_taxes.py - custom tax calculations
  • income_measures.py - income measure calculations
  • inequality.py - standalone inequality functions
  • io.py - I/O utilities
  • poverty.py - standalone poverty functions
  • tax.py - tax calculations
  • ubi.py - UBI calculations
  • utils.py - utility functions
  • weighted.py - standalone weighted operations
  • _optional.py - optional dependency management (no longer needed)
  • taxcalc.py - all taxcalc functionality (previously removed)

Breaking changes

This is a major breaking change (v1.0.0) that removes most of the package's functionality. Users who depend on the removed features will need to stay on v0.x or implement the functionality themselves.

Benefits

  • Significantly reduced package size and complexity
  • Lower maintenance burden
  • Clear focus on core functionality used by PolicyEngine
  • All tests pass for the remaining functionality
  • Enables move to Python 3.13 by removing taxcalc dependency

Issues resolved

This PR closes the following issues by removing the problematic functionality:

Closes #255
Closes #252 (Move to python-13 - taxcalc dependency removed)
Closes #143 (taxcalc vs taxbrain - no longer relevant)
Closes #141 (weighting.ipynb taxcalc_helpers - taxcalc removed)
Closes #116 (Move functions out of utils.py - utils.py removed)
Closes #110 (Finish recalculate() - taxcalc.py removed)
Closes #111 (Add test for recalculate() - taxcalc.py removed)
Closes #109 (calc_df group_vars - calc_df removed)
Closes #185 (Deprecate add_weighted_quantiles - weighted.py removed)
Closes #184 (concat DataFrames - concat.py removed)
Closes #177 (Test aggregate functions - agg.py removed)
Closes #163 (groupby arg to poverty functions - poverty.py removed)
Closes #144 (year argument to fpl - poverty.py removed)
Closes #145 (Update fpl to 2020 - poverty.py removed)
Closes #146 (state argument to fpl - poverty.py removed)
Closes #107 (add_weighted_quantiles negatives - weighted.py removed)
Closes #100 (add_weighted_metrics warning - function removed)
Closes #95 (Remove read_stata_zip - io.py removed)

🤖 Generated with Claude Code

MaxGhenis and others added 6 commits July 22, 2025 12:38
This is a major breaking change that removes all functionality not actively
used by PolicyEngine repositories, significantly simplifying the package.

Kept:
- MicroDataFrame class
- MicroSeries class (including .gini() method)
- MicroSeriesGroupBy and MicroDataFrameGroupBy classes

Removed:
- All standalone modules: agg, concat, constants, custom_taxes,
  income_measures, inequality, io, poverty, tax, ubi, utils, weighted
- _optional module (no longer needed)
- Associated test files for deleted modules
- Test for concat functionality in test_microseries_dataframe.py

The package now focuses solely on providing weighted pandas-like data
structures that PolicyEngine depends on for microsimulation and
inequality calculations.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Update gini.ipynb to use MicroSeries.gini() instead of removed mdf.gini()
- Add missing newlines to __init__.py and changelog_entry.yaml

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Bump version to 1.0.0 for major breaking change
- Update package description to focus on weighted DataFrames/Series
- Rewrite README with clearer overview and usage examples
- Update badges to point to PolicyEngine organization
- Update maintainer email to max@policyengine.org

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
It's a standard Python tool that doesn't need to be explicitly listed.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Update black target-version to py313
- Update all CI workflows to use Python 3.13 as main version
- Test against Python 3.9, 3.10, 3.11, 3.12, and 3.13 in CI
- Remove taxcalc from CI install commands (already removed)
- Python 3.13 is the current bugfix version (released Oct 2023)
- Python 3.9 is the oldest version with security support

This change is enabled by the removal of taxcalc dependency.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit c15f45e into master Jul 22, 2025
8 checks passed
@nwoodruff-co
nwoodruff-co deleted the remove-unused-functionality branch December 18, 2025 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment