Sub-Saharan Africa Health Facility Lakehouse

A Medallion Lakehouse built with Polars, DuckDB, and Parquet. Ingests, cleans, and analyzes 98,000+ health facility records across 50 Sub-Saharan African countries.

Data Architecture

• Bronze Layer (\data/bronze/): Ingests raw XLSX records from the Humanitarian Data Exchange (HDX) / KEMRI-Wellcome Trust and outputs raw Parquet.
• Silver Layer (\data/silver/): Deduplicates records, fixes UTF-8 string encodings, normalizes coordinates, maps 172 raw facility types into 3 operational tiers (Primary, Secondary, Tertiary), and categorizes ownership into 4 sectors (Public, Private, Faith-Based, NGO).
• Gold Layer (\data/gold/): Aggregates metrics into two analytical Parquet datasets:
  • egional_facility_metrics.parquet: Subnational breakdown across 619 administrative regions.
  • ational_facility_summary.parquet: Country-level summary across 50 nations.

Project Structure

africa-health-lakehouse/
├── data/                       # Ignored from Git (Parquet storage)
├── src/
│   ├── ingest_health_data.py   # Downloads raw data and converts to Bronze Parquet
│   ├── inspect_bronze.py       # Audits raw Bronze schema and row counts
│   ├── inspect_facility_types.py # Profiles distinct facility strings in raw data
│   ├── transform_lakehouse.py  # Runs Silver cleaning and Gold dataset aggregation
│   ├── read_silver.py          # Previews clean Silver layer schemas and tier distributions
│   ├── read_gold.py            # Queries Gold Parquet datasets and surface core findings
│   ├── country_facilities.py   # Extracts single-country subnational metrics
│   └── ownership_breakdown.py  # Analyzes public vs non-state reliance metrics
├── .gitignore
└── README.md

Core Findings

Core Findings

1. National Facility Volume: Infrastructure volume varies widely across Sub-Saharan Africa. Nigeria leads total facility count (20,733), followed by DRC (14,573), Tanzania (6,304), Kenya (6,144), Ethiopia (5,214), and South Africa (4,303).
2. Primary Care Concentration: Frontline facilities (clinics, dispensaries, health posts) make up over 90% of all health infrastructure in major nations like Nigeria (95.5%), DRC (88.5%), and Tanzania (96.5%).
3. Non-State Reliance Patterns: Heavy non-state reliance (>50% private, NGO, or faith-based) occurs in conflict zones (Bakool, Somalia at 97.1%) and commercial urban centers (Nairobi, Kenya at 62.6%).
4. Tertiary Care Scarcity: Specialized tertiary and teaching hospitals account for less than 1% of total infrastructure across almost all 50 countries (Nigeria: 0.1%, Kenya: 0.1%, DRC: 0.0%), pointing to a severe shortage of advanced regional care.
5. Subnational Sector Disparities: Non-state reliance isn't uniform within countries. In Kenya, urban Nairobi reaches 62.6% non-state reliance, whereas rural administrative regions remain almost exclusively dependent on state-run public health centers.
6. Public Sector Baseline Dominance: Nations like Cameroon (100.0%) and Madagascar (100.0%) report fully public ownership profiles in the underlying datasets, signaling either strong public-driven health delivery or strict national reporting conventions.
7. Spatial Coverage Gaps: Combining lat/long coordinates with administrative unit boundaries exposes subnational regions where high facility counts are densely packed into urban capital corridors, leaving remote boundary districts under-resourced.
8. Data Quality Issues: Raw dataset omissions leave zero primary care records for Sudan regions due to source mapping limits.

Execution

1. Install dependencies:

   pip install polars duckdb httpx python-calamine pandas
   

2. Run pipeline:

   python src/ingest_health_data.py
   python src/transform_lakehouse.py
   python src/read_gold.py
   \