Skip to content

Refuse a survey_pop with unknown bands instead of under-counting - #381

Draft
sbfnk-bot wants to merge 2 commits into
advance-deprecationsfrom
reject-na-population
Draft

Refuse a survey_pop with unknown bands instead of under-counting#381
sbfnk-bot wants to merge 2 commits into
advance-deprecationsfrom
reject-na-population

Conversation

@sbfnk-bot

Copy link
Copy Markdown
Collaborator

Fixes #379.

add_survey_upper_age_limit() drops rows whose population is NA before the population is aggregated. The age group containing them is then reported and weighted as though those people did not exist, and nothing says so:

uk    <- polymod[country == "United Kingdom"]
gappy <- data.frame(lower.age.limit = 0:90, population = rep(1e5, 91))
gappy$population[gappy$lower.age.limit %in% 30:40] <- NA_real_   # 11 unknown bands

contact_matrix(uk, age_limits = c(0, 20, 60), symmetric = TRUE,
               survey_pop = gappy, return_demography = TRUE)
#> $demography [20,60) population 2.9e6   (the true figure is 4.0e6)
#> $matrix[1, 2] 4.3584                   (13.5% below the correct 5.0394)

The only warning raised is "Large differences in the size of the sub-populations…", which is a consequence of the under-count and sends the reader to their age limits rather than to their population data.

NA in a population table means unknown, not nobody. Treating it as nobody is the one reading that yields a plausible number rather than a complaint, which is why this survives unnoticed.

contact_matrix() now rejects it:

Population data must be known across the age groups asked for.
i `survey_pop` has no population for ages 30, 31, 32, 33, 34, 35 and 5 more.
i Those rows are dropped before the population is aggregated, which would leave
  the age group holding them short by however many people they stand for.
i Supply a population for those ages, or ask for age groups that do not cover them.

This is the behaviour the compute_matrix() pipeline already had — joint_population_vector() requires a row per grouping level, so the same population errors there with "survey_pop is missing population entries for some grouping". The two paths now agree.

It joins the population rules #374 added — fine enough, reaching high enough, starting no higher than the youngest group — as a fourth: known across the range in use.

Scope

Only bands at or above the youngest age group asked for are checked. Bands below it are dropped whether or not they hold a population, so an unknown one there changes nothing and is still accepted.

The all-NA and zero-row cases keep their existing, more specific message ("survey_pop holds no population data"), which fires earlier.

Verification

The three added tests fail on advance-deprecations with this commit reverted (2 failures) and pass here, so they pin the change rather than the wording. Pluralisation checked at one, two and eleven unknown bands. 507 tests pass; lint_package() on an installed copy reports no lints.

Based on advance-deprecations (#374), which is where the sibling population rules live; it will retarget to main when that merges. Parallel to #380 rather than stacked on it — the two touch different parts of contact_matrix.R.

…ed for

Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
@sbfnk-bot
sbfnk-bot requested a review from sbfnk September 4, 2026 10:09
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant