Estimate learning from paired pre-test and post-test knowledge responses while accounting for lucky guesses and explicit "don't know" responses.
The package exposes separate models for separate estimands. Choose the model before interpreting its output.
| Function | Estimand | Main assumption |
|---|---|---|
fit_item_lca() |
Proportion who learned each item | Each item has its own latent transition distribution |
fit_person_lca() |
Shared person-level trajectory and posterior probability of learning | A person has one gg, gk, or kk trajectory across all items |
stnd_cor() |
Corrected pre, post, and gain scores | The guessing probability is supplied by the user |
group_adj() |
Guessing-adjusted group estimates | Guessing probabilities are supplied for each item |
estimate_logit_score(), cross_sectional_learning(), and
cross_sectional_learning_score() are descriptive score baselines. They do
not fit Rasch or IRT models, and the bounded score is not a calibrated
probability of learning.
install.packages("guess")
# Development version
devtools::install_github("finite-sample/guess")Pass matching data frames with one row per person and one column per item. Pre-test and post-test columns must have the same unique names. Valid response codes are:
| Code | Meaning |
|---|---|
0 or "0" |
Incorrect answer |
1 or "1" |
Correct answer |
"d" or "DK" |
Observed "don't know" response |
NA |
Observed "don't know" by default |
Use na_as = "missing" when NA records a structural failure, such as an
item that was not shown or a response lost to a technical error. Incomplete
pre/post pairs are then omitted by default. Set missing_action = "error" to
reject them instead.
fit <- fit_item_lca(pre_test, post_test, na_as = "missing")
fit <- fit_item_lca(
pre_test,
post_test,
na_as = "missing",
missing_action = "error"
)Explicit "d" and "DK" values remain observed responses under either
setting. Structural missingness is not treated as a latent response class.
fit_item_lca() is the direct entry point for the model in Cor and Sood
(2016). It fits every item separately from its paired response transitions.
library(guess)
item_sim <- simulate_lca(
n = 1500,
n_items = 4,
gg = 0.40,
gk = 0.30,
kk = 0.30,
gamma = c(0.15, 0.25, 0.35, 0.45),
seed = 123
)
item_fit <- fit_item_lca(item_sim$pre, item_sim$post)
item_fit$learning
item_fit$paramsFor binary responses, the parameter rows are:
| Parameter | Meaning |
|---|---|
gg |
Guess at both waves |
gk |
Guess before, know after; the item-level learning estimand |
kk |
Know at both waves |
gamma |
Probability of a correct response while guessing |
count_item_transitions() and fit_item_lca_counts() expose the same workflow
in two steps when you already work with transition counts.
transitions <- count_item_transitions(item_sim$pre, item_sim$post)
count_fit <- fit_item_lca_counts(transitions)fit_person_lca() jointly uses all repeated items. It estimates shared
class proportions, item-specific guessing rates, and one posterior trajectory
for each person.
person_sim <- simulate_lca(
n = 1500,
n_items = 5,
gg = 0.35,
gk = 0.35,
kk = 0.30,
gamma = 0.25,
seed = 456,
return_classes = TRUE
)
person_fit <- fit_person_lca(person_sim$pre, person_sim$post)
person_fit$class_priors
person_fit$gamma
posterior <- posterior_class_probs(person_fit)
p_learned <- posterior_learned(person_fit)This model is useful only when one common trajectory across items is
substantively defensible. It does not allow the same person to know one item,
learn another, and remain ignorant on a third. Use fit_item_lca() when the
item-specific learning proportions are the target.
The person model currently supports binary responses but not the explicit DK model.
Observed DK responses select the nine-cell model. Its latent transition
parameters are gg, gk, gd, kk, dg, dk, and dd, plus gamma.
Learning is gk + dk: learning from a guessing state plus learning from an
observed don't-know state.
dk_sim <- simulate_lca_dk(
n = 1800,
n_items = 3,
gg = 0.25,
gk = 0.15,
gd = 0.10,
kk = 0.15,
dg = 0.10,
dk = 0.10,
dd = 0.15,
gamma = 0.25,
seed = 789
)
dk_fit <- fit_item_lca(dk_sim$pre, dk_sim$post)
dk_fit$learning
dk_fit$paramsThe latent class correction relies on assumptions that should be reported with the estimate:
- People do not lose item knowledge over the interval. Know-to-guess and know-to-DK transitions are fixed to zero.
- A person who knows an item answers it correctly. The current model has no slip parameter.
- An item's guessing probability is stable across waves.
- Structural missingness is ignorable when incomplete pairs are omitted.
- The item-wise model treats items independently. The person model instead imposes one shared trajectory across items.
A correct-to-incorrect response is therefore attributed to guessing rather than knowledge loss. That restriction identifies the learning parameters and should be tested through sensitivity analysis when the interval is long or the content can be forgotten.
The binary item model is saturated, so it has no residual degrees of freedom
for a goodness-of-fit test. The DK model has one over-identifying restriction,
and assess_item_lca_fit() reports its Pearson diagnostic.
fit_stats <- assess_item_lca_fit(dk_fit, dk_sim$pre, dk_sim$post)Use held-out likelihood and perplexity to compare predictive performance.
transitions <- count_item_transitions(item_sim$pre, item_sim$post)
score_item_lca(item_fit, transitions)$perplexity
score_individual_lca(item_fit, item_sim$pre, item_sim$post)$perplexity
cv_individual_lca(item_sim$pre, item_sim$post, k = 5, seed = 321)Validate recovery under sample sizes, item counts, class proportions, and guessing rates that resemble the intended application.
recovery <- validate_recovery(
c(gg = 0.40, gk = 0.30, kk = 0.30, gamma = 0.25),
n = 500,
n_items = 4,
n_sims = 100,
seed = 654
)
recoveryThis package does not fit a longitudinal IRT model.
vignette("using_guess", package = "guess")
vignette("model_validation", package = "guess")Cor, K., and G. Sood. 2016. "Guessing and Forgetting: A Latent Class Model for Measuring Learning." Political Analysis 24(2): 226-242.
MIT