Confirmatory Factor Analysis of a PISA Reading Self-Assessment Scale

This repository contains a course project for the Statistics course at HSE University. The project examines the internal structure and reliability of a six-item reading self-assessment scale from PISA 2018 using Confirmatory Factor Analysis (CFA).

The analysis compares alternative factor structures, evaluates model fit and reliability, examines modification indices, and identifies the most statistically and theoretically appropriate final model.

Project Overview

The study uses six PISA items measuring students' perceived reading competence and reading difficulties:

• ST161Q01HA — I am a good reader
• ST161Q02HA — I am able to understand difficult texts
• ST161Q03HA — I read fluently
• ST161Q06HA — I have always had difficulty with reading
• ST161Q07HA — I have to read a text several times before completely understanding it
• ST161Q08HA — I find it difficult to answer questions about a text

All items use a four-point Likert-type response scale ranging from 1 (Strongly disagree) to 4 (Strongly agree).

The analytical sample consists of 1,835 students, after listwise deletion of missing observations from the original sample of 2,016 students.

Dataset source

The dataset is part of the official OECD — PISA 2018 Database, which provides the PISA 2018 data files together with questionnaires, codebooks, and other documentation. The variable definitions and questionnaire item wording were taken from the PISA 2018 Student Questionnaire Codebook.

Packages

The analysis was conducted in R 4.5.2 and uses the following packages:

• lavaan — structural equation modeling and CFA
• semTools — reliability estimation
• semPlot — model visualization
• lavaanPlot — CFA path diagrams
• rio — data import
• car — recoding
• dplyr — data manipulation

Analysis

The CFA was conducted in R 4.5.2 using the lavaan package. Because the indicators are ordinal, the models were estimated using WLSMV/DWLS with ordered indicators. Latent variable variances were fixed to 1 using std.lv = TRUE.

Five models were considered:

Model 1 — one-factor model with all six original items.
Model 2 — one-factor model with the three negatively worded items reverse-coded.
Model 3 — one-factor model with five items, excluding the poorly functioning fluency item.
Model 4 — one-factor model with five items and a correlated residual between Items 1 and 2.
Model 5 — two-factor model separating perceived reading competence from perceived reading difficulty.

Model fit was assessed using CFI, TLI, RMSEA, and SRMR, following the criteria proposed by Hu and Bentler (1999). Internal consistency was evaluated using McDonald's omega.

Model Selection

The initial one-factor model showed poor fit and very low reliability. Reverse-coding the negatively worded items substantially improved reliability but did not improve global model fit. Removing the weakly performing fluency item produced acceptable reliability, but model fit remained inadequate.

Modification indices indicated substantial residual covariance between Items 1 and 2. Adding this residual covariance resulted in excellent model fit:

• CFI = .993
• TLI = .982
• RMSEA = .056
• SRMR = .020

A theoretically interpretable two-factor model produced the same excellent global fit while avoiding correlated residuals. The two factors represent:

• Perceived reading competence — Items 1 and 2
• Perceived reading difficulty — reverse-coded Items 6, 7, and 8

The two-factor model was selected as the final model. However, its reliability estimates remained below the conventional .70 threshold for both factors, and the first factor consists of only two indicators. These limitations suggest that further item development would be desirable.

Repository Contents

CFA report.docx
CFA task description.docx
CFA.Rmd
PISA_MR.sav
STUDENTS questionnaire PISA 2018 main.pdf
LICENSE
README.md

CFA report.docx: final report presenting the CFA analysis, model development and selection, model fit, factor loadings, reliability estimates, model comparisons, and interpretation of the final model.
CFA task description.docx: description of the course assignment.
CFA.Rmd: R Markdown source file containing the complete analysis code. It includes data preparation, descriptive statistics, reverse-coding, specification and estimation of five CFA models, assessment of model fit and reliability, modification-index analysis, model comparisons, visualizations, and generation of summary tables.
PISA_MR.sav: the PISA data used for the CFA.
STUDENTS questionnaire PISA 2018 main.pdf: the codebook and reference for interpreting the variables.

References

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

McDonald, R. P. (1999). Test Theory: A Unified Treatment. Lawrence Erlbaum Associates.

Organisation for Economic Co-operation and Development (OECD). PISA 2018 Technical Report. OECD Publishing. https://www.oecd.org/en/data/datasets/pisa-2018-database.html

Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02