Merge pull request #48 from michalmaj/test/add-coverage

test: add local coverage reporting

Author: michalmaj

README.md

@@ -29,6 +29,7 @@ By following the guide, you will learn how to:
 - configure the project in `pyproject.toml`,
 - install a package in editable mode and build distributions,
 - test code with `pytest`,
+- inspect statement and branch coverage with pytest-cov,
 - lint and format code with Ruff,
 - check type annotations with mypy,
 - run automated quality checks in GitHub Actions,
@@ -53,7 +54,8 @@ understand why each tool and file is introduced.
 9. [Common beginner mistakes](docs/08_common_mistakes.md)
 10. [Building distributions](docs/10_building_distributions.md)
 11. [Static type checking with mypy](docs/11_type_checking_with_mypy.md)
-12. [Project checklist](docs/09_checklist.md)
+12. [Test coverage with pytest-cov](docs/12_test_coverage.md)
+13. [Project checklist](docs/09_checklist.md)
 
 ## Reference material
 

docs/01_project_structure.md

@@ -116,6 +116,7 @@ In this guide, it contains:
 - dependency configuration,
 - pytest configuration,
 - mypy configuration,
+- coverage configuration,
 - Ruff configuration.
 
 Keeping these settings together makes the project easier to inspect and maintain.

docs/02_uv.md

@@ -105,7 +105,9 @@ uv sync
 uv add --group dev pytest
 uv add --group dev ruff
 uv add --group dev mypy
+uv add --group dev pytest-cov
 uv run pytest
+uv run pytest --cov=text_toolkit --cov-report=term-missing
 uv run ruff check .
 uv run ruff format .
 uv run mypy

docs/03_pyproject_toml.md

@@ -41,6 +41,7 @@ build-backend = "uv_build"
 dev = [
     "mypy>=2.3.1",
     "pytest>=9.0.3",
+    "pytest-cov>=7.1.0",
     "ruff>=0.15.12",
 ]
 
@@ -52,6 +53,13 @@ python_version = "3.12"
 files = ["src", "tests"]
 strict = true
 
+[tool.coverage.run]
+branch = true
+source = ["text_toolkit"]
+
+[tool.coverage.report]
+show_missing = true
+
 [tool.ruff]
 line-length = 88
 target-version = "py312"
@@ -68,7 +76,7 @@ select = [
 ```
 
 This describes the project, its build system, its development dependencies, and
-the settings used by pytest, mypy, and Ruff.
+the settings used by pytest, mypy, Coverage.py, and Ruff.
 
 ## The `[project]` section
 
@@ -136,8 +144,8 @@ This is intentional.
 
 The example package starts with standard library code only.
 
-Development tools such as `pytest`, mypy, and Ruff are stored separately in the
-`dev` dependency group.
+Development tools such as `pytest`, pytest-cov, mypy, and Ruff are stored
+separately in the `dev` dependency group.
 
 ## The `[build-system]` section
 
@@ -182,6 +190,7 @@ Development dependencies are needed only while developing the project.
 Examples of development dependencies:
 
 - `pytest`,
+- `pytest-cov`,
 - `mypy`,
 - `ruff`,
 - test coverage tools,
@@ -209,6 +218,12 @@ and:
 
 and:
 
+```toml
+[tool.coverage.run]
+```
+
+and:
+
 ```toml
 [tool.ruff]
 ```
@@ -231,7 +246,8 @@ This guide was built using a slower approach:
 6. configure linting and formatting,
 7. add continuous integration,
 8. add a build system and install the project,
-9. add static type checking.
+9. add static type checking,
+10. add test coverage reporting.
 
 Each step should explain one idea clearly.
 

docs/09_checklist.md

@@ -274,5 +274,5 @@ Let the project grow gradually.
 
 ## Continue
 
-- [Previous: Static Type Checking with mypy](11_type_checking_with_mypy.md)
+- [Previous: Test Coverage with pytest-cov](12_test_coverage.md)
 - [Back to README](../README.md)

docs/11_type_checking_with_mypy.md

@@ -120,5 +120,5 @@ Use pytest to check what the code actually does when it runs.
 ## Continue
 
 - [Previous: Building Distributions](10_building_distributions.md)
-- [Next: Project Checklist](09_checklist.md)
+- [Next: Test Coverage with pytest-cov](12_test_coverage.md)
 - [Back to README](../README.md)

docs/12_test_coverage.md

@@ -0,0 +1,136 @@
+# Test Coverage with pytest-cov
+
+Test coverage measures which parts of the source code run while the test suite
+is executing.
+
+It can reveal code that no test reaches, but it cannot prove that the tests make
+the right assertions.
+
+## Coverage is a diagnostic tool
+
+A coverage report helps answer questions such as:
+
+- Which source lines ran during the tests?
+- Which branches of a condition were exercised?
+- Is there production code that the test suite never reaches?
+
+It does not answer:
+
+- Are the assertions correct?
+- Are important requirements missing?
+- Are the tests readable and maintainable?
+- Is the program design good?
+
+Coverage provides evidence about test execution, not a complete quality score.
+
+## Add pytest-cov
+
+This project uses pytest-cov to collect coverage while pytest runs. It is a
+development dependency:
+
+```bash
+uv add --group dev pytest-cov
+```
+
+The package is not needed by users of `text-toolkit`.
+
+## Configure coverage
+
+Coverage.py reads its settings from `pyproject.toml`:
+
+```toml
+[tool.coverage.run]
+branch = true
+source = ["text_toolkit"]
+
+[tool.coverage.report]
+show_missing = true
+```
+
+The settings mean:
+
+- `source` limits measurement to the example package,
+- `branch` records possible paths through conditions,
+- `show_missing` lists source lines that did not run.
+
+Tests themselves are not included in the percentage. The report measures the
+package behavior that the tests exercise.
+
+## Statement and branch coverage
+
+Statement coverage checks whether executable statements ran.
+
+Branch coverage also checks different paths through decisions. For example, an
+`if` statement may execute both its true and false branches. Running the line
+with the `if` is not enough to prove that both paths were exercised.
+
+The example package has a condition in `count_words`. Tests cover both an empty
+input and text containing words, so both outcomes are represented.
+
+## Run the coverage report
+
+Run:
+
+```bash
+uv run pytest --cov=text_toolkit --cov-report=term-missing
+```
+
+pytest still runs the same tests. pytest-cov additionally records execution and
+prints a table similar to:
+
+```text
+Name                             Stmts   Miss Branch BrPart  Cover   Missing
+----------------------------------------------------------------------------
+src/text_toolkit/__init__.py         2      0      0      0   100%
+src/text_toolkit/text_stats.py      11      0      4      0   100%
+----------------------------------------------------------------------------
+TOTAL                               13      0      4      0   100%
+```
+
+The important columns are:
+
+- `Stmts`: executable statements found,
+- `Miss`: statements that did not run,
+- `Branch`: branch destinations found,
+- `BrPart`: partially covered branches,
+- `Missing`: source line numbers that did not run.
+
+The exact counts will change when the package changes.
+
+## Why there is no percentage gate
+
+This project intentionally does not configure `fail_under`.
+
+A fixed threshold can encourage tests written only to increase a number. A
+suite can reach 100% coverage while using weak assertions or missing important
+requirements.
+
+The current report happens to show 100%, but that is a property of a very small
+example package. It is not a promise that every future change must preserve a
+perfect score.
+
+When coverage decreases, inspect the missing lines and decide whether they
+represent meaningful untested behavior. Do not add a test merely to satisfy a
+percentage.
+
+## Generated data
+
+Coverage.py stores local measurement data in:
+
+```text
+.coverage
+```
+
+This file is generated and listed in `.gitignore`. Do not commit it.
+
+## Rule of thumb
+
+Use coverage to find questions worth asking about the test suite.
+
+Do not use the percentage as a substitute for those questions.
+
+## Continue
+
+- [Previous: Static Type Checking with mypy](11_type_checking_with_mypy.md)
+- [Next: Project Checklist](09_checklist.md)
+- [Back to README](../README.md)

docs/ROADMAP.md

@@ -88,6 +88,7 @@ These parts are already included in the repository.
 - [x] Add mypy as a development dependency
 - [x] Configure strict local type checking
 - [x] Run type checks against source code and tests
+- [x] Add pytest-cov and local branch coverage reporting
 
 ### Continuous integration
 
@@ -130,11 +131,17 @@ These parts are already included in the repository.
 - [x] Review commit history and self-review coverage in the existing guides
 - [x] Standardize terminology and dependency examples across the documentation
 - [x] Explain type hints and static type checking with mypy
+- [x] Explain what coverage does and does not measure
+- [x] Avoid treating coverage percentage as a quality guarantee
 
 ## Next
 
 These are good candidates for upcoming small pull requests.
 
+### Test coverage
+
+- [ ] Run coverage reporting in CI and align workflow documentation
+
 ### Repository polish
 
 - [ ] Add issue templates only if the repository starts receiving external feedback
@@ -155,12 +162,6 @@ These topics are intentionally not part of the first foundation.
 
 They may be added later as separate chapters.
 
-### Test coverage
-
-- [ ] Add coverage tooling
-- [ ] Explain what coverage does and does not mean
-- [ ] Avoid treating coverage percentage as a quality guarantee
-
 ### Pre-commit hooks
 
 - [ ] Add pre-commit hooks

docs/common_questions.md

@@ -253,17 +253,20 @@ Readers can now learn those foundations first and then continue to the
 [type-checking chapter](11_type_checking_with_mypy.md). This order keeps type
 hints and the tool that checks them from becoming another unexplained layer.
 
-## Why not add coverage yet?
+## Why was coverage added after the testing basics?
 
 Coverage tools can show which lines are executed by tests.
 
 However, coverage can be misleading if introduced too early.
 
 High coverage does not automatically mean good tests.
 
-This guide first focuses on writing clear tests that check behavior.
+This guide first focuses on writing clear tests that check behavior. It then
+introduces coverage as a diagnostic tool in the
+[coverage chapter](12_test_coverage.md).
 
-Coverage can be added later after the testing basics are clear.
+The project deliberately has no percentage gate. Readers should inspect missing
+behavior instead of treating a high number as proof of test quality.
 
 ## Why does CI use uv sync --locked --group dev?
 

docs/glossary.md

@@ -47,6 +47,20 @@ uv build --no-sources
 
 CI helps protect the `main` branch from broken changes.
 
+## Coverage
+
+Coverage measures which source statements and branches run while tests execute.
+
+This project generates a terminal report with:
+
+```bash
+uv run pytest --cov=text_toolkit --cov-report=term-missing
+```
+
+A high percentage does not prove that tests use meaningful assertions or cover
+every requirement. Coverage is evidence about execution, not a complete quality
+score.
+
 ## Commit
 
 A commit is a saved change in Git history.