Skip to content

Retire the Google Sheet as a build input; lint and build from the committed templates only - #629

Open
turbomam wants to merge 1 commit into
mainfrom
fix/602-retire-google-sheet-build-input
Open

Retire the Google Sheet as a build input; lint and build from the committed templates only#629
turbomam wants to merge 1 commit into
mainfrom
fix/602-retire-google-sheet-build-input

Conversation

@turbomam

@turbomam turbomam commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Why

Closes #602. The committed templates are the source of truth, but the machinery that could overwrite them from Google Sheets was still in place, and one piece of it ran on every push: the metpo-content-lint workflow curled the live Sheet and linted that rather than the files in the repository. The Sheet has had only two tabs since 2026-07-31; the root Makefile still computed GIDs for five that no longer exist.

What changed

Removed, across seven files (16 insertions, 295 deletions):

  • src/ontology/metpo.Makefile: the SRC_URL_* variables and sheets_config.py calls, both curl fetch rules, the save-drafts / install-drafts / diff-drafts / diff-sheets targets, and the two template deletions in clean-templates. The component rule is unchanged.
  • Makefile (root): the "Google Sheets Download Targets" block and the downloads/sheets cleanup.
  • sheets.yaml, metpo/sheets_config.py, metpo/scripts/diff_templates.py, metpo/scripts/qc_metpo_sheets.py, tests/test_diff_templates.py: deleted, with their pyproject.toml console scripts and the ruff.toml per-file ignore.
  • .github/workflows/metpo-content-lint.yml: lints src/templates/metpo_sheet.tsv and metpo-properties.tsv instead of two curl downloads.
  • docs/google_sheets_template_sync.md: same filename so links hold; now a short note that the Sheet is retired.

Validation

  • Built with the repo's template rule before and after: cmp reports the output ontology byte-identical to main.
  • make -n squeaky-clean no longer names either template; make -n components/metpo_sheet.owl still shows the ROBOT template recipe.
  • grep for docs.google.com, sheets_config, sheets.yaml, SRC_URL across Makefiles, workflows and Python: zero hits.
  • metpo-proposal-lint on the committed templates with the existing baselines: 0 errors, 0 warnings, 513 baselined.
  • pytest: 14 passed. Two test modules do not collect in this local environment (Levenshtein, vcr not installed); they fail identically on main and pass in CI.

Documentation impact

docs/google_sheets_template_sync.md rewritten. docs/metpo_id_ranges_research.md gets one updated line noting the retirement.

Scope and complexity

Mostly deletion. The one behavioural change is the workflow's input, and the lint result on that input is clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FcR9NMbL1o3svKyBqsNKpA

… the source of truth

The templates in src/templates have been the effective source of truth since the
fetch rules were made prerequisite-free, but the machinery to overwrite them
from Google Sheets was still present: curl rules in metpo.Makefile that fired
when a template was missing, a clean-templates target that deleted the templates
so they would be refetched, a download block in the root Makefile computing GIDs
for five tabs that no longer exist, two scripts and a test for diffing and QCing
the live Sheet, and a CI workflow that curled the live Sheet on every push and
linted that instead of the committed files.

All of it is removed. metpo-content-lint now lints src/templates directly; the
lint passes with the existing baseline (0 errors, 0 warnings, 513 baselined).
docs/google_sheets_template_sync.md keeps its name and now says the Sheet is
retired. The built ontology is byte-identical to main.

Closes #602

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FcR9NMbL1o3svKyBqsNKpA
Copilot AI lite review requested due to automatic review settings September 3, 2026 19:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

It removes diff-templates but leaves broken references (e.g., diff-release target and an existing CI workflow still invoking uv run diff-templates), and some updated docs still contain obsolete sheet/diff-sheets instructions.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR retires the Google Sheets–based template download/sync machinery and makes the repository-committed ROBOT templates under src/templates/ the sole inputs to linting and ontology builds, eliminating network dependency during CI/content validation.

Changes:

  • Removed Google Sheets config + download tooling (Makefile targets, sheets.yaml, and related Python CLIs/tests).
  • Updated the ontology Makefile cleanup behavior so squeaky-clean no longer deletes/re-fetches committed TSV templates.
  • Updated the metpo-content-lint workflow to lint committed templates rather than curling live Sheet tabs, and trimmed related docs.
File summaries
File Description
tests/test_diff_templates.py Removes tests for the deleted diff-templates CLI.
src/ontology/metpo.Makefile Deletes Sheet-fetch/drafts/diff-sheets targets; preserves committed TSVs during clean; keeps build rules.
sheets.yaml Removes centralized Sheet GID configuration file.
ruff.toml Removes per-file ignore for the deleted qc_metpo_sheets.py.
pyproject.toml Removes console scripts for deleted CLIs (diff-templates, qc-metpo-sheets).
metpo/sheets_config.py Removes module/script that resolved Google Sheet export URLs from sheets.yaml.
metpo/scripts/qc_metpo_sheets.py Removes QC script that validated live/downloaded Sheet-derived templates.
metpo/scripts/diff_templates.py Removes structured diff CLI for comparing templates across sources (including gsheet/git/file).
Makefile Removes Google Sheets download targets and related cleanup of downloads/sheets/.
docs/metpo_id_ranges_research.md Updates narrative to reflect Sheet retirement (but still contains obsolete “diff-sheets” guidance further down).
docs/google_sheets_template_sync.md Replaces sync guide with a short “Sheet retired” note.
.github/workflows/metpo-content-lint.yml Switches lint inputs from curled live Sheet TSVs to committed src/templates/*.tsv.
Review details

Suppressed comments (1)

src/ontology/metpo.Makefile:22

  • diff-release still invokes uv run diff-templates, but diff-templates has been removed from pyproject.toml and metpo/scripts/, so this make target will now fail. Update the target to use git diff directly (or reintroduce an equivalent CLI).
# Diff current working templates against the last tagged release
diff-release:
	@command -v uv >/dev/null 2>&1 || { echo "Error: 'uv' is required for diff-release (host-only target)."; exit 1; }
	@command -v git >/dev/null 2>&1 || { echo "Error: 'git' is required for diff-release."; exit 1; }
	@git rev-parse --is-inside-work-tree >/dev/null 2>&1 || { echo "Error: diff-release must be run from within a git work tree."; exit 1; }
  • Files reviewed: 12/12 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread pyproject.toml
Comment on lines 121 to 126
# Quality Control
audit-id-allocation = "metpo.scripts.audit_id_allocation:main"
metpo-proposal-lint = "metpo.scripts.metpo_proposal_lint:main"
generate-deprecated-template = "metpo.scripts.generate_deprecated_template:main"
diff-templates = "metpo.scripts.diff_templates:main"
qc-metpo-sheets = "metpo.scripts.qc_metpo_sheets:main"
fetch-metatraits = "metpo.scripts.fetch_metatraits:main"
resolve-metatraits-in-sheets = "metpo.scripts.resolve_metatraits_in_sheets:main"
Comment on lines +3 to +8
The Google Sheet was retired as a METPO build input on 2026-09-03.
The committed ROBOT templates in `src/templates/` are now the source of truth.
`src/ontology/metpo.Makefile` builds from those TSVs directly.
`make squeaky-clean` no longer deletes or re-fetches `metpo_sheet.tsv` or `metpo-properties.tsv`.
Edit template rows in `src/templates/` and send changes through pull requests.
Keep review, CI, and history in Git rather than syncing live Sheet tabs.
Comment on lines 277 to 281
### Sheet gid values

> **Superseded.** All Google Sheets GIDs are now centralized in `sheets.yaml` at the repo root.
> **Superseded twice.** GIDs were centralized in `sheets.yaml`, and on 2026-09-03 the Google Sheet was retired as a build input altogether (https://github.com/berkeleybop/metpo/issues/602); the committed templates in `src/templates/` are the source of truth.
> See https://github.com/berkeleybop/metpo/issues/372 and PR #373.
> The GID table that was here (dated 2026-02-19) listed stale/deprecated tabs as "Active."
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make the committed TSVs the source of truth and retire the Google Sheet as a build input

2 participants