fix things missed in the ingest migration - #2622
Draft
damonmcc wants to merge 7 commits into
Draft
Conversation
damonmcc
force-pushed
the
dm-library-facdb
branch
from
September 12, 2026 19:10
4117cf1 to
54e9c8a
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files
☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
damonmcc
force-pushed
the
dm-library-facdb
branch
from
September 13, 2026 01:53
f990bbc to
7c1f6f4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
related to #1255 and a follow-up to recent PRs/ingest runs
Fixes the four product builds that nightly QA has been failing on since the mappluto and pops ingest migrations landed. Consumers were still reading
wkb_geometryfrom datasets whose ingest archives now carrygeom, so FacDB, KPDB and CPDB died in SQL and green_fast_track died on a source test._create_facdb_spatial.sql, kpdbcombine.sqlanddcp_housing.sql, cpdbstg__dcp_mappluto_wi.sqlnow readgeomfordcp_mappluto_widcp_pops.ymlno longer renamesgeomtowkb_geometry. That rename was mine and it was pointless: the only postgres consumer builds its own geometry fromlocation/xcoordinate, while green_fast_track reads the parquet through duckdb and expectsgeomlike every other parquet source. Re-archived at 20260713, same 38 columns and 392 rows, only the column name differsstg__pluto.sqlcastsspdist3to varchar. Arrow nulls land as INTEGER in duckdb, so theLIKEagainst it failed there even though the same SQL is fine on postgresTwo unrelated fixes that were in the way:
capture_output=Trueplus a singlelogger.infoturned a wholedbt buildinto one log record, so CI showed nothing for five minutes and then a wall of text that the log viewer wrapped mid-word, with the formatter's color codes nested inside dbt'sprojects_in_geographiesbuilds one engine instead of 210. It was callingcreate_enginetwice per geography and never disposingBuilt all four on this branch: cpdb, facilities, knownprojects, green_fast_track. green_fast_track is
PASS=313 WARN=6 ERROR=0 SKIP=0across 319 nodes, the 6 warns being the stale equality fixtures already tracked in that product's issues.md.