feat(dw-data-access): Invoke-DwMojibakeCensus.ps1 - #98
Open
Justinvolved wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave-1 script 7 of 7 (4.30.0), stacked on the debrand PR. READ-ONLY double-encoded-UTF-8 census per table.column across every string column (or -Table subset): broken markers vs healthy typography as the partial-vs-wholesale contrast, U+FFFD context spans with every non-ASCII character escaped. Markers built from code points, so the file's own encoding cannot corrupt the needles and the repo validator does not trip on its own detector. Comparisons run COLLATE Latin1_General_BIN2 — the default CI collation case-folds the stray-Â marker onto healthy â and gives U+FFFD an undefined weight so CHARINDEX never finds it (caught live in smoke). Repair stays a separate deliberate step; U+FFFD is called out as unrecoverable in place. Consolidates the three scattered mojibake notes (visual-qa.md hard pointer, modelling-discipline.md soft pointer, the repo-file/ftfy side untouched).
Verification: validator 0 errors, parses. Live smoke: clean verdict with healthy-contrast on a repaired demo DB; synthetic damaged table detected both marker classes and produced escaped FFFD context spans; test table dropped after.