Auto-translation tool produces malformed output (code-fence wraps, duplicate YAML keys)

[beagandica]

Summary

The auto-translation GitHub Action (.github/workflows/auto-translation.yaml) has been producing translated files with two recurring defects that require manual cleanup before every merge:

1. Code-fence wrapping (~30% of files) — the LLM wraps the entire translated file in ```markdown ... ``` or ```yaml ... ```, which breaks Hugo's frontmatter parser.
2. Duplicate YAML keys — some translated answer-key files contain two hidden: true lines, which the YAML parser rejects.

Observed in recent bot PRs:

• [Github Actions] AI Workshop Translations - #594 #658 — 31 out of 78 translated files needed manual fence-stripping.
• [Github Actions] AI Workshop Translations - #605 #606 — 2 fence-wrap fixes plus 3 duplicate-key fixes; without the fix the hugo build broke.
• [Github Actions] AI Workshop Translations - #601 #602 — clean by luck.

This is a workflow-level bug, not a content bug. The fix belongs in the translator itself, not in each PR.

Root cause

The translator lives in the translation-tool submodule. Looking at WorkshopAutoTranslation/WorkShopTranslationV2/Program.cs, three things were driving the defects:

• The prompt did not explicitly forbid code-fence wrapping or extra frontmatter keys.
• Temperature = 1.0 for what is a deterministic task.
• MaxTokens = 1500 — several workshop pages exceed this, getting silently truncated mid-translation.
• The LLM response was written to disk with no post-processing and no validation of any kind.

There's also a path-replace substring bug — filePath.Replace("english", newFolder) would corrupt any future workshop whose name contains the substring "english" (e.g. my-english-club).

Proposed fix (PR open)

📦 NuevoFoundation/WorkshopsAutoTranslation#7 — Fix recurring AI translation output bugs (fences, dup keys, truncation)

Summary of the change:

• Rewritten prompt: explicit anti-fence rules, anti-extra-keys rules, whitelist of translatable frontmatter values (title, description, summary).
• Temperature: 1.0 → 0.2; MaxTokens: 1500 → 8192.
• Post-processor strips outer fence wrappers, BOM, and frontmatter-only fence wraps (conservative — never touches legitimate body code blocks).
• Validator (using YamlDotNet with WithDuplicateKeyChecking()) catches dup keys, key-set mismatches, empty bodies, residual fences, and FinishReason = length truncation.
• Files that fail validation are skipped (no broken output written) and the process exits non-zero so CI surfaces the problem.
• Fixed the Replace("english", ...) substring bug with a segment-aware path replace.
• 12/12 local smoke tests covering the actual PR [Github Actions] AI Workshop Translations - #605 #606 + [Github Actions] AI Workshop Translations - #594 #658 failure patterns.

Follow-up after the tool PR merges

A separate PR will be needed here in workshops to:

1. Bump the translation-tool submodule pointer from 46d1968 to the new HEAD on main.
2. Bump .github/workflows/auto-translation.yaml dotnet-version from 8.0.x to 10.0.x (the tool repo upgraded to .NET 10 in their Page - By tag pages #6 but the workshops side never caught up — this is a latent bug that would have broken the next submodule bump regardless of this fix).

Both changes must ship together.

cc @ozhang22 — flagging you since you're admin on both repos and will probably be the one reviewing the tool PR.