Skip to content

Code-fence and MDX output policy #45

Description

@NivaldoFarias

Summary

pt-br.react.dev maintainers repeatedly reject otherwise-good translation PRs because content inside fenced code blocks (and some MDX components) was translated against project conventions. The runner system prompt currently instructs the model to translate code comments and user-facing string literals, which conflicts with pt-br rules.

Problem

Human review on pt-br.react.dev translation PRs shows a recurring pattern: demo JSX/UI strings and React API terms inside fences are translated when they must stay in English, or code comments are only half-translated.

Evidence (open or recently reviewed PRs):

PR File Issue
reactjs/pt-br.react.dev#1215 purity.md JSX UI strings translated (Created at: → Portuguese)
reactjs/pt-br.react.dev#1186 sunsetting-create-react-app.md JSX <h1> text translated inside a code block
reactjs/pt-br.react.dev#1218 refs.md state translated inside a // comment
reactjs/pt-br.react.dev#1213 globals.md Mixed EN/PT in comment (// OK no effect vs efeito)
reactjs/pt-br.react.dev#1182 react-19-upgrade-guide.md <ConsoleLogLine> error text translated; maintainer flagged for policy decision (match runtime English vs localized docs)

The default markdown system prompt (translation-prompt.builder.ts) says to translate code comments and string literals with user-facing text, while also saying to keep identifiers unchanged. pt-br expects:

  • Do not translate string literals that represent UI copy inside demo code fences.
  • Keep React API vocabulary in comments (state, effect, etc.) unless the glossary maps them.
  • Fully translate Portuguese comments when comments are translated (no English leftovers like effect inside PT text).

Existing validation only guards renamed function identifiers in fences (fenceFunctionIdentifiersGuard), not JSX strings, comment language, or MDX console lines.

Proposed direction

  1. Prompt and locale

    • Extend pt-br.locale.ts rules.specific with explicit fence rules aligned to pt-br wiki/glossary.
    • Adjust the global “What to Translate” section for pt-br (or when guidelines are loaded) so it does not contradict upstream conventions.
  2. Validation (retryable guards where possible)

    • Compare fenced regions against source: flag translated JSX text nodes / string literals that differ from source when they look like UI demo strings.
    • Flag React glossary tokens translated inside // comments (heuristic or glossary-driven).
    • Document a ConsoleBlock / ConsoleLogLine policy (English literal vs localized); encode in prompt and optionally a soft guard or maintainer-only note in PR body.
  3. Tests

    • Fixtures based on maintainer before/after snippets from the PRs above.

Acceptance criteria

  • pt-br-specific prompt/locale rules document JSX demo strings, comment handling, and ConsoleBlock policy.
  • At least one post-translation guard detects a known failure mode from #1215 or #1186 and supplies a retry hint.
  • Integration or unit tests cover fence policy regressions.
  • No change that increases false positives on legitimately translatable prose outside fences.

References

  • pt-br PRs: #1215, #1218, #1213, #1186, #1182
  • src/app/services/translator/llm/translation-prompt.builder.ts
  • src/app/services/translator/validation/guards/fence-function-identifiers.guard.ts

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions