fix(data): honor explicit prompt column names

[yaoyu-33]

Summary

Direct Hugging Face SFT lets callers explicitly select raw prompt and completion columns. A valid paired-text source that selects messages as an ordinary string prompt column currently fails during normalization, even though the configuration validates and the row does not contain a structured conversation.

Root cause

normalize_sft_example() inferred chat schema from conventional key names before consuming the selected prompt/completion columns. Therefore any non-None messages, conversation, or conversations value was treated as structured chat, including a string field explicitly selected as prompt text.

Fix

Exclude explicitly selected prompt/completion columns from conventional chat-key inference. Other populated chat keys still trigger the existing ambiguity/rejection paths, and genuine structured conversations remain rejected by prompt/completion preprocessing.

Validation

• Before the fix:
  uv run --no-sync python -m pytest -q --confcutdir=tests/unit_tests/data tests/unit_tests/data/test_sft_processing.py -k 'explicit_chat_named_text_column or rejects_structured_conversation'
  — 1 failed, 1 passed with the expected structured-conversation ValueError.
• After the fix, the same command — 2 passed.
• Full focused module:
  uv run --no-sync python -m pytest -q --confcutdir=tests/unit_tests/data tests/unit_tests/data/test_sft_processing.py
  — 21 passed.
• Adjacent native adapter coverage:
  uv run --no-sync python -m pytest -q --confcutdir=tests/unit_tests/data/sources tests/unit_tests/data/sources/test_hf_adapters.py
  — 21 passed.
• git diff --check — passed.
• UV_NO_SYNC=1 uv run pre-commit run --all-files — passed.

Scope

This changes only schema classification when a conventional chat key is explicitly selected as a paired-text column. It does not change public APIs, chat preprocessing, native adapter output, multimodal handling, dependencies, or CI configuration. No GPU or distributed validation is needed for this CPU-only normalization contract.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[yaoyu-33]

/ok to test fc69fb0

[claude]

Light review — LGTM

Clean, well-scoped bug fix. When PromptCompletionSFTPreprocessingConfig sets prompt_column/completion_column to a reserved conversation key (messages/conversation/conversations), the old has_conversation check would misclassify the row as a structured chat and raise "Prompt-completion preprocessing requires paired text columns...". Excluding the configured columns from conversation_keys fixes this, and since has_conversation feeds both the mutual-exclusion check and the prompt-completion branch, the fix stays consistent across both.

The added unit test (test_prompt_completion_honors_explicit_chat_named_text_column) exercises the fixed path directly.

Minor, non-blocking: consider adding a negative test where prompt_column="messages" AND the row also carries a genuine second conversation key (e.g. conversations) — that combination should still raise, guarding against the exclusion set being over-widened in a future edit.

Suggested test cases

• test_prompt_completion_honors_explicit_chat_named_text_column (added)
• test_prompt_completion_rejects_structured_conversation (verify still passes with the reduced key set)
• New case: prompt_column="messages" with the row also containing a non-None conversations/conversation key still raises the exactly-one-schema error

No perf tests impacted.