Skip to content

Commit 95a3bd7

Browse files
committed
test(data): allow cold imports in tokenizer fork safety check
The outer 30-second subprocess timeout includes Bridge model registration imports. A fresh local run spent 33.66 seconds importing and completed forked tokenization by 33.78 seconds. Use the existing 180-second import-test budget while preserving the 10-second result.get timeout that detects tokenizer deadlocks. Validation: all 41 GPT SFT dataset tests passed with the pinned MCore dataset helper built in a temporary container copy; pre-commit --all-files passed; independent diff review found no issues. Signed-off-by: Chen Cui <chcui@nvidia.com>
1 parent 3a8db15 commit 95a3bd7

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

‎tests/unit_tests/data/datasets/test_gpt_sft.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,9 @@ def encode_batch():
148148
capture_output=True,
149149
env=environment,
150150
text=True,
151-
timeout=30,
151+
# A fresh Bridge import loads all model registrations and can exceed 30s
152+
# on CI. Keep the actual forked tokenization bounded by result.get(10s).
153+
timeout=180,
152154
check=False,
153155
)
154156
assert result.returncode == 0, result.stderr

0 commit comments

Comments
 (0)