Skip to content

Commit 9655840

Browse files
rsareddy0329rsareddy0329
andauthored
test(train): make SFT recipe-override integ tests recipe-proof (#6548)
* test(train): make SFT recipe-override integ tests recipe-proof The canaries-v3-master (sagemaker-train) job started failing after the JumpStart Hub SFT recipe for meta-textgeneration-llama-3-2-1b-instruct was updated (hub content 2.17.0 / template v1.0.3): training_args nesting was removed, max_len renamed to dataset_max_len, the default learning_rate changed, and some validation bounds were relaxed. The SFT recipe-override integ tests pinned the old recipe shape/values, so they broke with no SDK or test change. Remove the 15 structure-pinning SFT tests; their merge/override/flatten/ validation behavior is already covered Hub-independently by the pinned- fixture unit tests in tests/unit/train/test_recipe_resolver.py. Add a schema-agnostic llama-SFT serverless test mirroring the existing RLVR and Nova serverless tests: keys are discovered dynamically from the live Hub spec and overrides are asserted by value, so the test survives future recipe/version drift. Leave the already recipe-proof suites (ModelTrainer local-recipe, evaluator, RLVR/Nova serverless) untouched. * test(train): assert override flows into resolved recipe, not serverless HP The scoped CodeBuild run surfaced that asserting override flow through the serverless-filtered hyperparameters is wrong for renamed-leaf keys (e.g. dataset_max_len -> training_config.data.max_length): the serverless path in _apply_recipe_to_hyperparameters filters flattened recipe leaf names against user-provided spec names, so renamed-leaf overrides are dropped. Assert override application against the resolved recipe (the resolver's contract) instead, keeping the test recipe-proof and leaf-name-agnostic. The serverless drop of renamed-leaf overrides is a separate SDK issue to be raised with the recipe owners. --------- Co-authored-by: rsareddy0329 <rsareddy@amazon.com>
1 parent 200b6f5 commit 9655840

1 file changed

Lines changed: 102 additions & 459 deletions

File tree

0 commit comments

Comments
 (0)