Repository navigation
Commit 9655840
test(train): make SFT recipe-override integ tests recipe-proof (#6548)
* test(train): make SFT recipe-override integ tests recipe-proof
The canaries-v3-master (sagemaker-train) job started failing after the
JumpStart Hub SFT recipe for meta-textgeneration-llama-3-2-1b-instruct
was updated (hub content 2.17.0 / template v1.0.3): training_args
nesting was removed, max_len renamed to dataset_max_len, the default
learning_rate changed, and some validation bounds were relaxed. The SFT
recipe-override integ tests pinned the old recipe shape/values, so they
broke with no SDK or test change.
Remove the 15 structure-pinning SFT tests; their merge/override/flatten/
validation behavior is already covered Hub-independently by the pinned-
fixture unit tests in tests/unit/train/test_recipe_resolver.py. Add a
schema-agnostic llama-SFT serverless test mirroring the existing RLVR and
Nova serverless tests: keys are discovered dynamically from the live Hub
spec and overrides are asserted by value, so the test survives future
recipe/version drift. Leave the already recipe-proof suites (ModelTrainer
local-recipe, evaluator, RLVR/Nova serverless) untouched.
* test(train): assert override flows into resolved recipe, not serverless HP
The scoped CodeBuild run surfaced that asserting override flow through the
serverless-filtered hyperparameters is wrong for renamed-leaf keys (e.g.
dataset_max_len -> training_config.data.max_length): the serverless path
in _apply_recipe_to_hyperparameters filters flattened recipe leaf names
against user-provided spec names, so renamed-leaf overrides are dropped.
Assert override application against the resolved recipe (the resolver's
contract) instead, keeping the test recipe-proof and leaf-name-agnostic.
The serverless drop of renamed-leaf overrides is a separate SDK issue to
be raised with the recipe owners.
---------
Co-authored-by: rsareddy0329 <rsareddy@amazon.com>1 parent 200b6f5 commit 9655840
1 file changed
Lines changed: 102 additions & 459 deletions
0 commit comments