Skip to content

Commit 64f5c50

Browse files
author
nayan3107
committed
fix(train): complete PipelineSession support for V3 trainers (SFT/DPO/RLAIF/RLVR)
Add @runnable_by_pipeline decorator and PascalCase request serialization so TrainingStep can consume step_args from V3 fine-tuning trainers. Changes: - Add @runnable_by_pipeline decorator on train() for all 4 trainers - Build PascalCase serialized request (remove session/region, pop job name) - Fix Tags to PascalCase (JumpStart returns lowercase key/value) - Add source_code=None to BaseTrainer (required by get_code_hash) - Add consumer tests (TrainingStep.arguments validation) - Add regular session regression tests Fixes: #6163
1 parent 0316c38 commit 64f5c50

12 files changed

Lines changed: 1153 additions & 31 deletions

File tree

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{"chosen": [{"role": "user", "content": "What is ML?"}, {"role": "assistant", "content": "ML is a subset of AI that enables systems to learn from data."}], "rejected": [{"role": "user", "content": "What is ML?"}, {"role": "assistant", "content": "I don't know."}]}
2+
{"chosen": [{"role": "user", "content": "What is SageMaker?"}, {"role": "assistant", "content": "SageMaker is a managed machine learning service by AWS."}], "rejected": [{"role": "user", "content": "What is SageMaker?"}, {"role": "assistant", "content": "It's a thing."}]}
3+
{"chosen": [{"role": "user", "content": "What is fine-tuning?"}, {"role": "assistant", "content": "Fine-tuning adapts a pre-trained model to a specific task."}], "rejected": [{"role": "user", "content": "What is fine-tuning?"}, {"role": "assistant", "content": "Making things fine."}]}
4+
{"chosen": [{"role": "user", "content": "What is LoRA?"}, {"role": "assistant", "content": "LoRA is Low-Rank Adaptation for efficient model fine-tuning."}], "rejected": [{"role": "user", "content": "What is LoRA?"}, {"role": "assistant", "content": "A name."}]}
5+
{"chosen": [{"role": "user", "content": "What is RLHF?"}, {"role": "assistant", "content": "RLHF is Reinforcement Learning from Human Feedback."}], "rejected": [{"role": "user", "content": "What is RLHF?"}, {"role": "assistant", "content": "An acronym."}]}
6+
{"chosen": [{"role": "user", "content": "What is DPO?"}, {"role": "assistant", "content": "DPO is Direct Preference Optimization for alignment."}], "rejected": [{"role": "user", "content": "What is DPO?"}, {"role": "assistant", "content": "Something."}]}
7+
{"chosen": [{"role": "user", "content": "What is a transformer?"}, {"role": "assistant", "content": "A neural network architecture using self-attention mechanisms."}], "rejected": [{"role": "user", "content": "What is a transformer?"}, {"role": "assistant", "content": "A robot."}]}
8+
{"chosen": [{"role": "user", "content": "What is tokenization?"}, {"role": "assistant", "content": "Splitting text into smaller units called tokens."}], "rejected": [{"role": "user", "content": "What is tokenization?"}, {"role": "assistant", "content": "Tokens."}]}
9+
{"chosen": [{"role": "user", "content": "What is inference?"}, {"role": "assistant", "content": "Using a trained model to generate predictions on new data."}], "rejected": [{"role": "user", "content": "What is inference?"}, {"role": "assistant", "content": "Guessing."}]}
10+
{"chosen": [{"role": "user", "content": "What is a GPU?"}, {"role": "assistant", "content": "A processor for parallel computation used in ML training."}], "rejected": [{"role": "user", "content": "What is a GPU?"}, {"role": "assistant", "content": "A card."}]}
11+
{"chosen": [{"role": "user", "content": "What is backpropagation?"}, {"role": "assistant", "content": "An algorithm for computing gradients in neural networks."}], "rejected": [{"role": "user", "content": "What is backpropagation?"}, {"role": "assistant", "content": "Going back."}]}
12+
{"chosen": [{"role": "user", "content": "What is an epoch?"}, {"role": "assistant", "content": "One complete pass through the training dataset."}], "rejected": [{"role": "user", "content": "What is an epoch?"}, {"role": "assistant", "content": "A time period."}]}
13+
{"chosen": [{"role": "user", "content": "What is overfitting?"}, {"role": "assistant", "content": "When a model memorizes training data instead of learning."}], "rejected": [{"role": "user", "content": "What is overfitting?"}, {"role": "assistant", "content": "Too much fitting."}]}
14+
{"chosen": [{"role": "user", "content": "What is a loss function?"}, {"role": "assistant", "content": "Measures how far predictions are from actual values."}], "rejected": [{"role": "user", "content": "What is a loss function?"}, {"role": "assistant", "content": "Loss."}]}
15+
{"chosen": [{"role": "user", "content": "What is batch size?"}, {"role": "assistant", "content": "Number of samples processed before updating weights."}], "rejected": [{"role": "user", "content": "What is batch size?"}, {"role": "assistant", "content": "A size."}]}
16+
{"chosen": [{"role": "user", "content": "What is learning rate?"}, {"role": "assistant", "content": "Controls how much weights change during training."}], "rejected": [{"role": "user", "content": "What is learning rate?"}, {"role": "assistant", "content": "A rate."}]}
17+
{"chosen": [{"role": "user", "content": "What is regularization?"}, {"role": "assistant", "content": "Techniques to prevent overfitting in models."}], "rejected": [{"role": "user", "content": "What is regularization?"}, {"role": "assistant", "content": "Making regular."}]}
18+
{"chosen": [{"role": "user", "content": "What is an embedding?"}, {"role": "assistant", "content": "A dense vector representation of discrete data."}], "rejected": [{"role": "user", "content": "What is an embedding?"}, {"role": "assistant", "content": "A vector."}]}
19+
{"chosen": [{"role": "user", "content": "What is attention?"}, {"role": "assistant", "content": "A mechanism that weighs importance of different inputs."}], "rejected": [{"role": "user", "content": "What is attention?"}, {"role": "assistant", "content": "Focus."}]}
20+
{"chosen": [{"role": "user", "content": "What is a pipeline?"}, {"role": "assistant", "content": "An automated workflow for ML model development."}], "rejected": [{"role": "user", "content": "What is a pipeline?"}, {"role": "assistant", "content": "A pipe."}]}
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{"messages": [{"role": "user", "content": "What is ML?"}, {"role": "assistant", "content": "ML is a subset of AI that enables systems to learn from data."}]}
2+
{"messages": [{"role": "user", "content": "What is SageMaker?"}, {"role": "assistant", "content": "SageMaker is a managed machine learning service by AWS."}]}
3+
{"messages": [{"role": "user", "content": "What is fine-tuning?"}, {"role": "assistant", "content": "Fine-tuning adapts a pre-trained model to a specific task."}]}
4+
{"messages": [{"role": "user", "content": "What is LoRA?"}, {"role": "assistant", "content": "LoRA is Low-Rank Adaptation for efficient model fine-tuning."}]}
5+
{"messages": [{"role": "user", "content": "What is RLHF?"}, {"role": "assistant", "content": "RLHF is Reinforcement Learning from Human Feedback."}]}
6+
{"messages": [{"role": "user", "content": "What is DPO?"}, {"role": "assistant", "content": "DPO is Direct Preference Optimization for alignment."}]}
7+
{"messages": [{"role": "user", "content": "What is a transformer?"}, {"role": "assistant", "content": "A neural network architecture using self-attention mechanisms."}]}
8+
{"messages": [{"role": "user", "content": "What is tokenization?"}, {"role": "assistant", "content": "Splitting text into smaller units called tokens for processing."}]}
9+
{"messages": [{"role": "user", "content": "What is inference?"}, {"role": "assistant", "content": "Using a trained model to generate predictions on new data."}]}
10+
{"messages": [{"role": "user", "content": "What is a GPU?"}, {"role": "assistant", "content": "A processor designed for parallel computation used in ML training."}]}
11+
{"messages": [{"role": "user", "content": "What is backpropagation?"}, {"role": "assistant", "content": "An algorithm for computing gradients to update neural network weights."}]}
12+
{"messages": [{"role": "user", "content": "What is an epoch?"}, {"role": "assistant", "content": "One complete pass through the entire training dataset."}]}
13+
{"messages": [{"role": "user", "content": "What is overfitting?"}, {"role": "assistant", "content": "When a model memorizes training data instead of learning patterns."}]}
14+
{"messages": [{"role": "user", "content": "What is a loss function?"}, {"role": "assistant", "content": "A function that measures how far predictions are from actual values."}]}
15+
{"messages": [{"role": "user", "content": "What is batch size?"}, {"role": "assistant", "content": "The number of training samples processed before updating model weights."}]}
16+
{"messages": [{"role": "user", "content": "What is learning rate?"}, {"role": "assistant", "content": "A hyperparameter controlling how much weights change during training."}]}
17+
{"messages": [{"role": "user", "content": "What is regularization?"}, {"role": "assistant", "content": "Techniques to prevent overfitting by adding constraints to the model."}]}
18+
{"messages": [{"role": "user", "content": "What is an embedding?"}, {"role": "assistant", "content": "A dense vector representation of discrete data like words or items."}]}
19+
{"messages": [{"role": "user", "content": "What is attention?"}, {"role": "assistant", "content": "A mechanism that weighs the importance of different input elements."}]}
20+
{"messages": [{"role": "user", "content": "What is a pipeline?"}, {"role": "assistant", "content": "An automated workflow for building and deploying ML models."}]}

0 commit comments

Comments
 (0)