Skip to content

Commit fd50653

Browse files
feat: Add support for Tinker for finetuning after data is generated with curator (#710)
* Code clean up * feat: add Tinker SDK integration for LoRA fine-tuning with checkpointing Integrate the Tinker SDK to enable real LoRA fine-tuning via the Tinker API. When a TINKER_API_KEY is provided and the tinker package is installed, training runs against the Tinker service with real forward_backward and optim_step calls. Without the SDK or key, the trainer falls back to mock mode for development and testing. Key changes: - Wire up ServiceClient, TrainingClient, and SamplingClient from tinker SDK - Create proper tinker.Datum objects with tokenized chat templates and assistant-only loss masking - Add checkpointing support (per-epoch and per-N-steps) via save_state, with checkpoint paths accessible on the Tinker dashboard - Add tinker as an optional dependency under the "finetune" extra - Add examples in examples/tinker/ and examples/poem_finetuning_example.py - Add comprehensive tests for config, types, data formatter, and trainer Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: correct multiple Tinker training integration bugs - Apply causal-LM shift (input=tokens[:-1], target=tokens[1:]) in data formatter so fine-tunes train on the correct next-token objective - Fix checkpoint resume: use create_lora_training_client_from_state() instead of load_state_with_optimizer(), and skip already-completed steps/epochs in the training loop - Pass api_key to tinker.ServiceClient() so programmatic config actually authenticates - Wire LoRA alpha, dropout, and target_modules to the training client - Implement gradient accumulation by deferring optim_step until the configured number of batches have accumulated Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use correct Tinker resume API, flush partial grad accum, fix poem example - Use create_training_client_from_state_with_optimizer() which is the documented Tinker API for resuming with optimizer state - Fix end-of-epoch resume: compute loop start bounds upfront (cookbook pattern) instead of skipping inside loops, preventing ZeroDivisionError - Flush final partial gradient accumulation window so the last microbatches are not silently dropped - Normalize loss by effective target weight instead of total token count - Fix NameError in poem_finetuning_example.py (dataset -> poem_data) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add Tinker fine-tuning section to README and deduplicate poem example - Add Fine-Tuning with Tinker section covering basic usage, checkpoint resume, and custom data formats - Add poem fine-tuning example to the examples table - Replace inline duplicate in poem example with call to convert_to_chat_format() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: restrict tinker dependency to Python >=3.11 The tinker package requires Python >=3.11 but the project supports >=3.10, causing poetry lock to fail in CI. Add a python version marker so the dependency is only resolved for compatible versions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: correct top_k=0 truthiness check and deduplicate loss_history entries - Use `is not None` instead of truthiness check for top_k so that top_k=0 is passed through correctly instead of being replaced with -1 - Skip appending to loss_history at epoch end when the last step was already logged by per-step logging, preventing duplicate entries Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 2b1904e commit fd50653

20 files changed

Lines changed: 3035 additions & 389 deletions

‎README.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@ pip install bespokelabs-curator
6262
| **Product feature extraction** | <a target="_blank" href="https://colab.research.google.com/drive/1YoA23-cBcWpaSErULzBI2bo2LPGo37GQ"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> | Finetuning a model to identify features of a product |
6363
| **Sentiment analysis** | <a href="https://colab.research.google.com/drive/1Zfl3g7POsqqYQqkzXdyhYRSAymLhZugn?usp=sharing" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a> | Aspect-based sentiment analysis of restaurant reviews and finetuning using Together.ai |
6464
| **RAFT for domain-specific RAG** | <a href="https://github.com/bespokelabsai/curator/tree/main/examples/blocks/raft" target="_blank">Code</a> | Implement Retrieval Augmented Fine-Tuning (RAFT) that processes domain-specific documents, generates questions, and prepares data for fine-tuning LLMs. |
65+
| **Poem generation & LoRA fine-tuning** | <a href="https://github.com/bespokelabsai/curator/blob/main/examples/poem_finetuning_example.py" target="_blank">Code</a> | End-to-end pipeline: curate poem data with Curator, then LoRA fine-tune with TinkerTrainer |
6566

6667
### Data Generation
6768
| **Task** | **Link(s)** | **Goal** |
@@ -296,6 +297,76 @@ See documentation:
296297
* [kluster.ai batch mode](https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/using-kluster.ai-for-batch-inference)
297298
* [Mistral batch mode](https://docs.bespokelabs.ai/bespoke-curator/how-to-guides/using-mistral-for-batch-inference)
298299

300+
## 🔧 Fine-Tuning with Tinker
301+
302+
Curator integrates with the [Tinker SDK](https://github.com/thinking-machines-lab/tinker-cookbook) so you can go from curated data to a LoRA fine-tuned model in a few lines of Python.
303+
304+
```bash
305+
pip install bespokelabs-curator tinker
306+
export TINKER_API_KEY="your-tinker-key"
307+
```
308+
309+
```python
310+
from bespokelabs.curator import TinkerTrainer, TinkerTrainerConfig
311+
312+
# Configure training
313+
config = TinkerTrainerConfig(
314+
base_model="Qwen/Qwen3-8B",
315+
epochs=3,
316+
batch_size=4,
317+
lora_config={"rank": 16, "alpha": 32, "dropout": 0.05},
318+
checkpoint_every_epoch=True,
319+
)
320+
321+
# Training data is a list of chat-format dicts (or a HuggingFace Dataset)
322+
training_data = [
323+
{"messages": [
324+
{"role": "user", "content": "What is Python?"},
325+
{"role": "assistant", "content": "Python is a programming language."},
326+
]},
327+
# ...
328+
]
329+
330+
# Train
331+
trainer = TinkerTrainer(config)
332+
result = trainer.train(training_data)
333+
print(f"Final loss: {result.final_loss:.4f}")
334+
335+
# Sample from the fine-tuned model
336+
response = trainer.sample("Explain recursion in Python")
337+
print(response)
338+
```
339+
340+
### Checkpoint Resume
341+
342+
Training can be resumed from any saved checkpoint. The trainer restores both model weights and optimizer state, then continues from where it left off — no data is replayed.
343+
344+
```python
345+
# Resume from an earlier run's checkpoint
346+
checkpoints = result.checkpoints # list of CheckpointInfo
347+
trainer = TinkerTrainer(config)
348+
trainer.load_checkpoint(checkpoints[-1])
349+
result = trainer.train(training_data) # continues from the checkpoint
350+
```
351+
352+
### Custom Data Formats
353+
354+
Subclass `TinkerTrainer` to handle non-standard data layouts:
355+
356+
```python
357+
class MyTrainer(TinkerTrainer):
358+
def format_example(self, row):
359+
return TrainingExample.from_dict_messages([
360+
{"role": "user", "content": row["question"]},
361+
{"role": "assistant", "content": row["answer"]},
362+
])
363+
364+
trainer = MyTrainer(config)
365+
result = trainer.train([{"question": "What is 2+2?", "answer": "4"}, ...])
366+
```
367+
368+
See the full [poem fine-tuning example](examples/poem_finetuning_example.py) for an end-to-end pipeline that curates data with `curator.LLM` and then fine-tunes with `TinkerTrainer`.
369+
299370
## Bespoke Curator Viewer
300371
The hosted curator viewer is a rich interface to visualize data -- and makes visually inspecting the data much easier.
301372

0 commit comments

Comments
 (0)