AbhilekhMeda/qwen3-1.7b-multiturn-ultrachat

Qwen3-1.7B Multi-turn SFT on UltraChat

This repo stores the training script and will receive checkpoints from an SFT run of Qwen/Qwen3-1.7B on HuggingFaceH4/ultrachat_200k.

Training objective

Improve context maintenance across several exchanges by fine-tuning on a multi-turn conversational dataset while computing loss only on assistant turns.

Dataset choice

Selected dataset: HuggingFaceH4/ultrachat_200k

Why:

• verified multi-turn messages structure with role/content
• widely used conversational SFT dataset
• directly compatible with TRL SFTTrainer

Training details

• Trainer: trl.SFTTrainer
• Dataset format: conversational language modeling (messages)
• Loss masking: assistant_only_loss=True
• Packing: False to preserve dialogue boundaries
• Chat template handling: rely on TRL's patched Qwen3 training template for assistant-token masking

Key correctness point

For Qwen3, current TRL docs state that when assistant_only_loss=True is enabled, TRL automatically swaps in a patched Qwen3 training chat template containing {% generation %} / {% endgeneration %} markers so only assistant tokens contribute to the loss.

References

• TRL SFT docs: https://huggingface.co/docs/trl/sft_trainer
• TRL chat templates docs: https://huggingface.co/docs/trl/chat_templates
• Dataset: https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k