Customization techniques
Customization techniques define how your model learns from data. Each technique requires a different dataset format and has its own hyperparameters. Choose based on what data you have and what behavior you want to achieve.
SFT (Supervised Fine-Tuning) — Train on labeled prompt-response pairs. Best for teaching a model specific style, format, or domain knowledge.
DPO (Direct Preference Optimization) — Train on preferred vs rejected response pairs. Best for aligning with human preferences and avoiding undesirable outputs.
RFT (Reinforcement Fine-Tuning) — Optimize via reward signals: code-based verification (RLVR) or an LLM judge (RLAIF). Best for improving factual accuracy or subjective quality.
Multi-turn reinforcement learning (Multi-Turn Reinforcement Learning) — Train agents for multi-step agentic tasks with growing context.
Continuous customization (Continuous Customization) — Chain multiple techniques sequentially (for example, SFT → DPO → RLVR). Requires LoRA training type.
Each technique can be combined with either LoRA or FFT training type. See Training types for details on choosing between them.