Fine-Tuning LLMs
Adapt pre-trained models to your domain with custom data.
Last updated: July 2026
You start from a pre-trained model that already understands language broadly.
Fine tuning means taking a pre-trained AI model (one already trained on huge amounts of general data) and continuing its training on a smaller, task-specific dataset so it performs better for one particular use case. Instead of building a model from scratch, you start with the language knowledge it already has and shape it to fit your domain, style, or task. Picture a newly graduated medical doctor. After years of general medical school (the pre-training), they understand anatomy, pharmacology, and diagnostics broadly. When they finish a cardiology residency (the fine tuning), they are not relearning medicine. They are deepening their expertise in one area using focused, real-world case data.
“Training a large language model from scratch costs millions of dollars and needs enormous computing infrastructure. Fine tuning lets organizations reach specialist-level performance for a small fraction of that cost. A model that already understands grammar, reasoning, and world knowledge just needs to learn your domain's patterns instead of starting from zero.”
| Component | What it does | Example |
|---|---|---|
Pre-trained Base Model | The starting foundation. A model already trained on billions of text examples that understands language broadly but has no specialization yet. | GPT-5.5 understands general English well but may not know your company's internal product naming conventions or response format |
Task-Specific Dataset | A curated set of labeled input-output pairs built for your target task. Quality matters far more than quantity. | A legal firm collects 2,000 example contracts paired with extracted clause summaries to teach the model legal document summarization |
Frozen vs. Trainable Parameters | In full fine tuning, all weights update. In PEFT (Parameter-Efficient Fine-Tuning) methods like LoRA (a lightweight, low-cost method for fine-tuning large models), only a small subset of weights change while the rest stay frozen. | A startup with limited GPU resources uses LoRA adapter layers, like adding sticky notes to a textbook rather than rewriting it |
Loss Calculation & Weight Update | The model compares its output to the labeled answer. The gap (the loss) adjusts the weights through gradient descent so the responsible parameters improve. | If the model outputs 'the patient has flu symptoms' but the label is 'the patient presents with influenza,' the loss guides the model toward clinical phrasing |
Validation & Early Stopping | A held-out validation set watches for learning versus memorization. Training stops when validation performance stops improving. | An e-commerce team watches validation accuracy plateau at epoch 3 and stops, which avoids overfitting to last season's catalog |
Deployment | The fine-tuned model is served through an API or inference endpoint, replacing or adding to the base model for production tasks. | A healthcare startup deploys their fine-tuned clinical notes model via a private API endpoint used only by their internal EHR software |
Full assembled example
A customer support team: (1) starts with a GPT-5.5 base model, (2) curates 1,500 labeled ticket/response pairs across 8 intent categories, (3) applies LoRA to update only adapter layers (0.06% of parameters), (4) monitors validation loss per epoch and stops at epoch 2, (5) deploys the fine-tuned model to their support chat endpoint. The result: responses match the brand voice, categorize intents correctly, and escalate edge cases, all without expensive full retraining.
Core Principles
- 1Start with a strong base model - A company fine tuning a legal chatbot gets 40% better results using a law-adjacent base model than a general chat model with the same fine tuning dataset
- 2Quality of data beats quantity - A customer support team finds that 500 carefully written, varied example tickets outperform 5,000 auto-exported tickets full of duplicate boilerplate text
- 3Each training example needs a clear input and a clear output - 'Summarize this article in 2 sentences using formal tone' paired with a labeled output works far better than an unlabeled article alone
- 4Fine tuning reshapes behavior and style, not facts - Fine tuning a model on medical Q&A won't teach it about a drug approved last month. For new facts, use RAG. Fine tuning teaches the model to answer in the structured clinical format your doctors prefer.
- 5Prevent catastrophic forgetting - A customer service model fine-tuned only on refund scenarios starts giving nonsensical answers when asked general product questions. Use PEFT methods or multi-task datasets to keep general capability.
- 6Train for no more than 2 epochs on most datasets - A developer who runs 10 epochs notices the model repeating exact phrases from the training data rather than generalizing, a classic sign of memorization
Base model vs. fine-tuned model
Before
A general-purpose LLM answers customer support questions inconsistently. Sometimes too verbose, sometimes using the wrong brand terminology, and occasionally suggesting policies that don't apply.
After
After fine tuning on 1,500 curated support examples, the model responds in the brand voice, uses the correct escalation path, and structures answers in the company's approved format, consistently across all intents.
Supervised Fine Tuning (SFT) / Instruction Fine Tuning
The model is trained on labeled instruction-response pairs. Each example tells the model what task to do and shows the desired answer.
Prompt
Instruction: 'Classify the sentiment of this review as Positive, Negative, or Neutral.' Input: 'The product broke within a week. Terrible quality.' Output: 'Negative'
Output
A model that reliably classifies, summarizes, or formats outputs in exactly the structure your examples demonstrate
Full Fine Tuning
All parameters of the model update during training, producing a completely new model version with maximum specialization.
Prompt
A financial institution trains every layer of a 7B-parameter model on 50,000 internal earnings call transcripts to build a highly specialized financial analyst tool.
Output
A new model version fully optimized for the target task with no performance degradation from partial updates
Parameter-Efficient Fine Tuning (PEFT)
Only a small subset of model parameters update (typically 0.1–15% of total weights), keeping the rest frozen. Methods include LoRA, Adapters, and Prefix Tuning.
Prompt
A startup fine tunes a 13B LLM on a single GPU in hours by adding LoRA adapter layers, rather than updating the entire model.
Output
A specialized model that keeps its general capabilities while learning the target task at a fraction of the compute cost
Low-Rank Adaptation (LoRA / QLoRA)
A specific PEFT technique that injects small trainable matrices into transformer layers, sharply reducing trainable parameters. QLoRA (a memory-efficient variant of LoRA fine-tuning) adds 4-bit quantization (shrinking a model by storing its numbers at lower precision), cutting memory by about 33%.
Prompt
from peft import LoraConfig, get_peft_model config = LoraConfig(r=16, lora_alpha=32, target_modules=['q_proj', 'v_proj'], lora_dropout=0.05) model = get_peft_model(base_model, config) # trainable params: 4,194,304 || all params: 6,742,609,920 || trainable%: 0.06%
Output
A fine-tuned model using 0.06% of total parameters, which enables training of 7B+ models on consumer or cloud GPUs
Reinforcement Learning from Human Feedback (RLHF)
The model is trained using human preference data. Annotators rank pairs of outputs, and the model learns to produce outputs that humans prefer.
Prompt
Two responses to the same customer complaint are shown to annotators. They pick the more empathetic, helpful one. Over thousands of such pairs, the model learns human-preferred tone.
Output
A model aligned with human preferences for tone, helpfulness, and safety across open-ended generation tasks
Direct Preference Optimization (DPO)
A simpler alternative to RLHF (Reinforcement Learning from Human Feedback) that uses 'chosen' and 'rejected' response pairs directly, without a separate reward model.
Prompt
Prompt: 'Write a product return policy.' Chosen: 'Returns are accepted within 30 days with a receipt...' Rejected: 'You can maybe return stuff sometimes if you have a receipt or not.'
Output
RLHF-like alignment improvements with a simpler, faster training pipeline, supported natively by OpenAI's fine tuning API
Best Practices
- 1Define the task precisely before building any dataset - Before fine tuning a support bot, a team documents 12 response categories with example inputs and ideal outputs for each, then builds the dataset to match, not the other way around
- 2Use diverse examples, not repetitive ones - A legal team building a contract review model includes examples from real estate, employment, and software licensing contracts, not 500 near-identical NDAs
- 3Split your dataset into train, validation, and test sets - A developer uses 80% for training, 10% for validation during training, and holds back 10% as a final blind test set, never evaluating on training data
- 4Set a low learning rate - A team starts with lr=2e-5 and watches validation loss. Fine tuning uses much smaller learning rates than pre-training (1e-5 to 5e-5). Set it too high and the model unlearns its prior knowledge.
- 5Establish a baseline before fine tuning - A team runs their test set through the base model and records ROUGE (a metric for evaluating text summaries)/BLEU (a metric for evaluating machine translation) scores first. After fine tuning, they compare objectively rather than relying on feel.
- 6Use LoRA unless you have a specific reason for full fine tuning - A small team fine tunes a 7B model on a single A100 GPU in 4 hours using QLoRA, something impossible with full fine tuning. It also reduces overfitting risk and preserves the base model.
- 7Monitor training with experiment tracking - A developer notices validation loss starting to diverge at epoch 2 in their W&B dashboard and stops training before overfitting sets in, avoiding wasted compute and a degraded model
Common Mistakes
| Mistake | Real-world scenario | Fix |
|---|---|---|
| Training beyond 2 epochs | A team runs 10 epochs; the model starts quoting verbatim from training data on unseen prompts. | Stop at 1–2 epochs; use early stopping with validation loss monitoring. |
| Using noisy or repetitive training data | 5,000 near-identical customer support tickets produce a model that responds with one boilerplate answer. | Deduplicate data; ensure variety across intents, phrasings, and edge cases. |
| Expecting fine tuning to inject new facts | A company fine tunes a model on internal wikis hoping it 'learns' new product info; it hallucinates instead. | Use RAG for factual knowledge retrieval; use fine tuning only for style, format, and behavior. |
| Skipping a baseline evaluation | A team celebrates improved feel of outputs but has no metric showing actual improvement. | Always measure base model performance on your test set first. |
| Setting learning rate too high | Model 'forgets' general language patterns after fine tuning; responses become incoherent off-task. | Use a small learning rate (1e-5 to 5e-5); use a learning rate scheduler. |
| No post-deployment monitoring | Fine-tuned model drifts as real-world input distribution shifts; performance degrades silently. | Implement logging, output sampling, and periodic re-evaluation in production. |
| Catastrophic forgetting via single-task tuning | A coding assistant fine-tuned only on Python loses ability to help with JavaScript. | Use multi-task datasets or PEFT methods to preserve general capabilities. |
Use Cases
| Use case | Real-world example | Best approach |
|---|---|---|
| Customer support bot | E-commerce brand trains model on 3,000 tagged support tickets to respond in brand voice | SFT / Instruction fine tuning + DPO for tone alignment |
| Medical note generation | Hospital fine tunes a model on clinical transcription pairs to auto-draft SOAP notes | Full fine tuning or LoRA on a domain-adapted base model |
| Legal document analysis | Law firm trains model to extract obligations and deadlines from contracts | Instruction fine tuning with structured output format examples |
| Code completion / review | Developer tools company fine tunes on internal codebases for proprietary framework autocompletion | LoRA fine tuning on a code-focused base model |
| Financial report summarization | Asset manager fine tunes model on earnings calls to produce structured analyst-style summaries | Full fine tuning with multi-document summarization examples |
| Content moderation | Platform fine tunes a classifier to detect policy-violating content using labeled examples | Classification-head fine tuning with balanced class examples |
| HR / recruiting screening | Company trains a model on job descriptions and ideal candidate profiles to screen resumes | SFT with structured scoring rubric in output format |
| Retail product description | Brand fine tunes on approved product copy to maintain consistent style across SKUs | PEFT / LoRA on a base generative model |
Sentiment Classification: Customer Reviews
Weak prompt
Ask the base LLM to classify sentiment without any fine tuning. The results are inconsistent, sometimes returning explanations instead of a single label.
Strong prompt
Fine tune on labeled instruction-response pairs:
[
{"instruction": "Classify the sentiment of the following product review.", "input": "This blender is incredible - quiet, powerful, and easy to clean!", "output": "Positive"},
{"instruction": "Classify the sentiment of the following product review.", "input": "Stopped working after 3 uses. Absolute waste of money.", "output": "Negative"},
{"instruction": "Classify the sentiment of the following product review.", "input": "It works as described. Nothing special, does the job.", "output": "Neutral"}
]
At inference:
Instruction: 'Classify the sentiment of the following product review.'
Input: 'Battery dies within an hour. Expected much better for the price.'Output
Negative
Legal Clause Extraction
Weak prompt
Prompt the base model: 'Find the termination clause.' The model returns a paragraph-length summary with variable formatting and misses the specific clause text.
Strong prompt
Fine tune with structured input-output pairs:
{"instruction": "Extract the termination clause from the following contract excerpt.", "input": "...Either party may terminate this Agreement upon thirty (30) days written notice...", "output": "Termination Clause: Either party may terminate with 30 days written notice."}
At inference:
Instruction: 'Extract the termination clause from the following contract excerpt.'
Input: 'The client may cancel services at any time with 14 days advance written notice delivered via email.'Output
Termination Clause: The client may cancel services with 14 days' advance written notice via email.
Medical Note Formatting: SOAP Note Generation
Weak prompt
Ask the base LLM to 'write a SOAP note' from a transcript. The model returns a free-form paragraph with inconsistent section labeling and non-clinical phrasing.
Strong prompt
Fine tune with labeled clinical transcript → SOAP note pairs:
{"instruction": "Convert the following clinical transcript into a structured SOAP note.", "input": "Patient is a 45-year-old male presenting with chest tightness for 2 days. No history of cardiac issues. BP is 140/90. Prescribed amlodipine 5mg.", "output": "S: 45M with 2-day chest tightness, no prior cardiac history.\nO: BP 140/90 mmHg.\nA: Possible hypertension-related chest discomfort.\nP: Amlodipine 5mg prescribed. Follow-up in 2 weeks."}
At inference:
Input: '58-year-old female, reports knee pain for three weeks, worse on stairs. X-ray shows mild osteoarthritis. Recommend ibuprofen 400mg and physio referral.'Output
S: 58F with 3-week bilateral knee pain, exacerbated by stairs. O: X-ray: mild osteoarthritis. A: Osteoarthritis, knee. P: Ibuprofen 400mg PRN. Physiotherapy referral placed.
Tools & Platforms
| Tool | Best for | Link |
|---|---|---|
| Hugging Face Transformers + PEFT | Full fine tuning and LoRA/QLoRA workflows across open-source models | https://huggingface.co/docs/peft |
| OpenAI Fine Tuning API | SFT and DPO fine tuning for GPT-5.5, GPT-5.5-mini, and GPT-5.5-nano | https://platform.openai.com/docs/guides/fine-tuning |
| Google Vertex AI | Fine tuning Gemini and other Google foundation models at scale | https://cloud.google.com/vertex-ai |
| Unsloth | Ultra-fast LoRA/QLoRA fine tuning, up to 5x faster than standard PEFT | https://github.com/unslothai/unsloth |
| Axolotl | Flexible fine tuning configuration for Llama, Mistral, Falcon, and others | https://github.com/OpenAccess-AI-Collective/axolotl |
| Weights & Biases (W&B) | Experiment tracking, loss curve visualization, and hyperparameter sweeps during training | https://wandb.ai |
| DeepSpeed | Distributed fine tuning with ZeRO memory optimization for very large models | https://github.com/microsoft/DeepSpeed |
| Databricks + MLflow | Enterprise fine tuning pipelines with tracking and governance | https://www.databricks.com |
OpenAI Fine Tuning Guide
Official documentation covering SFT, DPO (Direct Preference Optimization, a way to align models from preference data), data format, and API usage
Hugging Face PEFT Docs
LoRA, QLoRA, and adapter method implementations for open-source models
arXiv: The Ultimate Guide to Fine-Tuning LLMs
Academic 7-stage pipeline reference covering the full fine tuning lifecycle
Frequently asked questions
What is fine-tuning in AI models?
Fine-tuning in AI involves taking a pre-trained model and continuing its training on a smaller, task-specific dataset to improve performance in a particular domain. This approach leverages the model's existing language understanding and adapts it to specific tasks, reducing the need for building a model from scratch.
How does fine-tuning improve model performance?
Fine-tuning enhances model performance by training it on a curated dataset specific to a desired task, allowing it to learn domain-specific patterns. This process refines the model's existing knowledge, enabling it to perform specialized tasks more effectively without the high costs of full retraining.
What are the key components of fine-tuning a language model?
The key components of fine-tuning include starting with a pre-trained base model, using a task-specific dataset, choosing between full or parameter-efficient fine-tuning, and validating the model to prevent overfitting. These steps help tailor the model to specific tasks while maintaining general language understanding.
What is the difference between full fine-tuning and parameter-efficient fine-tuning?
Full fine-tuning updates all model parameters, offering maximum specialization but requiring significant computational resources. Parameter-efficient fine-tuning, such as LoRA, updates only a subset of parameters, reducing computational costs while preserving the model's general capabilities.
Why is data quality important in fine-tuning?
Data quality is crucial in fine-tuning because high-quality, labeled examples ensure the model learns the correct patterns and outputs. A smaller dataset with clear, diverse examples can outperform larger, less curated datasets, leading to better task-specific performance.