How to Fine-Tune LLMs for Your Domain: LoRA, QLoRA, and Full Fine-Tuning

Step-by-Step Guide

1. 1

   Understand When to Fine-Tune

   Fine-tuning is not always the answer. Use fine-tuning when: you need a specific output format, the model lacks domain knowledge not available via RAG, you need lower latency (smaller fine-tuned model vs. large model with long prompts), or you need consistent behavior that prompting cannot achieve.

   💡

   Tip: Try RAG + prompt engineering first. Only fine-tune when you have exhausted prompt-based approaches and have a clear, measurable improvement target.
2. 2

   Prepare Your Training Data

   Quality data is more important than quantity. 500-1,000 high-quality examples often outperform 10,000 mediocre ones. Format data as instruction-response pairs. Ensure diversity in inputs. Remove duplicates and near-duplicates. Split into train/validation/test sets (80/10/10).

   ⚠️

   Warning: Never fine-tune on data that contains PII, copyrighted material you don't have rights to, or factually incorrect content. The model will learn and reproduce these.
3. 3

   Choose: Full vs. LoRA vs. QLoRA

   Full fine-tuning: best quality, requires multi-GPU, expensive ($100-1000s). LoRA: 95% of full quality, single GPU, cheap ($1-10). QLoRA: 90% of full quality, single GPU with limited VRAM, cheapest. For most use cases, LoRA is the sweet spot.

   text Copy

   Fine-Tuning Decision Matrix:
   Full FT:  70B model → 8x A100 80GB → $500+/run
   LoRA:    70B model → 2x A100 80GB → $20/run
   QLoRA:   70B model → 1x A100 80GB → $10/run
            7B model  → 1x T4 16GB   → $2/run

4. 4

   Implement LoRA Fine-Tuning

   LoRA adds trainable low-rank matrices (A and B) to frozen weights: W_new = W_frozen + A × B. The rank r is typically 8-64. Higher rank = more capacity but more parameters to train. Target the attention projection matrices (Q, K, V, O) for best results.

   python Copy

   from peft import LoraConfig, get_peft_model
   
   config = LoraConfig(
       r=16,  # rank
       lora_alpha=32,
       target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
       lora_dropout=0.05,
       task_type="CAUSAL_LM"
   )
   model = get_peft_model(base_model, config)
   # Only 0.1% of parameters are trainable!

5. 5

   Implement QLoRA for Large Models

   QLoRA combines 4-bit quantization of the base model with LoRA adapters. This enables fine-tuning of 70B models on a single 48GB GPU. The key innovation is NF4 (Normal Float 4) quantization which preserves the normal distribution of weights.

   💡

   Tip: Use bitsandbytes library for QLoRA. Set compute_dtype=torch.bfloat16 for stability. Double quantization (quantizing the quantization constants) saves additional memory.
6. 6

   Configure Training Hyperparameters

   Key hyperparameters for LoRA fine-tuning: learning rate 1e-4 to 3e-4, batch size 4-32 (with gradient accumulation), warmup ratio 0.03, cosine learning rate schedule, epochs 2-5, max sequence length 2048. Monitor validation loss — stop when it stops improving.

   ⚠️

   Warning: Learning rate is the most critical hyperparameter. Too high → catastrophic forgetting. Too low → no learning. Start with 2e-4 and adjust.
7. 7

   Train with Distributed Setup

   For full fine-tuning or large LoRA, use distributed training. DeepSpeed ZeRO Stage 2 or 3 shards optimizer states and gradients across GPUs. FSDP (Fully Sharded Data Parallel) is PyTorch's native alternative. For QLoRA, single GPU is usually sufficient.

   python Copy

   # DeepSpeed config for 70B full fine-tuning
   {
     "zero_optimization": {
       "stage": 3,
       "offload_optimizer": { "device": "cpu" },
       "offload_param": { "device": "cpu" }
     },
     "train_batch_size": "auto",
     "gradient_accumulation_steps": "auto"
   }

8. 8

   Prevent Catastrophic Forgetting

   Fine-tuning can cause the model to forget its pre-trained capabilities. Mitigation strategies: use LoRA (inherently preserves base weights), mix general data with task-specific data (10-20% general), use lower learning rates, and add a regularization term that penalizes drift from the original model.

9. 9

   Evaluate the Fine-Tuned Model

   Evaluate on: your task-specific test set, general benchmarks (MMLU, HumanEval) to check for forgetting, and human evaluation for quality. Compare against the base model with prompt engineering — if the fine-tuned model doesn't significantly beat the prompted base model, the fine-tuning wasn't worth it.

   

10. 10

    Merge and Deploy LoRA Adapters

    LoRA adapters can be merged into the base model for deployment (no additional inference cost) or kept separate for dynamic switching. Merging: W_final = W_base + (alpha/r) × A × B. Use peft library's merge_and_unload() for merging. For multi-tenant serving, keep adapters separate and swap them per request.

    💡

    Tip: Keeping adapters separate allows serving multiple fine-tuned variants from a single base model — saving massive GPU memory.
11. 11

    Set Up Continuous Fine-Tuning

    In production, your task may evolve. Set up a continuous fine-tuning pipeline: collect new data, filter and clean, run evaluation on current model, fine-tune if performance drops, A/B test new vs. old, deploy if better. Automate this pipeline with CI/CD for ML.