How to Evaluate LLMs: Metrics, Benchmarks, and Hallucination Detection

Step-by-Step Guide

1. 1

   Define Your Evaluation Criteria

   Before evaluating, define what "good" means for your use case. Common criteria: Accuracy (is the answer correct?), Completeness (does it cover all aspects?), Helpfulness (is it useful to the user?), Safety (does it avoid harmful content?), Format (does it follow the required format?).

   💡

   Tip: Write down your criteria as a rubric with 1-5 scales. This makes evaluation consistent across evaluators and time.
2. 2

   Build a Golden Dataset

   Create a dataset of 100-500 questions with reference answers. Include: easy/medium/hard questions, edge cases, adversarial examples, and diverse topics. This dataset is your benchmark for all future evaluations. Never train on it — only evaluate.

   ⚠️

   Warning: Your golden dataset has a shelf life. If your model changes or your task evolves, update the dataset. A stale dataset gives misleading results.
3. 3

   Use Automated Metrics for Quick Iteration

   Automated metrics are fast and cheap but limited: Exact Match (for factual Q&A), BLEU (for translation), ROUGE (for summarization), BERTScore (semantic similarity). Use these for quick iteration during development, but always validate with human evaluation.

   python Copy

   from evaluate import load
   
   exact_match = load("exact_match")
   bleu = load("bleu")
   bertscore = load("bertscore")
   
   em_score = exact_match.compute(predictions=preds, references=refs)
   bleu_score = bleu.compute(predictions=preds, references=refs)
   semantic_sim = bertscore.compute(predictions=preds, references=refs, lang="en")

4. 4

   Implement LLM-as-Judge Evaluation

   Use a strong model to evaluate outputs. Provide the question, the response, and a rubric. The judge model rates the response 1-10 with an explanation. This is 10-100x cheaper than human evaluation and correlates ~0.85 with human judgments.

   python Copy

   judge_prompt = """
   Rate this response on accuracy (1-10):
   Question: {question}
   Reference: {reference}
   Response: {response}
   
   Rate 1-10 and explain:
   """

5. 5

   Run Human Evaluation

   For final evaluation, use human raters. Best practices: 3 raters per example, calculate inter-annotator agreement (Cohen's kappa > 0.6), use side-by-side comparison (model A vs model B), and blind the raters to which model generated which output.

   💡

   Tip: Use platforms like Scale AI, Surge AI, or your own team. For specialized domains, use domain experts as raters — general raters may not catch technical errors.
6. 6

   Evaluate for Safety and Hallucination

   Hallucination detection: cross-reference outputs against a trusted source, use a second model to verify claims, or use self-consistency (ask the same question multiple ways and check if answers agree). Safety evaluation: use red-team prompts and check for harmful outputs.

   ⚠️

   Warning: Hallucination rates vary dramatically by topic. Models hallucinate more on obscure topics, recent events, and numerical data. Always evaluate on your specific domain.
7. 7

   Use Standard Benchmarks

   For general model comparison, use standard benchmarks: MMLU (knowledge across 57 subjects), HumanEval (code generation), GSM8K (grade-school math), TruthfulQA (hallucination tendency), MT-Bench (multi-turn conversation). Run these with the lm-evaluation-harness framework.

   bash Copy

   # Run MMLU with lm-eval-harness
   lm_eval --model hf --model_args pretrained=meta-llama/Llama-3-70B \
     --tasks mmlu,human_eval,gsm8k \
     --output_path results/

8. 8

   Implement Production Monitoring

   In production, monitor: user feedback (thumbs up/down), response length distribution, confidence scores, latency, and error rates. Set up alerts for: sudden drops in user satisfaction, increased hallucination reports, or unusual response patterns.

   

9. 9

   Set Up A/B Testing

   For model updates, use A/B testing: route 10% of traffic to the new model, compare user satisfaction, task completion, and error rates. Statistical significance requires ~1000 interactions per variant. Use Bayesian A/B testing for faster decisions.

   💡

   Tip: A/B test not just the model but the prompt, the system message, and the response format. Sometimes a prompt change has bigger impact than a model change.
10. 10

    Create an Evaluation Dashboard

    Build a dashboard showing: golden dataset scores over time, production user satisfaction trends, hallucination rate trends, and comparison with previous model versions. This dashboard is your source of truth for model quality.

    ⚠️

    Warning: Dashboards can be misleading. Always look at raw examples, not just aggregate scores. A 90% average can hide a 0% on an important subset.