All Projects
NLPTransformersClassificationDeployed

LLM Preference Classifier

Predicting which of two LLM responses a human judge would prefer

Predict which of two LLM responses a human judge preferred — model_a, model_b, or tie — from the LMSYS Chatbot Arena human-preference dataset. A 3-class classification problem scored by multi-class log loss, where calibrated probabilities matter as much as picking the top choice. Try the deployed baseline live below.

Dataset

Source

Kaggle competition — LMSYS Chatbot Arena human preference predictions

  • •57,477 labeled training rows: a prompt, two LLM responses (response_a / response_b), and which one a human judge preferred
  • •Label distribution: model_a wins 34.9%, model_b wins 34.2%, tie 30.9% — close to balanced across all three classes
  • •87.6% single-turn conversations; median prompt 17 words, median response ~158 words
  • •Prompts and responses are JSON-encoded lists (supports multi-turn conversations, parsed before use)
  • •Public test.csv ships with only 3 rows — the real scored test set is held out on Kaggle's servers

Approach

  • •Deployed baseline (the live demo below): TF-IDF on the combined prompt + both responses (word 1-2 grams, 50K features) plus TF-IDF on the two responses alone (word unigrams, 20K features), concatenated with 4 hand-crafted length features (len_a, len_b, len_ratio, len_diff), fed into multinomial Logistic Regression
  • •Flagship architecture (designed, not yet trained — see Training below): fine-tune microsoft/deberta-v3-large on 'Prompt / Response A / Response B' formatted as one sequence, CLS token → Dropout(0.1) → Linear(hidden, 3) classification head
  • •A/B swap augmentation planned for the DeBERTa run: swapping response_a/response_b and flipping the label doubles the effective training data and forces the model to judge on content rather than position

Training

  • •Baseline (actually run): 5-fold stratified CV via scikit-learn's cross_val_predict, LogisticRegression(C=0.5, max_iter=300, solver='lbfgs')
  • •Note: the lbfgs solver hit the max_iter=300 limit before fully converging in this run — a straightforward, unexplored lever to push the baseline slightly further
  • •DeBERTa (designed, not yet run): 5-fold stratified CV, AdamW with cosine schedule + 10% linear warmup, learning_rate=2e-5, effective batch size 32 (4 per-device × 8 gradient-accumulation steps), 3 epochs, max_length=1024 tokens (covers the 90th percentile), fp16 mixed precision

Results

CV log loss (baseline)1.0745 (baseline)
  • •Baseline 5-fold CV: log loss 1.0745, accuracy 44.4% — a modest improvement over random 3-class guessing (log loss ln(3) ≈ 1.0986), since TF-IDF n-grams have limited ability to judge response quality compared to a model that actually understands language
  • •The DeBERTa fine-tune hasn't been trained in this repo yet, so there's no verified score for it here. Public results for deberta-v3-large-class models on this competition typically land around 0.85–0.90 log loss — a reasonable expectation based on similar setups, not a number this specific run has achieved

Try It Live

Paste a prompt and two candidate responses — the model scores which one a human judge would more likely prefer.

LLM Preference Classifier — MLworld