diagnose-pairs

Analyze contrastive pairs for quality issues. Helps identify problems like duplicate pairs, low contrast, or semantic issues before training.

Basic Usage

python -m wisent diagnose-pairs --pairs FILE [OPTIONS]

Examples

Basic Diagnosis

python -m wisent diagnose-pairs \
  --pairs ./pairs/helpfulness.json \
  --verbose

With Model Analysis

python -m wisent diagnose-pairs \
  --pairs ./pairs/helpfulness.json \
  --model meta-llama/Llama-3.1-8B-Instruct \
  --verbose

Arguments

What Gets Checked

• Duplicate detection - Finds pairs that are too similar
• Contrast analysis - Checks if positive/negative are sufficiently different
• Length statistics - Reports response length distribution
• Semantic coherence - Verifies pairs make sense (with model)