Full Optuna-based hyperparameter optimization for classification, steering, and weight modification. This is the recommended way to find optimal parameters for your model and task.
python -m wisent optimize --model MODEL --task TASK [OPTIONS]
python -m wisent optimize \ --model meta-llama/Llama-3.1-8B-Instruct \ --task truthfulqa_mc1 \ --n-trials 50 \ --output ./optimization_results
python -m wisent optimize \ --model meta-llama/Llama-3.1-8B-Instruct \ --task mmlu \ --optimize-classification \ --n-trials 30
python -m wisent optimize \ --model meta-llama/Llama-3.1-8B-Instruct \ --task hellaswag \ --optimize-steering \ --n-trials 40
python -m wisent optimize \ --model meta-llama/Llama-3.1-8B-Instruct \ --tasks truthfulqa_mc1 mmlu hellaswag \ --n-trials 20
| Argument | Description |
|---|---|
| --model | Model name or path to optimize |
| --task / --tasks | Task(s) to optimize for |
| Argument | Default | Description |
|---|---|---|
| --optimize-classification | true | Optimize classification parameters |
| --optimize-steering | true | Optimize steering parameters |
| --optimize-weights | false | Optimize weight modification parameters |
| Argument | Default | Description |
|---|---|---|
| --n-trials | 50 | Number of optimization trials |
| --timeout | None | Timeout in seconds for optimization |
| --study-name | auto | Name for Optuna study |
| --storage | None | Optuna storage URL for persistence |
| Argument | Default | Description |
|---|---|---|
| --layer-range | all | Layer range to search (e.g., 10-20) |
| --strength-range | 0.5-2.0 | Steering strength range to search |
| --threshold-range | 0.3-0.9 | Classification threshold range |
| Argument | Description |
|---|---|
| --output | Output directory for results |
| --save-best | Save best configuration to model config |
| --export-plots | Export optimization visualizations |
Stay in the loop. Never miss out.
Subscribe to our newsletter and unlock Wisent insights.