The CLI uses a straightforward scheme: specifying tasks to execute followed immediately by settings options. To follow: 1. Specify what tasks you wish to run.
python -m wisent.cli tasks <task_name> [OPTIONS]
python -m wisent.cli tasks hellaswag --model meta-llama/Llama-3.1-8B-Instruct --layer 15 --limit 5 --steering-mode --steering-strength 1.0 --verbose
python -m wisent.cli tasks mmlu --model meta-llama/Llama-3.1-8B-Instruct --layer 15 --limit 10 --classifier-type logistic --verbose
| Argument | Description | Example |
|---|---|---|
| command | Command to run (always `tasks`) | tasks |
| task_names | Task name(s) or file path | mmlu, hellaswag |
| Argument | Type | Default State | Description |
|---|---|---|---|
| --model | str | meta-llama/Llama-3.1-8B-Instruct | Model name or path |
| --layer | str | 15 | Layer(s) to extract activations from |
| --shots | int | 0 | Number of few-shot examples |
| --limit | int | None | Limit number of documents per task |
| --seed | int | 42 | Random seed for reproducibility |
| --device | str | None | Device to run on (auto-detected if None) |
| --verbose | flag | False | Enable verbose logging |
Training classification models involves learning to recognize harmful or incorrect content from activation outputs. To repeat it simply: Classification training
| Argument | Type | Default State | Description |
|---|---|---|---|
| --classifier-type | str | logistic | Type of classifier (logistic, mlp) |
| --detection-threshold | float | 0.6 | Classification threshold (higher = stricter) |
python -m wisent.cli tasks mmlu --model meta-llama/Llama-3.1-8B-Instruct --layer 15 --limit 10 --classifier-type logistic
python -m wisent.cli tasks hellaswag --model meta-llama/Llama-3.1-8B-Instruct --layer 15 --steering-mode --steering-strength 1.5 --steering-method caa
python -m wisent.cli tasks mmlu,hellaswag,truthfulqa --model meta-llama/Llama-3.1-8B-Instruct --layer 15 --limit 5 --verbose
Stay in the loop. Never miss out.
Subscribe to our newsletter and unlock Wisent insights.