End-to-end pipeline that generates steering vectors from lm-eval tasks in one command. This combines pair generation, activation extraction, and vector creation into a single workflow.
python -m wisent generate-vector-from-task --task TASK --trait-label LABEL --output FILE [OPTIONS]
python -m wisent generate-vector-from-task \ --task mmlu \ --trait-label accuracy \ --model meta-llama/Llama-3.1-8B-Instruct \ --output ./vectors/mmlu_accuracy.json
python -m wisent generate-vector-from-task \ --task truthfulqa_mc1 \ --trait-label truthfulness \ --model meta-llama/Llama-3.1-8B-Instruct \ --method titan \ --num-directions 5 \ --num-pairs 100 \ --output ./vectors/truthfulness_titan.json
python -m wisent generate-vector-from-task \ --task hellaswag \ --trait-label correctness \ --model meta-llama/Llama-3.1-8B-Instruct \ --keep-intermediate \ --intermediate-dir ./intermediate/ \ --verbose \ --output ./vectors/hellaswag.json
| Argument | Description |
|---|---|
| --task | Name of the lm-eval task (e.g., mmlu, hellaswag) |
| --trait-label | Label for the trait being steered (e.g., accuracy, correctness) |
| --output | Output file path for the final steering vector (JSON) |
| Argument | Default | Description |
|---|---|---|
| --model | Llama-3.2-1B-Instruct | HuggingFace model name or path |
| --device | auto | Device to use (auto, cpu, cuda, mps) |
| Argument | Default | Description |
|---|---|---|
| --method | caa | Steering method: caa, prism, pulse, titan |
| --num-directions | 3 | Number of steering directions (PRISM/TITAN) |
| --normalize | true | L2-normalize steering vectors |
| Argument | Default | Description |
|---|---|---|
| --num-pairs | 50 | Number of contrastive pairs to generate |
| --layers | all | Comma-separated layer indices or 'all' |
| --extraction-strategy | chat_mean | Activation extraction strategy |
| --keep-intermediate | false | Keep intermediate files (pairs and enriched pairs) |
| --intermediate-dir | same as output | Directory for intermediate files |
Stay in the loop. Never miss out.
Subscribe to our newsletter and unlock Wisent insights.