Common Issues

Proper representation engineering proves challenging; sometimes results fail to suit specific needs perfectly. If you encounter problems there are some suggestions as well as an itemized checklist of considerations prior to deployment of your tool. To rephrase the given text more naturally with equivalent meaning: Engineering proper representations can be very hard. Results often do not match up perfectly to specific needs. Should you run into difficulties, here are some tips along with a checklist

Using an unoptimised layer

Based on internal testing results for both Llama 3.1 8B and Mistral 7B, Layer 15 out of 32 performs best; however, using Layer 15 in other models might result in classifier and steering functions not working properly. Layers differ among models and so activation extraction needs careful checking to ensure use of the calibration layer we have selected specifically for each one.

Using a steering strength that is too large

While it may be alluring to direct the model very effectively toward higher performance levels, employing a high steering factor frequently leads to depersonalization or 'lobotomy' of the model. Using overly strong steering factors becomes evident if you use large token strengths for favorite control vectors; there is then an increased likelihood of nonsense tokens and repetition. Take care especially when steering excessively.

Troubleshooting Checklist