Activation Aggregation Method

The Activation Aggregation Method specifies how distinct activation vectors get combined and processed into meaningful results that facilitate analysis and recognition. The Activation Aggregation Method specifies how separate activation vectors are aggregated and processed into meaningful results useful for

Activation aggregation logic is specifically about controlling representations; in learning through contrasts pairs we directly train classifiers using activations collected. Method of aggregating activations lets users define transformation that transforms twice the number of elements relative to total set used for training into stable vectors which one uses to guide. These transformed activations are included during inference to ensure model behavior aligns closer to desired by users.

There exist different approaches for performing this task; currently, support is limited to just one technique. The current method uses Contrastive Activation Addition (CAA) which averages out pairwise differences between positive and negative activations across entire sets of contrasting data samples. By subtracting negatives from positives individually for each sample and then averaging those resultant difference vectors, this method creates a single controlling vector with consistent directionality that influences model representations at inference time. Resultant control vector retains the same dimensionality as original activations and may also be scaled based on a gain factor to regulate the level of modification imposed upon internal model representations.