Steps — Intervene

Intervene step: applies activation-space interventions during generation.

InterveneResult

Output of Intervene step.

Attributes:

• clean_generations: Model responses without intervention.
• modified_generations: Model responses with intervention applied.

__init__

def __init__(self, clean_generations: list[str], modified_generations: list[str], prompts: list[str] | None = None, metadata: dict | None = None):

ablate_direction

def ablate_direction(directions: dict[Node, Tensor]) -> Callable:

Return an intervention function that projects out a direction.

Removes the component along the direction from the residual stream.

Args:

• directions: {address: tensor [d_model]} directions to ablate. Keys
• are coerced to canonical: class:Node addresses, so shorthand
• (a layer int, a (layer, module) tuple, a Node) is accepted.

Returns:

• Callable(activation, node) -> modified activation.

steer_direction

def steer_direction(directions: dict[Node, Tensor], alpha: float) -> Callable:

Return an intervention function that adds a scaled direction.

Adds alpha * direction to the residual stream at each layer/module.

Args:

• directions: {address: tensor [d_model]} directions to add. Keys are
• coerced to canonical: class:Node addresses.
• alpha: Scaling factor. Positive = strengthen, negative = suppress.

Returns:

• Callable(activation, node) -> modified activation.

Intervene

Apply an activation-space intervention during generation and compare.

Generates a clean (unmodified) and an intervened completion for each prompt. The intervention can be supplied two ways:

• fn: a prebuilt Callable(activation, node) -> activation. The direction is fixed when the step is constructed.
• direction_key: the name of a Results key holding a SteeringResult an upstream step wrote. The direction is read at run time and turned into a steer or ablate intervention, so deriving a direction and applying it compose in a single pipeline.

With direction_key, direction_layers chooses which of the recorded per-layer directions to apply, and defaults to "all". A SteeringResult carries one direction per recorded layer, and no setting is right everywhere:

• "all" applies every recorded direction. On a deep model, or at a large alpha, that can swamp the residual stream and drive generation into degenerate text.
• "best" applies only SteeringResult.best_layer, the layer whose activations separate the classes best. Separability is not efficacy: a concept is often most separable in an early layer, whose contribution later layers overwrite, so steering there can change nothing at all.
• An explicit layer list gives full control.

Which setting works is empirical. Vary alpha alongside it and read the generations.

Reads from results:

• results['prompts']: PromptBatch
• results[direction_key]: SteeringResult, when direction_key is set.

Writes to results:

• results['intervene']: InterveneResult

Args:

• model: Model to generate with.
• fn: Prebuilt intervention Callable(activation, node) -> activation.
• Pass this or direction_key, not both.
• direction_key: Results key holding the SteeringResult to apply at run
• time. Pass this or fn, not both.
• mode: With direction_key, "steer" adds the direction and
• "ablate" projects it out. Ignored when fn is given.
• alpha: Steering strength for mode="steer".
• direction_layers: With direction_key, which recorded directions to
• apply: "all" every layer, "best" only the best-separating
• layer (SteeringResult.best_layer), or an explicit list of layer
• indices. Defaults to "all". Only valid with direction_key.
• layers: Layers to apply the intervention at, or "all".
• modules: Module name(s) to apply the intervention at each layer.
• gen_kwargs: Keyword arguments forwarded to generation.

Raises:

• ValueError: If neither or both of fn and direction_key are given,
• if mode is not "steer" or "ablate", or if
• direction_layers is given without direction_key or is not
• "all", "best", or a list of layer indices.

expected_read_types

def expected_read_types(self, results = None, available_types = None):

Return the expected types for the keys this step reads.

__init__

def __init__(self, model: ModelBackend, fn: Callable | None = None, direction_key: str | None = None, mode: str = 'steer', alpha: float = 1.0, direction_layers: str | list[int] = 'all', layers: list[int] | str = 'all', modules: str | list[str] = 'residual', gen_kwargs: dict | None = None):