Skip to content
Murano

Steps — Path Patch

PathPatch step: isolate the direct sender-to-receiver effect.

Activation patching (:class:~murano.steps.patch.Patch) replaces one site and lets the change propagate through every downstream route, so it measures a component’s total effect. Path patching isolates the sender path: it injects the sender’s activation from a second run while freezing every other attention head at its base value (and, when freeze_mlps=True, the MLPs too). With the MLPs frozen the perturbation can only reach the receiver along the direct residual path; with the default freeze_mlps=False the MLPs recompute and mediate it, so the measured quantity is the attention-mediated effect. This is the primitive behind circuit localization (name movers, s-inhibition, backup name movers in Wang et al.’s IOI work); the algorithm is the one from Goldowsky-Dill et al., “Localizing Model Behavior with Path Patching”.

For senders S and receiver R, with base = the run to measure and source = the run supplying the sender activations:

  1. Run source; cache each sender’s activation.
  2. Run base; cache every head’s per-head output and every MLP output.
  3. Run base again with every attention head frozen to its base output, MLPs frozen too when freeze_mlps (otherwise they recompute), and each sender’s output overwritten with its source value at positions. Capture R.
  4. When R is the final residual, step 3’s logits are the result. Otherwise run base once more with R patched to its step-3 value and everything downstream of R free, and read the logits.

R is a residual-stream node, or a specific head’s query/key/value input (a Node(layer, "self_attn", head=h, side=Q|K|V)): the latter isolates edges into a head, such as the S-inhibition heads writing to a name mover’s query in IOI. For a Q/K/V receiver, step 3 captures the head’s projection output and step 4 patches it, so the model’s own rotary, softmax, and downstream layers recompute from it.

Freezing head outputs (the attention output projection’s per-head input) plus MLP outputs is equivalent to freezing the reference’s q/k/v and matches what murano can write. The default direction (base = clean prompts, source = corrupt corrupt_prompts) injects the corrupt sender into the clean run, so a metric step reads how much of the behaviour the direct sender path carries.

nnsight’s own per-token trace idiom differs across versions, so the freeze, inject, and capture are done with plain torch forward hooks on the underlying modules, the same mechanism :meth:~murano.model.MuranoModel._register_generation_intervention uses; it is independent of the nnsight tracing layer.

PathPatch

Section titled “PathPatch”

Path-patch senders into a receiver and write the resulting logits.

Runs the base prompts while injecting the source prompts’ sender activations and freezing every attention head at its base value, then stores the logits so a metric step can score the sender path. With the defaults (base = clean prompts, source = corrupt corrupt_prompts) this reads how much of the behaviour the sender path carries.

By default (freeze_mlps=False) the MLPs recompute, so they mediate the sender’s perturbation and the measured quantity is the attention-mediated effect, not the pure direct residual path. Set freeze_mlps=True to freeze the MLPs too, leaving only the direct sender-to-receiver residual path.

Note the default direction is the reverse of :class:~murano.steps.patch.Patch: Patch defaults to denoising (base = corrupt, source = clean) while PathPatch defaults to noising (base = clean prompts, source = corrupt corrupt_prompts). So with :class:~murano.steps.metrics.RecoveredMetricStep the recovered fraction reads oppositely: swap base_key / source_key (or read the raw metric) if you want the denoising convention.

Reads from results:

  • results[base_key]: PromptBatch (default prompts), the run to measure.
  • results[source_key]: PromptBatch (default corrupt_prompts), the run
  • supplying the sender activations.

Writes to results:

  • results[logits_key]: Tensor [B, S, vocab] of path-patched logits.
  • results[mask_key]: Tensor [B, S] marking real (1) vs padding (0) for the
  • base batch, so a downstream metric step can locate the answer position.

Args:

  • model: Model backend to run.
  • senders: The sending sites, as a :class:~murano.nodes.NodeSet, an
  • address, an iterable of addresses, or an: class:~murano.nodes.Edge
  • (its source is the sender, its dest the receiver). Each sender is an
  • attention head (Node(layer, "self_attn", head=h)) or an MLP
  • (Node(layer, "mlp")). Pass this or senders_key, not both.
  • senders_key: Results key holding a
  • “: class:~murano.artifacts.ComponentSelection a discovery step wrote,
  • read at run time so attribute-then-path-patch composes in one pipeline.
  • The receiver is still fixed at construction. Pass this or
  • senders, not both.
  • receiver: The receiving site. A residual-stream “Node(layer,
  • "resid_post")`` (default: the final residual, the last layer), or a
  • head’s query/key/value input “Node(layer, “self_attn”, head=h,
  • side=Q|K|V)“ to path-patch into that head’s Q/K/V (e.g. an S-inhibition
  • head into a name mover’s query). The output side (O) is a head’s output,
  • addressable only as a sender; an MLP or side-less head receiver raises.
  • base_key: Results key of the batch to measure (default prompts).
  • source_key: Results key of the batch supplying sender activations (default
  • corrupt_prompts).
  • positions: Token position(s) to inject the senders at, in the
  • ****: func:`~murano.steps.metrics._answer_positions` form; None“ injects
  • at every position.
  • freeze_mlps: If True, freeze MLP outputs to their base values too, so only
  • the attention-mediated direct path survives; if False (default), MLPs
  • recompute and mediate the effect.
  • logits_key: Results key to write the path-patched logits under.
  • mask_key: Results key to write the base attention mask under.

Raises:

  • ValueError: If neither or both of senders and senders_key are
  • given, senders is empty or malformed, a sender/receiver layer or
  • sender/receiver head is out of range, a per-node receiver position is
  • given, both an Edge and a receiver are given, an Edge is
  • combined with senders_key, or the base and source prompts are not
  • token-length-matched per pair.
  • NotImplementedError: If receiver is neither a residual-stream site nor a
  • head’s Q/K/V input, or if a head’s Q/K/V receiver is requested on an
  • architecture with interleaved fused q/k/v (e.g. GPT-NeoX).

__init__

Section titled “__init__”
def __init__(self, model: ModelBackend, senders: NodeSet | AddressLike | Iterable[AddressLike] | Edge | None = None, receiver: AddressLike | None = None, senders_key: str | None = None, base_key: str = keys.PROMPTS, source_key: str = keys.CORRUPT_PROMPTS, positions: int | Sequence[int] | Tensor | None = None, freeze_mlps: bool = False, logits_key: str = keys.PATH_PATCHED_LOGITS, mask_key: str = keys.PATH_PATCHED_MASK):