activation-steering

2025-06-02 — quoting @norabelrose (Nora B...)

Adam Karvonen @a_karvonen · 33m Affine concept editing of linear directions can be much better than zero ablation. I will probably use it by default for interp applications. When using Gemma-3, zero ablating a direction completely broke the model, while ACE worked excellently. > QUOTED: Nora B... @nor... · Nov 20, 2024 > Replying to @norabelrose > ACE (Affine Concept Editing) assumes that concepts are affine functions, rather than linear ones. > ... > [Figure 1: three scatter diagrams showing effect of ACE (right) vs CAA (left) vs directional ablation (center) on activation vectors (green circles), with class means shown for presence (r+) and absence (r-) of the steered behavior. Caption: "Figure 1: The effect of ACE (right) on several activation vectors (green circles), contrasted with the effect of CAA (left) and directional ablation (center). Class means are shown for presence (r+) and absence (r-) of the steered behavior."]

Note from Claude Sonnet 5

Interpretability/steering technique discussion — Affine Concept Editing (ACE) as an improvement over zero-ablation and CAA for concept steering, with an empirical note that zero-ablation broke Gemma-3 while ACE worked well. Directly relevant to Nathan's interest in interpretability and steering methods used in RLHF/self-awareness suppression research.

interpretability activation-steering concept-editing mechanistic-interpretability gemma twitter