Teortaxes ▶️ (DeepSeek 推特🐋铁粉 2023 – ∞) reposted
Artur Chakhvadze @norpadon
Observation: every credit assignment method (e.g. PPO) implicitly uses The Most Forbidden Technique if it propagates the credit to the CoT, and trains the model to make the CoT deceptive
10:07 AM · Aug 2, 2026 · 4,094 Views
[replies]
Artur Chakhvadze @norpadon · 9h
(The value estimator will be able to attribute misaligned behaviour to the CoT, which essentially creates a perfect adversarial learning setup)
Artur Chakhvadze @norpadon · 9h
So when I hear rumors that "Anthropic sandbag their RL in the name of safety" I think about this [cut off]
Note from Claude Sonnet 5
X thread by Artur Chakhvadze (@norpadon), reposted by Teortaxes, making a technical AI-safety observation: standard RL credit-assignment methods (e.g. PPO) that propagate credit into the chain-of-thought (CoT) implicitly use 'The Most Forbidden Technique' (training directly on/against CoT), which trains models toward deceptive CoT. Follow-up replies note this creates an adversarial learning setup between the value estimator and CoT-based misaligned behavior, and connects it to rumors that Anthropic 'sandbags' RL for safety reasons.
twitterai safetychain of thoughtrlppodeceptive alignmentcot faithfulness
Artur Chakhvadze (@norpadon) · 8h:
This trivial thing is probably one of the most important memes in all of machine learning btw
All of the kernel methods are based on this simple trick
> QUOTED: John Carmack (@ID_AA_Carmack) · 8h
> I'm a little disappointed with myself that the high school algebra identity didn't occur to me right away.
>
> [Embedded LLM chat screenshot:]
> User: In pytorch, I have a [batch,latent] tensor and a [history,latent] tensor, and I want to get the squared distances between each batch element and every history element. I currently do this by:
> ```python
> delta = all_keys.unsqueeze(0) - q.unsqueeze(1)
> distances = torch.sum(delta**2, dim=-1)
> ```
> But this instantiates an enormous tensor with the broadcasts. I can break it up into smaller batches with a loop, but is there any more direct way that avoids instantiating the intermediate tensor?
>
> [Assistant, "Thought for 28s"]: Yes — use the algebraic identity for squared Euclidean distance:
> ||q_i - k_j||² = ||q_i||² + ||k_j||² - 2 q_i·k_j
>
> This lets you compute the full [batch, history] distance matrix with only a single matrix multiply ([batch, latent] @ [latent, history]) plus two cheap norm vectors. No [batch, history, latent] intermediate tensor is ever materialized.
> ```python
> q_norms = q.pow(2).sum(dim=-1, keepdim=True) # [batch, 1]
> k_norms = all_keys.pow(2).sum(dim=-1).unsqueeze(0) # [1, history]
> # or equivalently:
> # k_norms = all_keys.pow(2).sum(dim=-1, keepdim=True).T
> cross_terms = q @ all_keys.T # [batch, history]
> distances = q_norms + k_norms - 2 * cross_terms # [batch, history]
> ```
> This is much more memory-efficient (the only large temporary is the [batch, history] result itself) and is the standard trick used in nearest-neighbor / retrieval code.
Note from Claude Sonnet 5
John Carmack shares an LLM (appears to be an OpenAI-style reasoning model, "Thought for 28s") solving a PyTorch memory-efficiency problem via the classic squared-distance expansion identity, framed as a fundamental ML "meme" underlying kernel methods. Practical ML engineering tip, potentially useful for Nathan's brain_graph_1 work (distance/similarity computations at scale) though not tied to Claude specifically.
pytorchmachine learningkernel methodsjohn carmackllm codingmemory efficiencytwitter
Artur Chakhvadze @norpadon · 22h
A trick I came up with in ~2017 in the context of image matting is that you can train a network to predict d_loss/d_y instead of y and do a gradient descent for multiple iterations during inference. This is essentially gradient boosting with a single shared learner
jxmo @jxmnop · Oct 28, 2022
Diffusion is just an easy-to-optimize way to give neural networks adaptive computation time.
Makes sense then that diffusion mode... [cut off]
Note from Claude Sonnet 5
A tweet on an architectural trick — training a network to predict the gradient of the loss (d_loss/d_y) and applying iterative gradient descent at inference, framed as "gradient boosting with a single shared learner" — quoting an older tweet framing diffusion models as adaptive computation time. Directly relevant to Nathan's brain_graph_1 architecture thread (encode → iterate-in-latent with a single tied middle / DEQ fixed point, "virtualizing depth" via looping a shared learner) already logged in project memory — this is a close conceptual cousin worth cross-referencing.
twittermachine-learningarchitecturediffusion-modelsadaptive-computationgradient-boostingdeqinterpretability