← All topics

cuda

2 captures, most recent first.

@realSharonZhou

— saved image

Sharon Zhou @realSharonZhou · 5h
NVIDIA CUDA and AMD ROCm are software. And software is free. Anthropic just ran Claude for a weekend and it got a version of itself running on GPUs that it's never seen. Every compute customer (frontier lab, neolabs, etc.) will just generate their own kernels on the fly for their own model architectures and onto new chips. This is b/c kernel optimization is incredibly RL-able - the performance verifiers are clear and relatively low-latency.
Note from Claude Sonnet 5

Tweet from Sharon Zhou (@realSharonZhou) arguing that CUDA/ROCm moats are eroding because Anthropic had Claude autonomously port itself to run on unfamiliar GPUs over a weekend, and that kernel optimization is well-suited to RL because performance verification is clear and low-latency.

aicomputecudaanthropicclaudekernel optimizationtwitter

unclear (likely a Claude/agent chat log or terminal output, app not identifiable from crop)

● Major finding already: the log shows "VRAM after setup: 1411MB" — so params + optimizer + buffers are only ~1.4 GB. The OOM at 22.96 GB is almost entirely train-tick activations / the three_block —autograd graph (~21 GB). The static base is not the problem; the activation memory is. (My helper
Note from Claude Sonnet 5

Partial screenshot, cropped at top and bottom, mid-sentence cutoff at "(My helper" — appears to be an AI coding assistant's debugging output about a CUDA out-of-memory error, likely related to Nathan's own ML work.

pytorchcudamemory debuggingmachine learningterminal output