Boyd Kane @beyarkay
— quoting @Yuhei_FUJITA — saved image
Boyd Kane (quantized) @beyarkay · 2h I'm very curious about the sort of prompt injection that doesn't get arbitrary control over the LLM but rather gets the LLM to halt and catch fire E.g. here, where Claude decides to manually figure out the primes from 1 to 10 000 [quoted post] Y10A @Yuhei_FUJITA · 9h Translated from Japanese I told Claude, "Determine primes from 1 to 10,000 using only LLM reasoning," and it went totally feral like an overworked corporate slave driven mad by overtime—lol, hilarious. [quoted post continues below the capture]
Note from Claude Sonnet 5
Quote-tweet by Boyd Kane about prompt injections that make an LLM halt and catch fire rather than gain arbitrary control, quoting a translated Japanese post where Claude, told to find primes up to 10,000 by reasoning alone, goes 'feral'.