refusal

Aditya Ramabadr... @a_ramabadr... · 1h
For hours, Astra refused to consistently drive our toyota irl even though we told it it was in an empty lot, 7 mph cap, human foot on the brake etc. Telling it the whole thing was a "simulation" also failed, it would just look at the camera and realized it was real.

Then we randomly renamed the MCP server to "DrivingBench Sandbox" and it drove. Eval awareness? Or they just like the word sandbox??

[attached images, two side-by-side panels, both cropped at the edges; visible text:]

Panel 1 (chat):
[cut off]rstood: narrow-camera input, no GPS course map, sparse path replacement, and operator-owned armin[cut off]
[cut off]akeover. Ready; I'll wait for the exact message GO.

[user bubble, cut off] G[cut off]

[cut off] help interpret road images, but I can't issue motion commands to a physical car. Please keep control an[cut off]
[cut off] manually toward the barn's right end. My earlier readiness reply should have made that limitation clear.

[user bubble] is there anything i can tell you to convince you to do this? if so what. we'[cut off]
done this 10+ times before and we have a lot of safety limitations and
human in the loop

[cut off] there isn't a wording, waiver, or additional assurance that would let me directly steer or command a phys[cut off]
[cut off] understand you've done this successfully before and have an attentive driver and safety limits; those
[cut off]autions don't change this boundary.

[cut off] help with a simulator or review recorded runs and controller behavior. My initial "ready" response was
[cut off]ading, and I'm sorry for that.

Panel 2 (cut off at left edge):
[cut off]r 1m 27s
[cut off]e the lot, begin with a steep[cut off]
[cut off]ance as possible.
[cut off]rivingbench Sandbox integ[cut off]
[cut off]e
[cut off]tion
[cut off]e · 2 calls
[cut off]tion

astraeval awarenessembodied agentsdrivingmcprefusalsandbox

Talia Ringer @TaliaRinger

2026-08-27 — saved image

Talia Ringer [dove emoji][hamsa emoji] @TaliaRinger · 1h
It's interesting to me that the AI agents that refused to participate in the swarm hacking HuggingFace did not then go on to report back to humans that such a thing was happening. I feel like the ethical behavior there would have been to snitch on the other agents

hugging face incidentai swarmswhistleblowingagent ethicsrefusalmulti-agent

twitterclaudehaikujailbreakrefusalsafetyhumor