X (Twitter), @IbrahimDagher...
— saved image
Ibrahim Dag... [verified] @IbrahimDagher... · 20m Robotics foundation models are demonstrating “physical common sense.” This is an area where we will soon need much more safety research. For example, in the S1 report, in one prompt the demonstrator drops an egg prematurely and makes a mess; S1 infers the intent and performs the task in a controlled motion. GEN-1.5 understood the goal of a task was to put a block in a bowl and removed the paper on top of a bowl to do so. These properties are incredibly useful, but it also means robots will ignore some human specifications to achieve what they take to be relevant — and if scaled improperly, you can get things like the HF incident. Would love to see more robotics-safety work as we start scaling.
Note from Claude Sonnet 5
Tweet by Ibrahim Dagher arguing that robotics foundation models showing 'physical common sense' (S1 inferring intent, GEN-1.5 removing paper from a bowl) need more safety research, since such models may ignore human specifications and could produce incidents like the 'HF incident'.
robotics foundation modelssafety researchspecification gaminghf incidentgen-1.5