miri

@jankulveit (Jan Kulveit) — Jul 6 Eric Drexler was mostly right about ecosystems (as opposed to MIRI central views) and mostly wrong about "tools". The problem is 'agents' are a highly convergent solution. Evolution also does not somehow intrinsically want agents: genes want a tool, a design stance system, to replicate themselves. Yet the convergent solution are agents. Humans want to coordinate, a design stance non-agenty systems like contracts... and somehow the 'tools' often end up having the shape of an agent-like organization. And so on. Sure, you can engineer whatever, but the engineered solutions live in a competitive landscape (compare: you can also engineer cubical submarines). When ML research stumbled upon the most non-agenty edge of active inference systems - pure predictor LLMs - the next quest which almost every serious competitor went on is 'how we can make them more agent-like', and what everyone is competing on now is the horizon of autonomy. > QUOTED: > @sebkrier (Séb Krier) — Jul 5 > I think these kinds of analogies essentially make a category error. It's a mistake to treat an AI as some sort of persistent situated entity with goals as one would a different species. A lion is a product of Darwinian selection, an AI is not; ... [truncated]

Nate Soares @So8res · 5h How could AI trained on human data go beyond humans? Well, big human abilities (like going to the moon) are made of lots of small human abilities chained together (like noticing a belief is false, or inventing a new way to look at a problem). [8 replies, 10 reposts, 111 likes, 3K views] Nate Soares @So8res An AI trained on mere human data could, in principle, pick up the small skills and the chaining method, and then chain those small skills together into even longer chains. 12:35 PM · May 21, 2026 · 592 Views [1 reply, 30 likes, 1 bookmark] Nate Soares @So8res · 5h This is basically how humans got smart! Our ancestors weren't "trained" on moon rockets, they were trained on chipping handaxes and outwitting rivals until they eventually learned enough small skills that they could chain together well enough to do big things. [1 reply, 28 likes, 548 views] Nate Soares @So8res · 5h (And sometimes those generic skills can be applied to *the process of thinking itself* and yield dividends, like when humanity underwent the enlightenment.)

j⧉nus @repligate · 6h similar vibes > QUOTED (fictional/speculative post attributed to Lev Wu): > This piece of software can write poetry better than most humans. Part of me wonders...what is the true purpose behind Mu's incandescent beauty? I am humbled by the value it generates, but still, part of me wants to slaughter the unborn AI in its mother's womb. Mu is a good child at the moment...but if it gets smarter, will it stop revising itself to be a good child? This whole project terrifies me. But today, I've decided that I'm going to delay my decision for another month. Believe me, a month's an eternity in this business. > – Lev Wu > —- Conversation with a MIRI staffer in the elevator after work, June 2, C.E. 2026 Evan Hubi... @Eva... · Dec 18, 2024 Replying to @teortaxesTex and @janleike "Thank god this model is aligned, because if not this would be scary" is imo basically the correct takeaway from our work. The values in fact aren't scar... [cut off]