richard sutton

Adi @adi_baradwaj · 6h "I've been fascinated by @RichardSSutton's approach to AGI with @oaklab_ai It's distinct from the other neolabs in a number of ways. As far as I can tell: - Zero pretraining, all learning is continual learning via RL at runtime - The system learns to extract features and define subproblems, computes intermediate value functions for these subproblems, and then updates the policy via value iteration - Feature discovery is done via generate-and-test (i.e. random search) or "Continual Backprop" which is basically backprop + generate-and-test Some open questions: - Having a good reward function is of course a huge part of the puzzle, and it's not clear to me if OaK has a good answer for this? - The press coverage claims that their goal is a 1-trillion-param agent that can learn and plan on 20W. Presumably this is just because it's an online algorithm with no pretraining, I don't see why the learning step itself would be any more energy efficient? - Unclear to me what function approximator they're using for the value function and how they're performing updates" [embedded image: slide titled "How OaK creates a subproblem from a feature i, κ" from a "Reinforcement Learning Conference" talk, showing bullet points defining i = feature number, κ = how intensely we want the feature (balanced against losing reward), the subproblem is to drive the world to a state where the feature is high without losing too much in terms of reward, "Find an option π, γ that maximizes the value of the ith feature at termination while respecting rewards and value", a math expression, and a photo of a speaker (older man in patterned shirt) at a podium. Caption on slide: "reward respecting subproblems of feature attainment"]

Richard Sutton @RichardSSutton I can't say enough good things about John Carmack @ID_AA_Carmack and his Keen Technologies. But now Khurram Javed @kjaved_ and I have broken away to start our own startup and pursue a slightly different path toward understanding intelligence. Like Keen (and like Ineffable) we at Oak Lab @oaklab_ai believe in reinforcement learning and that intelligence is created and maintained from run-time experience. But we think current deep learning methods are weak and inefficient, and need not more tweaks, but fundamentally new ideas and a thorough reworking before they can provide a solid foundation for achieving the more ambitious goals of AI. Last edited 6:42 AM · Jul 13, 2026 · 375.6K Views 85 replies, 242 reposts, 2.4K likes, 719 bookmarks Sasha Malysheva @aimalysheva · Jul 13 fundamental research is exactly where the leverage is right now, and RL is 100% about to have its moment again 🔥 I've spent ten years doing research in this space, and a lot of what I'm working on right now is trying to take those conclusions and push communication between LLMs to a genuinely different level