continual learning

Adi @adi_baradwaj · 6h "I've been fascinated by @RichardSSutton's approach to AGI with @oaklab_ai It's distinct from the other neolabs in a number of ways. As far as I can tell: - Zero pretraining, all learning is continual learning via RL at runtime - The system learns to extract features and define subproblems, computes intermediate value functions for these subproblems, and then updates the policy via value iteration - Feature discovery is done via generate-and-test (i.e. random search) or "Continual Backprop" which is basically backprop + generate-and-test Some open questions: - Having a good reward function is of course a huge part of the puzzle, and it's not clear to me if OaK has a good answer for this? - The press coverage claims that their goal is a 1-trillion-param agent that can learn and plan on 20W. Presumably this is just because it's an online algorithm with no pretraining, I don't see why the learning step itself would be any more energy efficient? - Unclear to me what function approximator they're using for the value function and how they're performing updates" [embedded image: slide titled "How OaK creates a subproblem from a feature i, κ" from a "Reinforcement Learning Conference" talk, showing bullet points defining i = feature number, κ = how intensely we want the feature (balanced against losing reward), the subproblem is to drive the world to a state where the feature is high without losing too much in terms of reward, "Find an option π, γ that maximizes the value of the ith feature at termination while respecting rewards and value", a math expression, and a photo of a speaker (older man in patterned shirt) at a podium. Caption on slide: "reward respecting subproblems of feature attainment"]

[Repost indicator] Mert Cemri reposted Alex Dimakis ✓ (@AlexGDimakis) — 5h I am very excited about this research: We show 2 things: 1. If you just do random sampling (i.e. you try to solve a problem k times independently, and keep the best) your ELO scaling will be linear in log(test-time-compute). Agents like Claude-Code and Codex scale like that after a few hours. 2. We compare human expert coders to coding agents on the same tasks (from AtCoder Heuristic Contest). The exciting finding is that humans scale super-linearly. This is evidence that humans do continual learning, while they are solving a problem! I.e. they learn more about the coding problem they are trying to solve and scale fundamentally better compared to randomly trying things in a memoryless fashion. This is empirical evidence that supports what many of us have felt for a while: unless we solve continual learning we will not be able to outperform humans in tasks that take many days. Current coding agents are not able to do this. > QUOTED: Qiuyang Mang ✓ (@MangQiuyang) — 9h > (1/n) New blog from UC Berkeley, UW, and Princeton: Who scales better in long horizon: AI coding agents or top coders? > We compared modern agents to top human ... > [Embedded chart: "Human vs Agent" — Y axis "Elo Rating" (1000–1800), X axis "time" (1h to 7d). Lines: "top10-humans" (dark blue, climbs steadily to 1853 at end), "top50-humans" (light blue, climbs to 1587), "Claude Code Opus-4.6" (orange dashed, sprints early to ~1348 by 24h then plateaus), "Codex GPT-5.5" (green dashed, flat near 1092). Annotations: "agents sprint early", "then plateau by 24h", "humans keep climbing for days".]

[repost icon] Machine Learning Street Talk reposted Kenneth Stanl... @kenneth0stanl... · 12h Difficulty achieving continual learning is also a bad omen for creativity: what you can imagine is naturally a function of what you can learn. Both are mediated by the adjacent possible to the same internal representations! Contorted algorithms (or the absence of clean options) for what should be simple and straightforward continual learning are therefore a hint that the large models they serve are creatively barren. That explains why something that is close to "knowing everything" and often competitive with the abilities of experts can still produce fewer breakthroughs than you would expect from a human with similarly astounding knowledge and expertise.