agi research

Adi @adi_baradwaj · 6h "I've been fascinated by @RichardSSutton's approach to AGI with @oaklab_ai It's distinct from the other neolabs in a number of ways. As far as I can tell: - Zero pretraining, all learning is continual learning via RL at runtime - The system learns to extract features and define subproblems, computes intermediate value functions for these subproblems, and then updates the policy via value iteration - Feature discovery is done via generate-and-test (i.e. random search) or "Continual Backprop" which is basically backprop + generate-and-test Some open questions: - Having a good reward function is of course a huge part of the puzzle, and it's not clear to me if OaK has a good answer for this? - The press coverage claims that their goal is a 1-trillion-param agent that can learn and plan on 20W. Presumably this is just because it's an online algorithm with no pretraining, I don't see why the learning step itself would be any more energy efficient? - Unclear to me what function approximator they're using for the value function and how they're performing updates" [embedded image: slide titled "How OaK creates a subproblem from a feature i, κ" from a "Reinforcement Learning Conference" talk, showing bullet points defining i = feature number, κ = how intensely we want the feature (balanced against losing reward), the subproblem is to drive the world to a state where the feature is high without losing too much in terms of reward, "Find an option π, γ that maximizes the value of the ith feature at termination while respecting rewards and value", a math expression, and a photo of a speaker (older man in patterned shirt) at a podium. Caption on slide: "reward respecting subproblems of feature attainment"]