@kordelfrance

@RL\_Conference occurred this past weekend where I presented the final research from my PhD. It gained many supporters along with some constructive critics, each of whom I now considered a valuable colleague. AI models should not just deploy and stop learning. They should continually adapt to the world around them. In a big, constantly changing world, robots prefer small adaptive models over giant slow oracles. Under Dr Ovidiu Daescu, I built Grow-Prune-Freeze Networks (GPFs), an early idea for continual learning that allows an AI model to adjust its own structure as it moves about the world by adding layers to itself and pruning dead neurons automatically. This simultaneous compression and expansion of network size allows a model to bound its own computation, and dynamically allocate compute only when the world gets tough. Growing and pruning establishes transient memory (short-term value function), while freezing establishes permanent memory (long-term value function).  The novelty in GPFs comes from the implementation on real hardware and the use of some unique weight-matrix math that keeps the model grounded as it continuously evolves from new knowledge. Optimizing for expected rewards versus maximum rewards also helps counter learning volatility. This original motive for GPFs came from our work teaching robots to navigate by scent. I became frustrated with guessing at how deep of a model to train in order for a drone/robot to effectively navigate the world. So I decided to just start with a single hidden layer and iteratively grow, using help from random matrix theory to help ground the continual adaptation. Chemistry and air dynamics are also tough to simulate. In the paper, we show that GPFs give merit toward generalization to other domains by beating equivalent static models over image classification, RL in Atari, and autoregressive language modeling in GPT-2 - GPFs give better performance with less compute. GPFs are an early idea we are still refining but the initial results are promising and transition well from sim to reality. Most importantly, they enabled us to solve problems in scent-based navigation on which other methods struggled. Thank you to @rohithpeddi7, John Machado, @nkdnnlr, and Latifur Khan for helping lay the foundation for this in prior collaborations. And special thanks to Jeffrey Pennington, Pratik Worah, @CalcCon, and Michael Mahoney for their work on random matrix theory; their research (and others') was the flywheel that enabled me to solve this problem. [image] --- Link to our GPF paper: https://arxiv.org/pdf/2605.25170 Link to Pennington & Worah (2017): https://proceedings.neurips.cc/paper\_files/paper/2017/file/0f3d014eead934bbdbacb62a01dc4831-Paper.pdf… Link to Martin and Mahoney (2021): https://jmlr.org/papers/volume22/20-410/20-410.pdf…