continual learning

Rhythm Garg @rhythmrg · 19h
One of the most interesting unsolved problems in RL is how to train directly on production traces.

We know how to do this reasonably well when the feedback is instructive or prescriptive: distill from a strong teacher that can look at the trace and tell the model what it should have done instead.

But a lot of the most valuable production feedback is evaluative: did the task succeed, what score did it get, did the user accept the result, etc.

Normally you would train against this kind of feedback with environment replay (e.g. GRPO): rerun the policy, sample multiple trajectories, and optimize against their relative rewards. The problem is that for many real production traces, you can't recreate the exact environment state that generated the trace.

That's what makes algorithms like Flash-Reinforce, SAO, and BPCO so exciting – they point toward learning from one trajectory per prompt without needing to recreate the environment and generate a fresh group of samples.

If these methods work at scale and on real production workloads, they could become an important building block for continual learning

reinforcement learningproduction tracesgrpocontinual learningevaluative feedback

Minh Nhat Nguyen @menhguin

2026-08-29 — saved image

Minh Nhat Nguyen @menhguin · 4h
random criticisms of model arch hypes:
1. Continual learning - My problem isn't even architectural. It can be done, and I am even working on this! It's much stupider. Data preprocessing and curation is alr very bad w current LLMs. It will get be much harder to effectively make use of more neuroplastic and brittle weights at scale w current data methods. The good thing about scratchpads is that they are legible and inherently interpretable. You can just read them to debug, and most people don't even do that. If u put evolution in the weights layer, you have to be much more disciplined with context and scratchpads, to a level much less fault-tolerant than current data pipelines (assuming sample efficiency holds constant which it wouldn't be, tho).

2. World models - Generally it is not clear to me which the proposed video model architectures are necessary for AGI takeoff. At the limit yes, humans consume photons and there are way more photons than text. But at the limit we don't know what is optimal, and this is generally not the argument being made! The argument is often a more fundamental view of text being insufficient in some way to model the world, and I do argue that current LLMs already know and do far more than the average college grad could!

continual learningscratchpadsworld modelsmodel architecturedata curationagi takeoffinterpretability

@Hesamation

2026-08-25 — saved image

ħεsam @Hesamation · 9h
OX ALPHA MAY BE A CONTINUAL LEARNING MODEL

I've seen multiple posts showing it getting better every day. But OpenRouter says the model "retains prompts and completions but does not use them for training"

So there are 3 possibilities:

1. the provider is rapidly post-training and quietly deploying better checkpoints

2. a continual feedback loop where real world failures are discovered from usage, trained on, and pushed back into prod. the prompt aren't directly training the model, but they expose areas that need fixing

3. ACTUAL ONLINE LEARNING: the most interesting possibility. this is what many have speculated would be SSI's first launch, where the model is genuinely updating from live usage. as interesting as it is, we don't know for sure if this is exactly SSI's plan, or if this model is SSI's first release. just putting this out there.

None of the 3 seem more likely with the current evidence. But if Ox Alpha is really improving every day, giving it away for free for 1 week was the smartest move they could've made.

ox alphacontinual learningonline learningopenrouterssispeculation

shako @shakoistsLog

2026-08-01 — saved image

shako @shakoistsLog · 3h
agents are godlike at math, but can't update an invoice for a customer without fucking it up.

what does this tell us about reality? about humans?
[26 replies, 4 reposts, 199 likes, 7.9K views]

Tenobrus @tenobrus · 3h
imo something like..... general intelligence is very real and very powerful, but 1. domain specific intuition can get you *incredibly* far and 2. even very powerful general intelligence can be blocked by inherent lack of basic training and expertise in a domain
[3 replies, 38 likes, 1K views]

Sharmake Farah reposted
Tenobrus @tenobrus
imo a lot of what we think of as general intelligence is *capability to learn and acquire new skills*, and models do this exclusively ~offline or in context. if we had continual learning / ways to train intuition on the fly across multiple tasks invoices would look v different
11:12 AM · Aug 1, 2026 · 508 Views

ai capabilitiesagentsgeneral intelligencecontinual learningtwitter

@adi_baradwaj

2026-07-14

reinforcement learningagi researchrichard suttonoak architecturecontinual learning

@AlexGDimakis

2026-06-16 — reposted by Mert Cemri

ai capabilitiesbenchmarkscontinual learningcoding agentsresearch

continual learningcreativitymachine learningkenneth stanleytwitteropen-endednessllm limitations