Joey (e/λ) @shxf0072
this example is from rl paper, one of the best one
curiosity driven exploration,
if you squint eyes hard enough you can find jepa like arch trained end to end with rl
[Embedded image: page from a paper, "Curiosity-driven Exploration by Self-supervised Prediction" — Figure 2 diagram of an agent interacting with environment via policy π, ICM (Intrinsic Curiosity Module) computing intrinsic reward r^i from states s_t, s_t+1; right side shows ICM internals: forward model predicting φ̂(s_t+1) from φ(s_t) and a_t, inverse model predicting â_t from φ(s_t), φ(s_t+1), compared against actual a_t. Caption explains: agent optimizes sum of extrinsic reward r^e and curiosity-based intrinsic reward r^i; prediction error in feature space (not pixel space) is the curiosity signal, making exploration robust to uncontrollable environment aspects. Below, section 2.1 "Prediction error as curiosity reward" discusses why raw pixel-space prediction is undesirable (e.g. unpredictable leaf motion causing an "artificial curiosity trap"), and the need for a feature space that models controllable/agent-affecting elements while ignoring irrelevant background variation. Two passages hand-underlined in red: the paragraph about prediction error being unclear as an objective, and the passage about no known computationally feasible mechanism for measuring learning progress instead of prediction error.]
Note from Claude Sonnet 5
A tweet highlighting the classic Intrinsic Curiosity Module (ICM) paper (Pathak et al., "Curiosity-driven Exploration by Self-supervised Prediction") as an early example of a JEPA-like (joint embedding predictive architecture) approach trained end-to-end with RL — predicting in learned feature space rather than raw pixels to generate intrinsic/curiosity reward. Directly relevant to Nathan's brain_graph_1 project, which uses PVLV dopamine and curiosity-style intrinsic motivation signals; ICM's forward/inverse-model feature-space prediction is a foundational architecture pattern for that kind of RL exploration design.