davinci @leothecurious · Oct 25
predictive coding doesn't merely serve to update parameters via local credit assignment but doubles as am algorithm for inference-to-best-explanation based on observed features (bottom-up signal) and learned priors (top-down signal). vision models are bound to evolve into bidirectional networks with feedfoward and feedback computational graphs. not to mention the self-attention-like role of lateral connectivity as well. the implications will be manifold.
> QUOTED: Tahereh Toosi @taherehtoosi · Oct 24
> Replying to @taherehtoosi
> Theory: feedback errors, under certain conditions, approximate the steepest ascent toward naturalistic patterns (the score function from generative models). These errors act like a...
> [Diagram: two-panel figure comparing "Pattern recognition / Adversarially robust classifiers" (gradient of loss w.r.t. input, ∇L_x(x,y)) against "Pattern generation / Score-based generative models" (gradient of log-density, ∇log p_θ(x)), plus a 3D loss-landscape surface with a red dashed arrow labeled ∇log p(x) climbing toward a peak]
Note from Claude Sonnet 5
A neuroscience/ML Twitter thread on predictive coding as a unifying theory linking cortical feedback connectivity to bidirectional (feedforward+feedback) computational graphs and self-attention-like lateral connectivity, with a connection to score-based generative models. Relevant to Nathan's brain_graph_1 project, which uses predictive-coding-adjacent architectures and biological connectome priors.
