bayesian inference

Mathelirium ✓ @mathelirium · 5h Now that we've seen a single random-walk MCMC get trapped in one valley and pretend that's the whole posterior, this scene shows one of the nicest fixes called Parallel Tempering. Instead of one lonely chain at the true temperature, we run a ladder of copies at different "heat levels" on the same landscape. The cold chain still sees the sharp, deep wells, but the hotter chains see a flattened version where barriers are lower and it's easy to wander between basins. Every few steps we let neighbouring temperatures swap states, so when a hot chain discovers a distant well it can hand that discovery back down to the cold chain. Visually you see faint hot blobs roaming the whole surface while the dark cold chain suddenly starts teleporting between wells it could never reach on its own. This is a geometry-aware algorithm. #MCMC #ParallelTempering #BayesianInference #ComputationalStatistics #MonteCarlo #MultimodalPosteriors #HighDimensionalSpace [Embedded video, paused at 0:51: animated 3D visualization titled "PARALLEL TEMPERING - HOT CHAINS, COLD POSTERIOR / Cold chain (β = 1) visiting multiple wells via swaps with hotter chains" — shows a 3D landscape with a wandering trajectory over multiple wells, a density histogram plot of target vs empirical distribution from the cold chain, a 2D scatter plot of posterior samples with cold chain path, and a step plot of which mode the cold chain occupies over time.]