The Bayesian Second Law of Thermodynamics

Abstract

We derive a generalization of the Second Law of Thermodynamics that uses Bayesian updates to explicitly incorporate the effects of a measurement of a system at some point in its evolution. By allowing an experimenter's knowledge to be updated by the measurement process, this formulation resolves a tension between the fact that the entropy of a statistical system can sometimes fluctuate downward and the information-theoretic idea that knowledge of a stochastically-evolving system degrades over time. The Bayesian Second Law can be written as $ΔH(ρ_m, ρ) + \langle \mathcal{Q}\rangle_{F|m}\geq 0$, where $ΔH(ρ_m, ρ)$ is the change in the cross entropy between the original phase-space probability distribution $ρ$ and the measurement-updated distribution $ρ_m$, and $\langle \mathcal{Q}\rangle_{F|m}$ is the expectation value of a generalized heat flow out of the system. We also derive refined versions of the Second Law that bound the entropy increase from below by a non-negative number, as well as Bayesian versions of the Jarzynski equality. We demonstrate the formalism using simple analytical and numerical examples.

40 pages. Additional information and animations at http://preposterousuniverse.com/research/bsl/