Local Pinsker inequalities via Stein's discrete density approach

Abstract

Pinsker's inequality states that the relative entropy $d_{\mathrm{KL}}(X, Y)$ between two random variables $X$ and $Y$ dominates the square of the total variation distance $d_{\mathrm{TV}}(X,Y)$ between $X$ and $Y$. In this paper we introduce generalized Fisher information distances $\mathcal{J}(X, Y)$ between discrete distributions $X$ and $Y$ and prove that these also dominate the square of the total variation distance. To this end we introduce a general discrete Stein operator for which we prove a useful covariance identity. We illustrate our approach with several examples. Whenever competitor inequalities are available in the literature, the constants in ours are at least as good, and, in several cases, better.

This is a revised version of our paper "Discrete Stein characterizations and discrete information distances" (arXiv reference : arXiv:1201.0143). Essential changes have been made. Certain elements of the previous version remain relevant to the literature and have not been included in the present version, therefore we upload this as a new arXiv submission