Book ; Online: Finite-sample concentration of the empirical relative entropy around its mean
2022
Abstract: In this note, we show that the relative entropy of an empirical distribution of $n$ samples drawn from a set of size $k$ with respect to the true underlying distribution is exponentially concentrated around its expectation, with central moment generating ...
Abstract | In this note, we show that the relative entropy of an empirical distribution of $n$ samples drawn from a set of size $k$ with respect to the true underlying distribution is exponentially concentrated around its expectation, with central moment generating function bounded by that of a gamma distribution with shape $2k$ and rate $n/2$. This improves on recent work of Bhatt and Pensia (arXiv 2021) on the same problem, who showed such a similar bound with an additional polylogarithmic factor of $k$ in the shape, and also confirms a recent conjecture of Mardia et al. (Information and Inference 2020). The proof proceeds by reducing the case $k>3$ of the multinomial distribution to the simpler case $k=2$ of the binomial, for which the desired bound follows from standard results on the concentration of the binomial. |
---|---|
Keywords | Mathematics - Statistics Theory ; Computer Science - Information Theory ; Mathematics - Probability |
Publishing date | 2022-03-01 |
Publishing country | us |
Document type | Book ; Online |
Database | BASE - Bielefeld Academic Search Engine (life sciences selection) |
Full text online
More links
Kategorien
Inter-library loan at ZB MED
Your chosen title can be delivered directly to ZB MED Cologne location if you are registered as a user at ZB MED Cologne.