Abstract:
Entropy is a functional of probability and is a measurement of information contained
in a system; however, the practical problem of estimating entropy in applied settings remains a
challenging and relevant problem. The Dirichlet prior is a popular choice in the Bayesian framework
for estimation of entropy when considering a multinomial likelihood. In this work, previously
unconsidered Dirichlet type priors are introduced and studied. These priors include a class of
Dirichlet generators as well as a noncentral Dirichlet construction, and in both cases includes the
usual Dirichlet as a special case. These considerations allow for flexible behaviour and can account
for negative and positive correlation. Resultant estimators for a particular functional, the power
sum, under these priors and assuming squared error loss, are derived and represented in terms of
the product moments of the posterior. This representation facilitates closed-form estimators for the
Tsallis entropy, and thus expedite computations of this generalised Shannon form. Select cases of
these proposed priors are considered to investigate the impact and effect on the estimation of Tsallis
entropy subject to different parameter scenarios.
