Statistical information and properties of sufficiency

Source abstract

Abstract In the theory of statistical estimation put forward by Fisher, an essential feature is the concept of statistical information, defined for estimates normally distributed as the reciprocal of their sampling variance, and more generally by the variance of ∂L/∂θ, where L is the logarithm of the likelihood function corresponding to our estimate of a parameterθ. The expected value (∂L/∂θ) is zero, so that we may write I = E(∂L/∂θ)2. (1) If ∂L/∂θ corresponds instead to the original sample of n independent observations from which our estimate was derived, its variance will be n times the information per observation, since ∂L/∂θ is then the sum of n similar components, and (1) will give the information in our sample.