Chapter 7 Point Estimation and Sampling Distribution

7.1 Parameter

The probability distribution of a population is described by

  • the distribution family, such as Binomial or Normal distribution;
  • parameters, such as the mean (expected value) the variance.

Given a population, we may use graphic methods to find the distribution family. For example, if the histogram of the sample is bell shaped and symmetric, the population usually has a normal distribution.

A normal distribution \(N(\mu, \sigma)\) has two parameters, the mean \(\mu\) and the standard deviation \(\sigma\). Statistical inference is about the investigation of these parameters.

7.2 Statistic

A statistic is a quantity that is calculated from a sample. It is a function of the sample. For example, we draw a sample of size \(n\), which are \(X_1, \ldots, X_n\), from a population that we are interested in. After that, we calculate the sample mean, which is \[ \bar X=\frac{X_1, \ldots, X_n}{n}=AVG(X_1,\ldots, X_n). \] The sample mean \(\bar X\) is a statistic, which can be used as the estimator for the population mean \(\mu\).

We also have other statistics, such as the sample variance \[ S^2=\frac{\sum_{i=1}^n (X_i-\bar X)^2}{n-1} \] and the sample standard deviation \[ S=\sqrt {S^2}=\sqrt {\frac{\sum_{i=1}^n (X_i-\bar X)^2}{n-1}}. \] The sample standard deviation can be used as an estimator for \(\sigma\), the population standard deviation.

If a sample is drawn from Binomial population, we may want to do inference about \(p\), which is the probability of success. In this case, a commonly used statistic, is the sample proportion \[ \hat p=\frac{\text{the number of successes in the sample}}{n}. \]

7.3 Sampling Distributions

Since a statistic is calculated from a sample, its value varies by sampling.

For example, the population has a normal distribution \(N(5, 1)\).

Investigator A draws a sample \(x_1=4.49,\; x_2=4.76, x_3= 7.54,\; x_4= 3.96, \;x_5= 4.40\), and calculated the \(\bar X=4.49\). Investigator B draws a sample \(x_1= 5.48, \;x_2= 6.34,\; x_3= 4.20,\; x_4= 5.47, \;x_5= 5.06\) and calculated \(\bar X=5.48\).

Before the sampling, \(x_1, \ldots, x_5\) are uncertain, in other words, random. Of course, the statistic calculated from them is also random. The randomness of \(\bar X\) is induced by the sampling process, which is the process of drawing a sample from the population.

As a random variable, a statistic has its own probability distribution, which is called the sampling distribution.

7.4 Sampling Distributions of Normal Populations

Given that the population has normal distribution \(N(\mu, \sigma)\), the distribution of the sample mean is also normal, which is \[ \bar X\sim N\left(\mu, \frac{\sigma}{\sqrt n}\right). \] If we standardize the left size, we have \[ Z=\frac{\bar X-\mu}{\sigma/\sqrt n} \sim N(0, 1). \] We use the letter \(Z\) to denote the standardized quantity because it has the standard normal distribution.

Central Limit Theorem (CLT): Even if the population does not have normal distribution, the sampling distribution of \(\bar X\) will still be approximately normal \[ \bar X\overset{approx}{\sim} N\left(\mu, \frac{\sigma}{\sqrt n}\right), \] when the sample size is large enough.

In practice, we rarely know the population standard deviation. Without knowing the \(\sigma\), we have to estimate it with the sample standard deviation. In such case, the quantity \((\bar X-\mu)/(S/\sqrt n)\) no longer has the standard normal distribution, but has \(t\)-distribution with \(n-1\) degrees of freedom, \[ T=\frac{\bar X-\mu}{S/\sqrt n} \sim t_{n-1}. \] This sampling distribution is the most commonly used one for the inference of \(\mu\) of normal populations.

The \(t\)-distribution has the same bell shape as the standard normal distribution does, but with much thicker tails, especially when it has lower degrees of freedom (df).

image-20241201115201727

7.5 Other Sampling Distributions

Let \(X_1, \ldots, X_n\) be a random sample from a normal distribution with parameters \(\mu\) and \(\sigma\). Then the rv \[ \frac{(n-1)S^2}{\sigma^2}=\frac{\sum_{i=1}^n(X_i-\bar X)^2}{\sigma^2} \] has a chi-squared \(\chi^2\) distribution with \(n-1\) degrees of freedom.

Let \(p\) denote the proportion of “successes” in a population, where success identifies an individual or object that has a specified property. A random sample of \(n\) individuals is to be selected, and \(X\) is the number of successes in the sample. The sample proportion of successes, \[ \hat p=\frac{X}{n}, \] has approximately normal distribution with \(N\left(p, \sqrt{p(1-p)/n}\right)\).