[latexpage]

Let’s start our foray into inference by focusing on the sample mean.  Why are we so concerned with means? Two reasons: they give us a middle ground for comparison, and they are easy to calculate. In this section we will see what we can deduce about the sampling distribution of the sample mean.

The Central Limit Theorem for a Sample Mean

Applying the law of large numbers here, we could say that if you take larger and larger samples from a population, then the mean \(\overline{x}\) of the sample tends to get closer and closer to μ. From the central limit theorem, we know that as n gets larger and larger, the sample means follow a normal distribution. The larger n gets, the smaller the standard deviation gets. (Remember that the standard deviation for \(\overline{x}\) is [latex]\frac{\sigma }{\sqrt{n}}[/latex].) This means that the sample mean \(\overline{x}\) must be close to the population mean μ. We can say that μ is the value that the sample means approach as n gets larger. The central limit theorem illustrates the law of large numbers.

The size of the sample, n, that is required in order to be “large enough” depends on the original population from which the samples are drawn (the sample size should be at least 30 or the data should come from a normal distribution). If the original population is far from normal, then more observations are needed for the sample means or sums to be normal. Sampling is done with replacement.

The following images look at sampling distributions of the sample mean built from taking 1000 samples of different sample sizes from a normal Population.  What pattern do you notice?  

The following images look at sampling distributions of the sample mean built from taking 1000 samples of different sample sizes from a non-normal Population (in this case it happens to be exponential).  What pattern do you notice? 

What differences do you notice when sampling from a normal population vs. Non normal? 

We have just demonstrated the idea of central limit theorem (clt) for means, that as you increase the sample size, the sampling distribution of the sample mean tends toward a normal distribution.

To summarize, the central limit theorem for sample means says that if you keep drawing larger and larger samples (such as rolling one, two, five, and finally, ten dice) and calculating their means, the sample means form their own normal distribution (the sampling distribution). The normal distribution has the same mean as the original distribution and a variance that equals the original variance divided by the sample size. Standard deviation is the square root of variance, so the standard deviation of the sampling distribution (aka standard error) is the standard deviation of the original distribution divided by the square root of n. The variable n is the number of values that are averaged together, not the number of times the experiment is done.

It would be difficult to overstate the importance of the central limit theorem in statistical theory. Knowing that data, even if its distribution is not normal, behaves in a predictable way is a powerful tool.  We can simulate this idea using technology.

Suppose X is a random variable with a distribution that may be known or unknown (it can be any distribution). Using a subscript that matches the random variable, let:

• μ_X = the mean of X
• σ_X = the standard error of X

[latex]\sigma_{\overline{x}}\text{ = }\frac{\sigma}{\sqrt{n}}[/latex] = standard deviation of [latex]\overline{x}[/latex] and is called the standard error of the mean.  Note here we are assuming we know the population standard deviation.

If you draw random samples of size n, then as n increases, the random variable \(\overline{X}\) which consists of sample means, tends to be normally distributed and

[latex]\overline{x}[/latex] ~ N[latex]\left({\mu }_{x}\text{, }\frac{\sigma}{\sqrt{n}}\right)[/latex].

To put it more formally, if you draw random samples of size n, the distribution of the random variable \(\overline{X}\), which consists of sample means, is called the sampling distribution of the sample mean. The sampling distribution of the mean approaches a normal distribution as n, the sample size, increases.

Using the CLT

It is important to understand when to use the central limit theorem: 

• If you are being asked to find the probability of an individual value, do not use the CLT. Use the distribution of its random variable.
• If you are being asked to find the probability of the mean of a sample, then use the CLT for the mean.

The random variable [latex]\overline{X}[/latex] has a different z-score formula associated with it from that of a single observation. Remeber, The mean [latex]\overline{x}[/latex] is the mean of one sample and μ_X is the average, or center, of both X (The original distribution) and [latex]\overline{X}[/latex].

[latex]z=\frac{\overline{x}-{\mu }_{x}}{\left(\frac{{\sigma }_{x}}{\sqrt{n}}\right)}[/latex]

We can use our Z table and standardize just as we are already familiar with, or can use your technology of choice

Image References

Figure 6.4: Kindred Grey via Virginia Tech (2021). “Sampling Distributions of the Sample Mean from a Normal Population.” CC BY-SA 4.0. Retrieved from https://commons.wikimedia.org/wiki/File:Sampling_Distributions_of_the_Sample_Mean_from_a_Normal_Population.png

Figure 6.5: Kindred Grey via Virginia Tech (2021). “Sampling Distributions of the Sample Mean from a Non-Normal Population.” CC BY-SA 4.0. Retrieved from https://commons.wikimedia.org/wiki/File:Sampling_Distributions_of_the_Sample_Mean_from_a_Non-Normal_Population.png

Figure 6.6: Kindred Grey via Virginia Tech (2020). “Figure 6.4” CC BY-SA 4.0. Retrieved from https://commons.wikimedia.org/wiki/File:Figure_6.4.png . Adaptation of Figure 5.39 from OpenStax Introductory Statistics (2013) (CC BY 4.0). Retrieved from https://openstax.org/books/statistics/pages/5-practice