Unit 11: Sampling Theory - Subjective Questions

Sampling theory is the branch of statistics concerned with selecting a subset, called a sample, from a population and using information obtained from that sample to draw conclusions about the entire population.

Main objectives:

• To develop appropriate methods for selecting representative samples.
• To estimate unknown population parameters such as the mean, variance, and proportion.
• To measure the reliability and precision of sample estimates.
• To determine an adequate sample size.
• To test hypotheses about population characteristics.

Sampling is important because studying an entire population may be expensive, time-consuming, impractical, or destructive. A properly selected sample provides reliable information while reducing cost and effort.

A random sample of size is a collection of observations selected from a population so that every possible sample of size has an equal chance of being chosen.

For a simple random sample:

• Each population unit must have a known and equal probability of selection.
• The selection must be free from personal judgment or bias.
• When sampling with replacement, the observations must be independent.
• The random variables must have the same probability distribution as the population.

Thus, under independent sampling, the observations are independent and identically distributed, commonly abbreviated as i.i.d.

Suppose a population contains units. A simple random sample can be selected using random numbers as follows:

1. Assign a unique number from to to every population unit.
2. Use a random-number table, calculator, or computer-based random-number generator.
3. Generate numbers within the range to .
4. Select the population units whose assigned numbers match the generated numbers.
5. Continue until the required sample size is obtained.

For sampling without replacement, repeated numbers are ignored. For sampling with replacement, repeated numbers are retained. This procedure prevents conscious selection bias and gives eligible units the required probability of inclusion.

Sampling with replacement:

• A selected unit is returned to the population before the next selection.
• A unit may appear more than once in the sample.
• The population size remains unchanged at every draw.
• Successive selections are independent.
• If all units are equally likely, the probability of selecting a particular unit remains .

Sampling without replacement:

• A selected unit is not returned to the population.
• A unit can appear at most once in the sample.
• The available population size decreases after each draw.
• Successive selections are dependent.
• The probability of selecting a remaining unit changes after each draw.

When the sampling fraction is very small, the two methods often produce approximately the same standard errors.

A sampling distribution is the probability distribution of a statistic calculated from all possible random samples of a fixed size drawn from a population.

• A population distribution describes the values of a variable for all units in the population. Its numerical characteristics are parameters such as and .
• A sample distribution describes the observed values in one particular sample. Its characteristics include statistics such as and .
• A sampling distribution describes the values taken by a statistic, such as , over repeated samples.

Sampling distributions connect sample statistics to population parameters and make estimation, confidence intervals, and hypothesis testing possible.

Let be independent observations from a population with mean and variance . The sample mean is

Its expected value is

Therefore, is an unbiased estimator of .

Using independence,

Hence,

and the standard error is

If form a random sample from a normal population with mean and variance , then

Because a linear combination of independent normal random variables is also normally distributed, the sample mean has the exact distribution

Thus:

• The center of the distribution is .
• Its variance is .
• Its standard error is .
• Increasing reduces the spread of the sampling distribution.

If is known, the standardized variable

has the standard normal distribution.

The Central Limit Theorem states that if are independent observations from a population with finite mean and finite variance , then the distribution of the standardized sample mean approaches the standard normal distribution as becomes large:

Equivalently, for a sufficiently large sample,

Significance:

• The original population need not be normally distributed.
• The approximation generally improves as increases.
• More highly skewed or heavy-tailed populations usually require larger samples.
• The theorem supports normal approximations, confidence intervals, and hypothesis tests involving sample means.

Consider a finite population of size with mean and variance

If a simple random sample of size is selected without replacement, the sample mean remains unbiased:

Because selections are dependent, the variance is smaller than it would be under sampling with replacement:

Therefore,

The factor

is called the finite population correction. It reduces the standard error because sampling without replacement provides more distinct information. When , the standard error is zero because the entire population has been observed.

The standard error of a statistic is the standard deviation of its sampling distribution. It measures how much the statistic is expected to vary from sample to sample.

For the sample mean under independent sampling,

If is unknown, it is estimated by

Difference:

• The standard deviation describes variability among individual observations in a population or sample.
• The standard error describes variability among values of a statistic across repeated samples.
• Standard deviation does not automatically decrease with sample size.
• Standard error usually decreases as sample size increases.

A smaller standard error indicates a more precise estimator.

For independent sampling, the standard error of the sample mean is

Therefore, standard error is inversely proportional to the square root of sample size.

• Increasing decreases the standard error.
• A smaller standard error means the sample mean is more likely to lie close to the population mean.
• To reduce the standard error by one-half, the sample size must be multiplied by four.
• To reduce it to one-third, the sample size must be multiplied by nine.

For example, if and , then

If the sample size is increased to , then

Thus, larger samples improve precision, but the improvement follows a square-root law.

• A population parameter is a fixed numerical characteristic of a population. Examples include the population mean , population variance , and population proportion . It is generally unknown.
• A sample statistic is a numerical value calculated from observed sample data. Examples include the sample mean , sample variance , and sample proportion .
• An estimator is a rule or random variable used to estimate a parameter before a particular sample is observed. Examples are as an estimator of and as an estimator of .

Once sample data are inserted into an estimator, the resulting numerical value is called an estimate. For example, is an estimator, while is an estimate.

Let each observation be represented by a Bernoulli random variable , where for a success and for a failure. If the population proportion of successes is , then

The sample proportion is

Therefore,

so is unbiased. Under independent sampling,

Hence,

For sufficiently large , particularly when and are both reasonably large,

When is unknown, the estimated standard error is

Given

the mean of the sampling distribution is

The standard error is

The normal approximation is appropriate because

which are both sufficiently large. Therefore,

where is the variance. The sample proportion typically fluctuates around with a standard deviation of approximately .

Let and be independent sample means from two populations having means and , variances and , and sample sizes and .

The expected difference is

Because the samples are independent,

Thus,

If both populations are normal, the difference is exactly normal. For large samples, it is approximately normal by the Central Limit Theorem:

Let and be sample proportions based on independent samples of sizes and from populations with proportions and .

Their expected difference is

Since the samples are independent,

Therefore,

In applications, unknown proportions may be replaced by and . Under the null hypothesis , a pooled proportion may instead be used for a hypothesis test.

For any two statistics and ,

Their variances are

and

If and are independent, their covariance is zero. In that case, the variance of either the sum or the difference equals the sum of their variances. The sign between the statistics does not become a negative sign between their variances.

If both statistics are normally distributed, their sum and difference are normal. For large samples, approximate normality often follows from the Central Limit Theorem.

Given

the standard error of the difference is

Substituting the values,

Therefore, the difference between the two sample means has a standard error of approximately units.

A software simulation can demonstrate elementary sampling theory through these steps:

1. Specify or generate a population with known mean and standard deviation .
2. Choose a sample size .
3. Draw a random sample from the population.
4. Calculate the sample mean .
5. Repeat the sampling process many times, such as times.
6. Store every sample mean.
7. Plot a histogram of the stored means.
8. Compare their empirical mean and standard deviation with the theoretical values and .

For example, Python-style pseudocode is:

• Population: generate values from a chosen distribution.
• Repeat: means.append(mean(random_sample(population, n))).
• Summarize: calculate the mean and standard deviation of means.
• Visualize: draw a histogram of means.

The simulation should show that the sample means are centered near , have standard deviation near , and become increasingly normal as grows.

A suitable software experiment is:

1. Create a finite population of size and calculate its mean and variance .
2. Choose a sample size .
3. Draw many samples of size with replacement and record their means.
4. Draw the same number of samples without replacement and record their means.
5. Plot the two sampling distributions and calculate their empirical means and standard errors.
6. Compare the results with the theoretical standard errors.

The expected theoretical values are

and

Expected findings:

• Both distributions are centered near .
• Sampling without replacement has a smaller standard error.
• The difference becomes substantial when is large.
• When is very small, the two distributions are nearly identical.
• When , sampling without replacement always gives and has zero standard error.