Unit 14: Statistical Tools and Techniques - Subjective Questions

Bayesian inference updates existing knowledge about an unknown parameter using observed data.

• Prior distribution: represents beliefs about parameter before observing the data.
• Likelihood: describes the probability of observing data for a given value of .
• Marginal likelihood:
  
  It is also called the evidence and acts as a normalizing constant.
• Posterior distribution: represents updated beliefs about after observing .

Bayes' theorem combines these components:

Thus, the posterior is proportional to the product of the likelihood and prior:

Let denote the event that the person has the disease and denote a positive test.

Given:

Bayes' theorem gives:

Substituting the values:

Therefore, the probability that the person actually has the disease after receiving a positive result is approximately .

Bayesian and frequentist inference differ in their interpretation of probability and parameters.

• Meaning of probability: Bayesian probability represents a degree of belief, whereas frequentist probability represents long-run relative frequency.
• Treatment of parameters: In Bayesian inference, an unknown parameter is modeled using a probability distribution. In frequentist inference, the parameter is fixed but unknown.
• Use of prior information: Bayesian analysis combines a prior distribution with sample data. Frequentist analysis generally relies only on sample information.
• Interval estimation: A Bayesian credible interval directly assigns probability to the parameter. A frequentist confidence interval concerns the long-run success rate of the interval-producing procedure.
• Decision-making: Bayesian methods naturally incorporate loss functions and posterior expected loss. Frequentist methods often use sampling properties such as bias, variance, and significance levels.

Both approaches become increasingly data-driven as the sample size grows, but their interpretations remain different.

For from a binomial model, the likelihood kernel is:

Let the prior be:

with density kernel:

Multiplying the likelihood and prior gives:

Therefore:

For , , , and :

The posterior mean is:

Hence, the Bayes estimate under squared-error loss is . The beta prior is conjugate because the posterior remains in the beta family.

Suppose are independent Poisson observations with rate . The joint likelihood kernel is:

Let the prior distribution use the shape-rate parameterization:

where:

The posterior kernel is:

Thus:

Important posterior summaries are:

• Posterior mean:
  
• Posterior variance:
  

The gamma distribution is therefore a conjugate prior for the Poisson rate.

A Bayesian credible interval satisfies:

Given the observed data and selected prior, it means that the posterior probability that lies between and is .

A frequentist confidence interval is interpreted through repeated sampling. If the sampling procedure were repeated many times, approximately of the constructed intervals would contain the fixed parameter.

Key distinction:

• A credible interval makes a probability statement about conditional on observed data.
• A confidence interval makes a long-run statement about the interval-construction procedure.

Common Bayesian intervals include equal-tailed intervals and highest posterior density intervals.

The posterior predictive distribution gives the probability distribution of a future observation after accounting for uncertainty in the parameter .

It is defined as:

The expression averages the sampling distribution of the future observation over the posterior distribution of the parameter.

It is important because it is used for:

• Prediction: Estimating future values or events.
• Uncertainty quantification: Incorporating both random observation error and parameter uncertainty.
• Model checking: Comparing observed statistics with replicated data generated from the fitted model.
• Decision-making: Evaluating the expected consequences of future outcomes.

Unlike a prediction based only on a point estimate, posterior prediction accounts for the full range of plausible parameter values.

Bayesian decision theory selects an action by minimizing expected loss under the posterior distribution.

• Parameter space: The set of possible values of the unknown parameter .
• Action space: The set of decisions or estimates that may be selected.
• Loss function: measures the cost of taking action when the true parameter is .
• Posterior risk: The expected posterior loss of action is:
  
• Bayes rule: The optimal action is:
  

The resulting value is called a Bayes action or Bayes estimate. Its form depends on the selected loss function; for example, squared-error loss produces the posterior mean.

Under squared-error loss:

The posterior expected loss is:

Let . Write:

Then:

Since :

The posterior variance does not depend on , while is minimized at . Therefore:

Hence, the posterior mean is the Bayes estimator under squared-error loss.

The Bayes estimator depends on how estimation errors are penalized.

• Squared-error loss:
  
  The Bayes estimator is the posterior mean:
  
  Large errors receive a relatively severe penalty.

• Absolute-error loss:
  
  The Bayes estimator is any posterior median. It is less sensitive to extreme posterior values than the mean.

• Zero-one loss: For a discrete parameter, the loss is zero for a correct decision and one otherwise. The Bayes estimator is the posterior mode, also called the maximum a posteriori estimate:
  

Thus, the posterior mean, median, and mode arise from different assumptions about the consequences of estimation error.

Suppose:

where is known, and let:

Combining the normal likelihood with the normal prior gives:

where:

and

Equivalently:

Under squared-error loss, the Bayes estimator is the posterior mean:

It is a precision-weighted average of the prior mean and sample mean, illustrating shrinkage toward the prior mean.

• An informative prior expresses substantial previous knowledge about a parameter, often based on earlier studies, expert opinion, or historical data.
• A weakly informative prior restricts implausible parameter values without strongly determining the posterior result.
• A noninformative or diffuse prior is intended to have minimal influence, although no prior is completely neutral under every parameterization.

Prior sensitivity analysis repeats the analysis using several reasonable prior distributions and compares the resulting posterior summaries.

It is necessary because:

• Small samples may allow the prior to influence the posterior strongly.
• Different plausible priors may lead to different conclusions.
• Improper or excessively diffuse priors can create computational or inferential problems.
• It improves transparency and reveals whether conclusions are robust.

If posterior results remain similar across defensible priors, the conclusions are considered relatively insensitive to prior choice.

Microsoft Excel supports basic data preparation, summary, and visualization.

A suitable workflow is:

• Arrange observations in rows and variables in columns, with clear column headings.
• Check missing values, duplicate records, text-numeric inconsistencies, and data-entry errors.
• Use functions such as AVERAGE, MEDIAN, MODE.SNGL, VAR.S, STDEV.S, MIN, MAX, and COUNT.
• Use QUARTILE.INC or PERCENTILE.INC to obtain positional summaries.
• Enable the Analysis ToolPak and select Data Analysis → Descriptive Statistics for a consolidated summary.
• Construct histograms, box plots, scatterplots, and pivot tables to study distributions and relationships.

Excel is convenient for small and moderately sized data sets, but formulas, data ranges, and manual edits must be checked carefully to prevent unnoticed errors.

Create separate cells for the required probabilities:

• Disease prevalence:
• Sensitivity:
• False-positive rate:
• Complementary prevalence:

Calculate the total probability of a positive test:

Then calculate:

For example, if cells B2, B3, and B4 contain prevalence, sensitivity, and false-positive rate, respectively, an Excel formula is:

=(B3*B2)/((B3*B2)+(B4*(1-B2)))

Good practice includes:

• Labeling all inputs clearly.
• Using absolute references when copying formulas.
• Testing alternative assumptions with Data Tables or Scenario Manager.
• Formatting the result as a percentage.
• Checking that all probabilities lie between and .

The Analysis ToolPak provides menu-based statistical procedures.

For regression:

1. Arrange the response and explanatory variables in columns.
2. Select Data → Data Analysis → Regression.
3. Specify the input and input ranges.
4. Indicate whether labels are included and select residual or confidence-level output if required.
5. Interpret coefficients, standard errors, statistics, values, , adjusted , and the ANOVA table.

For correlation, select Data Analysis → Correlation and specify the variable range.

Limitations include:

• Restricted support for advanced statistical and Bayesian models.
• Greater risk of accidental formula or range errors.
• Limited reproducibility when steps are performed manually.
• Less flexible diagnostics and graphics than specialized software.
• Difficulty managing very large or complex data sets.

Regression assumptions should still be checked through residual analysis, regardless of the software used.

RStudio is an integrated development environment for the R programming language. It provides a source editor, console, environment viewer, plots panel, package tools, and documentation access.

A reproducible workflow generally includes:

1. Create an RStudio project for the analysis.
2. Import data using functions such as read.csv() or suitable packages.
3. Inspect data types, dimensions, missing values, and unusual observations.
4. Clean and transform data through scripted commands.
5. Perform exploratory analysis and construct graphs.
6. Fit statistical or Bayesian models.
7. examine diagnostics and validate assumptions.
8. Save commands in scripts rather than relying on console history.
9. Generate reports using Quarto or R Markdown.
10. Record package versions and use version control when appropriate.

RStudio improves reproducibility because the complete sequence from raw data to final results can be saved, reviewed, and rerun.

Suppose successes occur in trials and the prior is . The posterior is:

For example, with , , , and , useful R commands are:

• Posterior parameters: a_post <- 2 + 8 and b_post <- 2 + 10 - 8
• Posterior mean: a_post / (a_post + b_post)
• equal-tailed credible interval: qbeta(c(0.025, 0.975), a_post, b_post)
• Posterior simulation: theta <- rbeta(10000, a_post, b_post)
• Posterior probability : mean(theta > 0.5)
• Posterior-density graph: curve(dbeta(x, a_post, b_post), from=0, to=1)

Simulation is especially useful for probabilities, decisions, and transformations that are difficult to calculate analytically. A random seed should be set with set.seed() when reproducible simulation results are required.

A typical SPSS workflow is:

1. Define each variable in Variable View, including its name, type, label, value labels, missing-value codes, and measurement level.
2. Enter observations in Data View or import them from Excel, CSV, or database files.
3. Inspect the data using Analyze → Descriptive Statistics → Frequencies, Descriptives, or Explore.
4. Create graphs through Graphs → Chart Builder.
5. Select an inferential procedure, such as a test, ANOVA, correlation, chi-square test, or regression.
6. Specify dependent, independent, grouping, or factor variables as required.
7. Request confidence intervals, effect sizes, residuals, or diagnostic plots where available.
8. Interpret the Output Viewer, including estimates, test statistics, degrees of freedom, values, and confidence intervals.
9. Save SPSS syntax to make the analysis reproducible.

Before interpreting a significance test, the assumptions of the selected procedure should be checked.

For the regression model:

SPSS provides several important tables.

• Model Summary: measures the proportion of variation in explained by the model; adjusted accounts for the number of predictors.
• ANOVA table: The test evaluates whether the predictors jointly explain a statistically significant amount of variation.
• Coefficients table: The unstandardized coefficient measures the expected change in for a one-unit increase in a predictor, holding other predictors constant. The table also reports standard errors, statistics, values, and confidence intervals.

Major assumptions include:

• Linearity: Check scatterplots and residual-versus-predicted plots.
• Independence: Consider the study design and, where applicable, the Durbin-Watson statistic.
• Constant variance: Residual spread should remain approximately stable across fitted values.
• Normality of errors: Examine a histogram or normal probability plot of residuals.
• Absence of serious multicollinearity: Inspect tolerance and variance inflation factor values.
• Absence of influential outliers: Examine standardized residuals, leverage, and Cook's distance.

Microsoft Excel

• Familiar spreadsheet interface and easy data entry.
• Suitable for basic summaries, charts, correlation, and simple regression.
• Manual workflows can be difficult to reproduce and audit.
• Has limited native support for advanced Bayesian analysis.

RStudio

• Script-based environment with extensive statistical packages.
• Strong support for data manipulation, visualization, simulation, Bayesian modeling, and customized analysis.
• Highly reproducible when scripts, projects, and reports are preserved.
• Requires knowledge of the R programming language.

SPSS

• User-friendly menu interface for standard descriptive and inferential methods.
• Commonly used in social sciences and survey research.
• Syntax files can provide reproducibility, although menu-only workflows are less reproducible.
• Offers many standard models but is generally less flexible than R for customized Bayesian computation.

The best tool depends on the problem. Excel is convenient for small routine tasks, SPSS is effective for menu-driven standard analysis, and RStudio is generally strongest for programmable, reproducible, and advanced statistical work.