Simple Linear Regression Assignment Sample

Abstract

This report evaluates the relationship between weekly advertising expenditures and weekly sales revenue using a sample of 120 retail outlets. Utilizing Ordinary Least Squares (OLS) estimation, a simple linear regression model was fitted to the data. The independent predictor variable, weekly advertising spend (in thousands of dollars), was regressed against the dependent response variable, weekly sales revenue (in thousands of dollars). The resulting regression equation, Sales = 15.234 + 2.451(Advertising), demonstrated a high coefficient of determination (R² = 0.780), indicating that approximately 78% of the variability in weekly sales revenue is explained by weekly advertising expenditures. Hypothesis testing on the slope parameter yielded a test statistic of t = 20.45 (p < 0.001), indicating a highly significant positive relationship. While residual diagnostic analysis revealed minor patterns of heteroscedasticity, the Gauss-Markov assumptions were generally satisfied, confirming the reliability of the OLS parameters. The study concludes that advertising spend is a highly effective predictor of sales, although expanding the model to multiple regression is recommended to capture omitted covariates such as seasonal demand and regional demographics.

Introduction and Theoretical Framework

Modeling the predictive relationship between marketing input and revenue output represents a fundamental analytical requirement in applied economics (Galton, 1889). Simple linear regression provides the basic mathematical framework for estimating this relationship by defining a linear probabilistic model containing a single independent predictor variable and a continuous dependent response variable. Through this formulation, researchers can quantify the average rate of change in the response variable corresponding to a unit change in the predictor. The population regression model is expressed mathematically as:

Y_i = β₀ + β₁X_i + ε_i

Where Y_i represents the dependent variable (weekly sales revenue) for observation i, X_i represents the independent variable (weekly advertising spend), β₀ is the y-intercept, β₁ is the population slope coefficient, and ε_i represents the stochastic error term (Kutner et al., 2005). The error term accounts for the inherent variability in human behavior and any unobserved variables that influence sales.

To obtain the best linear unbiased estimators (BLUE) of the regression parameters, Ordinary Least Squares (OLS) estimation is applied. OLS functions by minimizing the sum of squared residuals, which represent the vertical distances between the observed values and the estimated regression line. The OLS estimator is widely used due to its property of being the Best Linear Unbiased Estimator (BLUE), as proven by the Gauss-Markov theorem (Aitken, 1935). The validity of these estimators depends on the Gauss-Markov theorem, which requires that five key assumptions be met: linearity of the parameters, zero conditional mean of the errors (E(ε|X) = 0), homoscedasticity (constant error variance, Var(ε) = σ²), no autocorrelation among residuals (Cov(ε_i, ε_j) = 0 for i ≠ j), and normality of the error distribution (Kutner et al., 2005). A violation of these assumptions leads to biased standard errors or inefficient parameter estimates, which compromises hypothesis testing and confidence intervals.

Methodology and Data Preprocessing

The dataset analyzed in this study consists of weekly performance records from 120 retail store locations during the 2025 fiscal year. The predictor variable, weekly advertising spend (X), is recorded in thousands of USD. The response variable, weekly sales revenue (Y), is also measured in thousands of USD. Prior to fitting the linear model, exploratory data analysis was conducted to summarize univariate distributions and evaluate the bivariate correlation between variables. Exploratory data analysis serves as a critical first step to detect outliers and evaluate the distribution of variables before executing model estimation (Tukey, 1977; Montgomery et al., 2012).

Below is a summary of the univariate descriptive statistics for the 120 store locations:

Bivariate examination was initiated by calculating the Pearson product-moment correlation coefficient (r) to determine the strength and direction of the linear association between advertising spend and sales revenue. The correlation coefficient was calculated as r = 0.883 (p < 0.001), indicating a strong, positive, and statistically significant linear relationship (Montgomery et al., 2012). Scatterplot visualization confirmed that the data points are distributed around a linear path with no visible curve, suggesting that a simple linear model is appropriate for fitting.

Simple Linear Regression Model Fitting

The sample regression model is defined by estimating the intercept parameter (b₀) and the slope parameter (b₁). The math formulations for parameter estimation are derived using the least-squares equations:

b₁ = Σ((X_i - X̄)(Y_i - Ȳ)) / Σ(X_i - X̄)²

b₀ = Ȳ - b₁X̄

Substituting the sample means (X̄ = 12.45, Ȳ = 45.74) and sum of squares from our dataset, the slope and intercept were computed as b₁ = 2.451 and b₀ = 15.234, respectively. Thus, the estimated regression equation is expressed as:

Ŷ_i = 15.234 + 2.451X_i

The slope coefficient of 2.451 implies that for every unit increase of $1,000 in weekly advertising spend, sales revenue is expected to increase by $2,451 on average. The intercept coefficient of 15.234 indicates that if weekly advertising spend is reduced to zero, the model predicts a baseline sales revenue of $15,234. In practice, this represents constant baseline customer demand, although extrapolating to zero should be done with caution since no observations fell below $4,200 in the sample (Kutner et al., 2005). The coefficient of determination, R², measures the proportion of total variation in the response variable explained by the regression model (Wright, 1921).

The fitting was executed in Python using the statsmodels library. Below is the code block used to build and summarize the model:

import pandas as pd
import numpy as np
import statsmodels.api as sm

# Load dataset and define variables
data = pd.read_csv('advertising_sales_data.csv')
x = data['AdvertisingSpend']
y = data['SalesRevenue']

# Add constant to predictor for intercept calculation
x_with_const = sm.add_constant(x)

# Fit OLS regression model
model = sm.OLS(y, x_with_const).fit()
print(model.summary())

Residual Diagnostics and Model Assumptions

To ensure that the OLS parameter estimates are the best linear unbiased estimators, residual diagnostic checks were performed to evaluate the Gauss-Markov assumptions (Montgomery et al., 2012). The residuals, defined as the difference between the observed and predicted values (e_i = Y_i - Ŷ_i), were calculated and plotted against predicted values to check for linearity and homoscedasticity.

A scatterplot of Residuals vs. Fitted values showed a relatively random distribution of residuals around the zero line, confirming the assumption of a linear functional form. However, a slight fan-shaped pattern was visible at higher fitted values, hinting at mild heteroscedasticity. To formally test for constant error variance, a Breusch-Pagan test was performed, yielding a test statistic of LM = 3.03 (p = 0.082). Since the p-value is greater than the standard alpha level of 0.05, the null hypothesis of homoscedasticity is not rejected, though the presence of borderline heteroscedasticity suggests that caution should be exercised at high expenditure levels.

The assumption of normality was assessed visually using a Normal Q-Q plot of the standardized residuals. The plotted residuals fell closely along the diagonal line, indicating that the error distribution is symmetric and approximately normal. This visual interpretation was corroborated by the Shapiro-Wilk test (W = 0.988, p = 0.342), which confirmed that the normality assumption holds. Finally, the independence of error terms was checked using the Durbin-Watson statistic. The test yielded a value of d = 1.94, which lies close to the ideal value of 2.0, indicating that no significant first-order autocorrelation is present in the residuals (Montgomery et al., 2012).

Results and Statistical Inference

Statistical inference was conducted to determine whether the positive relationship between advertising expenditures and sales revenue is statistically significant, rather than a product of random sampling variation. Hypothesis testing on regression parameters provides a mechanism for evaluating statistical significance under the assumption of normal errors (Fisher, 1925). The hypothesis test for the slope parameter was formulated as:

H₀: β₁ = 0   vs   H_a: β₁ ≠ 0

The null hypothesis states that advertising spend has no linear impact on sales revenue, while the alternative hypothesis states that a significant relationship exists (Wasserstein & Lazar, 2016). The t-statistic for the slope coefficient was calculated as t = 20.45. With 118 degrees of freedom (n - 2), this test statistic corresponds to a p-value of less than 0.001. Because the p-value is significantly smaller than our significance level (alpha = 0.05), we reject the null hypothesis and conclude that advertising spend is a highly significant predictor of sales revenue.

A 95% confidence interval for the slope coefficient was constructed, resulting in a range of [2.215, 2.687]. This means we can be 95% confident that the true population increase in sales revenue for every additional $1,000 spent on advertising lies between $2,215 and $2,687. The strength of the model was evaluated using the coefficient of determination (R²). The model returned an R² value of 0.780, indicating that 78.0% of the variance in weekly sales revenue is explained by weekly advertising spend, leaving only 22.0% of the variance unexplained. The overall model significance was confirmed by the F-statistic of F(1, 118) = 418.2 (p < 0.001).

Conclusion and Recommendations

This study demonstrates a strong, statistically significant positive linear relationship between weekly advertising expenditures and weekly sales revenue across the sample of 120 retail outlets. The fitted regression model, Sales = 15.234 + 2.451(Advertising), indicates that advertising spend accounts for 78% of the variability in sales revenue. Diagnostic testing confirmed that the Gauss-Markov assumptions are generally satisfied, with residuals demonstrating approximate normality, independence, and homoscedasticity. However, the model has clear limitations. Since it is restricted to a simple linear regression design, it cannot control for extraneous factors such as competitor activities, seasonal demand variations, or local store size. Omitting these variables may lead to omitted variable bias if they correlate with advertising spend. Therefore, it is recommended that future studies expand the model into a multiple linear regression framework to incorporate these covariates, thereby improving predictive accuracy and providing more precise policy recommendations for marketing budgets.

References

• Aitken, A. C. (1935). On Least Squares and Linear Combination of Observations. Proceedings of the Royal Society of Edinburgh, 55, 42-48.
• Fisher, R. A. (1925). Statistical Methods for Research Workers. Oliver and Boyd.
• Galton, F. (1889). Natural Inheritance. Macmillan.
• Kutner, M. H., Nachtsheim, C. J., Neter, J., & Li, W. (2005). Applied Linear Statistical Models (5th ed.). McGraw-Hill Irwin.
• Montgomery, D. C., Peck, E. A., & Vining, G. G. (2012). Introduction to Linear Regression Analysis (5th ed.). Wiley.
• Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley.
• Wasserstein, R. L., & Lazar, N. A. (2016). The ASA's statement on p-values: context, process, and purpose. The American Statistician, 70(2), 129-133.
• Wright, S. (1921). Correlation and causation. Journal of Agricultural Research, 20, 557-585.