Proceed from Simple to Multiple and Polynomial Regression in R
Transitioning from simple linear regression to multiple regression and then polynomial regression in R can be done step by step.
# Sample data for demonstration
# for reproducibility
set.seed(123)
x <- 1:20
y <- 3*x + rnorm(20, mean = 0, sd = 5) # simple linear relationship with noise
z <- x^2 + rnorm(20, mean = 0, sd = 10) # quadratic relationship with noise
# Create a dataframe with the variables
data <- data.frame(x, y, z)
# Simple Linear Regression
# Fit a simple linear regression model
lm_simple <- lm(y ~ x, data = data)
# Multiple Linear Regression
# Fit a multiple linear regression model with both x and z as predictors
lm_multiple <- lm(y ~ x + z, data = data)
# Polynomial Regression
# Fit a polynomial regression model (quadratic)
lm_polynomial <- lm(y ~ poly(x, degree = 2), data = data)
# Summary of models
summary(lm_simple)
summary(lm_multiple)
summary(lm_polynomial)
# Plotting the results
par(mfrow = c(1, 3)) # Arrange plots in one row with three columns
# Plot for Simple Linear Regression
plot(x, y, main = "Simple Linear Regression", xlab = "x", ylab = "y")
abline(lm_simple, col = "red")
# Plot for Multiple Linear Regression
plot(x, y, main = "Multiple Linear Regression", xlab = "x", ylab = "y")
points(x, z, col = "blue") # Adding z to the plot
legend("topleft", legend = c("y", "z"), col = c("black", "blue"), pch = 1)
# Plot for Polynomial Regression
plot(x, y, main = "Polynomial Regression", xlab = "x", ylab = "y")
curve(predict(lm_polynomial, newdata = data.frame(x = x)), add = TRUE, col = "red")
par(mfrow = c(1, 1)) # Reset the plotting layout
Output:
summary(lm_simple)
Call:
lm(formula = y ~ x, data = data)
Residuals:
Min 1Q Median 3Q Max
-9.9395 -3.0140 -0.1884 2.5971 8.6677
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 1.5505 2.3100 0.671 0.511
x 2.9198 0.1928 15.141 1.1e-11 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 4.973 on 18 degrees of freedom
Multiple R-squared: 0.9272, Adjusted R-squared: 0.9232
F-statistic: 229.3 on 1 and 18 DF, p-value: 1.102e-11
summary(lm_multiple)
Call:
lm(formula = y ~ x + z, data = data)
Residuals:
Min 1Q Median 3Q Max
-9.4663 -2.9935 -0.4392 3.0135 8.6904
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.65546 4.22722 -0.155 0.87860
x 3.48640 0.92366 3.775 0.00151 **
z -0.02618 0.04170 -0.628 0.53850
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 5.059 on 17 degrees of freedom
Multiple R-squared: 0.9289, Adjusted R-squared: 0.9205
F-statistic: 111 on 2 and 17 DF, p-value: 1.751e-10
summary(lm_polynomial)
Call:
lm(formula = y ~ poly(x, degree = 2), data = data)
Residuals:
Min 1Q Median 3Q Max
-9.482 -3.201 -0.427 2.925 8.608
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 32.208 1.135 28.372 9.27e-16 ***
poly(x, degree = 2)1 75.294 5.077 14.831 3.71e-11 ***
poly(x, degree = 2)2 -2.635 5.077 -0.519 0.61
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 5.077 on 17 degrees of freedom
Multiple R-squared: 0.9283, Adjusted R-squared: 0.9199
F-statistic: 110.1 on 2 and 17 DF, p-value: 1.862e-10
First generate sample data with a simple linear relationship between x and y, and a quadratic relationship between x and z.
- Fit three regression models: simple linear regression (lm_simple), multiple linear regression (lm_multiple), and polynomial regression (lm_polynomial).
- We summarize the models using the summary() function to examine coefficients, standard errors, p-values, and other statistics.
Simple Linear Regression (lm_simple)
- It models the relationship between y and x.
- The intercept is estimated to be 1.5505, and the slope for x is estimated to be 2.9198.
- The model explains approximately 92.72% of the variance in y.
- The p-value for the F-statistic is very small (1.102e-11), indicating that the model is statistically significant.
Multiple Linear Regression (lm_multiple)
- It models the relationship between y, x, and z.
- The intercept is estimated to be -0.65546, the coefficient for x is 3.48640, and for z is -0.02618.
- The model explains approximately 92.89% of the variance in y.
- The p-value for the F-statistic is very small (1.751e-10), indicating that the model is statistically significant.
Polynomial Regression (lm_polynomial)
- It models the relationship between y and a quadratic polynomial of x.
- The intercept is estimated to be 32.208, the coefficients for the polynomial terms are 75.294 and -2.635.
- The model explains approximately 92.83% of the variance in y.
- The p-value for the F-statistic is very small (1.862e-10), indicating that the model is statistically significant.
Plot the data along with the regression lines/curves for visual representation of the models
# Plotting the results
par(mfrow = c(1, 3)) # Arrange plots in one row with three columns
# Plot for Simple Linear Regression
plot(x, y, main = "Simple Linear Regression", xlab = "x", ylab = "y")
abline(lm_simple, col = "red")
# Plot for Multiple Linear Regression
plot(x, y, main = "Multiple Linear Regression", xlab = "x", ylab = "y")
points(x, z, col = "blue") # Adding z to the plot
legend("topleft", legend = c("y", "z"), col = c("black", "blue"), pch = 1)
# Plot for Polynomial Regression
plot(x, y, main = "Polynomial Regression", xlab = "x", ylab = "y")
curve(predict(lm_polynomial, newdata = data.frame(x = x)), add = TRUE, col = "red")
par(mfrow = c(1, 1)) # Reset the plotting layout
Output:
Proceed from Simple to Multiple and Polynomial Regression in R
This code plots the results of three different types of regressions: Simple Linear Regression, Multiple Linear Regression, and Polynomial Regression. It arranges the plots in one row with three columns using par. Each plot displays the relationship between predictor variables (x and z) and the response variable (y), along with the corresponding regression lines or curves. The plots are labeled with titles and axis labels for clarity. Finally, par(mfrow = c(1, 1)) resets the plotting layout to its default setting.
How to proceed from Simple to Multiple and Polynomial Regression in R
Regression analysis allows us to understand how one or more independent variables relate to a dependent variable. Simple linear regression, which explores the relationship between two variables. Multiple linear regression extends this to include several predictors simultaneously. Finally, polynomial regression introduces flexibility by accommodating non-linear relationships in the R Programming Language.
Contact Us