Ridge regression in R is fitted using the glmnet package with alpha = 0, which shrinks regression coefficients toward zero to correct for multicollinearity. The optimal shrinkage amount — the lambda (λ) parameter — is chosen using cv.glmnet(), which runs k-fold cross-validation and returns the lambda that minimizes prediction error. Below is the full workflow: diagnosing whether you actually need ridge regression, fitting it, choosing lambda, interpreting the output, and comparing it against lasso and elastic net.
Table of Contents
What Is Ridge Regression?
RIDGE
regression is a regularized version of linear regression. Ordinary least squares (OLS) finds coefficients that minimize the residual sum of squares (RSS).Ridge regression minimizes RSS plus an extra penalty term:
RSS + λ Σ βⱼ²
The second term is the L2 penalty — the sum of the squared coefficients, scaled by λ (lambda). As λ increases, coefficients shrink toward zero — never reaching exactly zero, which is the key distinction from lasso regression. When λ = 0, ridge regression is identical to OLS.
Info!
Ridge regression does not perform variable selection. Every predictor stays in the model; ridge only shrinks their influence. If you need a model that drops irrelevant predictors entirely, use lasso regression instead.
Do You Actually Need Ridge Regression? Check Multicollinearity First
Ridge regression solves one specific problem: multicollinearity — predictor variables that are highly correlated with each other. If your predictors aren't collinear, ridge regression buys you very little and a standard multiple linear regression model is simpler to interpret. Diagnose this before you reach for glmnet(), not after.
Two diagnostics matter here: a correlation matrix and the Variance Inflation Factor (VIF). Using mtcars with disp, hp, wt, and qsec as predictors of mpg:
cor(mtcars[, c("disp", "hp", "wt", "qsec")])| disp | hp | wt | qsec | |
|---|---|---|---|---|
| disp | 1.000 | 0.791 | 0.888 | -0.434 |
| hp | 0.791 | 1.000 | 0.659 | -0.708 |
| wt | 0.888 | 0.659 | 1.000 | -0.175 |
| qsec | -0.434 | -0.708 | -0.175 | 1.000 |
disp and wt correlate at 0.888 — engine displacement and car weight move together almost in lockstep. That's the multicollinearity signal. Confirm it with VIF (values above 5 are generally treated as a problem, above 10 as severe):
library(car)
model_ols <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars)
vif(model_ols)| Predictor | VIF |
|---|---|
| disp | 7.99 |
| wt | 6.92 |
| hp | 5.17 |
| qsec | 3.13 |
Three of the four predictors showed the VIF = 5 threshold. It is a legitimate multicollinearity case — ridge regression is an appropriate tool for this dataset, not just a convenient teaching example.
Fitting Ridge Regression in R with glmnet()
- Install and load the
glmnetpackage - Build a predictor matrix and response vector
- Fit the full ridge path with
alpha = 0 - Select lambda with
cv.glmnet() - Refit at the chosen lambda and read coefficients
install.packages("glmnet")
library(glmnet)glmnet() requires a predictor matrix and a response vector, not a formula — this trips up people used to lm() syntax.
x <- as.matrix(mtcars[, c("disp", "hp", "wt", "qsec")])
y <- mtcars$mpg
ridge_model <- glmnet(x, y, alpha = 0) # alpha = 0 = ridgeWarning!
Setting alpha = 1 fits lasso, not ridge. It is the single most common copy-paste error in ridge regression tutorials — always confirm alpha = 0 in your final script before you report results in a thesis or paper.
Choosing Lambda with cv.glmnet()
glmnet() fits an entire path of models across 100 lambda values by default. You still need to pick one. cv.glmnet() runs k-fold cross-validation (10 folds by default) and reports the lambda that minimizes mean cross-validated error:
set.seed(123)
cv_ridge <- cv.glmnet(x, y, alpha = 0, nfolds = 10)
best_lambda <- cv_ridge$lambda.min
best_lambdaCross-validating this model independently on standardized predictors returns a best-performing lambda in the range of λ ≈ 5.3, with the model explaining roughly 82% of the variance in mpg (R² ≈ 0.82, RMSE ≈ 2.51) — close to, but slightly more conservative than, the unregularized OLS fit (R² ≈ 0.84, RMSE ≈ 2.41). That gap is the cost of stability: ridge trades a small amount of training fit for coefficients that won't swing wildly if you resample the data.
plot(cv_ridge)
Fitting the Final Model and Reading Coefficients
final_ridge <- glmnet(x, y, alpha = 0, lambda = best_lambda)
coef(final_ridge)At the cross-validated optimum, all four predictors retain non-zero coefficients — wt carries the largest shrunk weight, consistent with car weight being the strongest mpg predictor even after correcting for its correlation with displacement. This is the expected ridge behavior: coefficients shrink in magnitude but none are eliminated.
plot(ridge_model, xvar = "lambda", label = TRUE)
Each line traces one predictor's coefficient as λ increases from left to right. Every line converges toward — but never touches — zero. That visual is the fastest way to explain to a thesis committee why ridge was chosen over lasso: no line ever fully drops out.
Ridge vs. Lasso vs. Elastic Net
All three are run through the same glmnet() function — only alpha changes.
| Method | alpha | Penalty | Sets coefficients to zero? | Best when |
|---|---|---|---|---|
| Ridge | 0 | L2 (Σβ²) | No | Predictors are collinear but all theoretically relevant |
| Lasso | 1 | L1 (Σ|β|) | Yes | You suspect several predictors are irrelevant and want automatic selection |
| Elastic Net | 0–1 | Mix of L1 + L2 | Partially | Collinearity and irrelevant predictors are both present |
On the mtcars example above, the three methods land close together in predictive performance — ridge (R² ≈ 0.82), lasso (R² ≈ 0.83, and it zeroed out disp entirely), and elastic net (R² ≈ 0.82). The near-identical fit is itself informative: when regularization methods converge on similar accuracy, the choice between them should be driven by whether you want a variable-selection story (lasso/elastic net) or a "keep everything, just stabilize it" story (ridge) — not by chasing a marginal R² difference.
Reporting Ridge Regression Results (Thesis / APA Style)
For a dissertation or journal write-up, report: the justification for regularization (VIF values), the method used to select λ (k-fold cross-validation, state k), the final λ value, and the model's R² and RMSE. Example phrasing:
Given evidence of multicollinearity among predictors (VIF range: 3.13–7.99), ridge regression was applied using the glmnet package (v4.1) in R. The optimal shrinkage parameter (λ = 5.34) was selected via 10-fold cross-validation, minimizing mean squared prediction error. The final model explained approximately 82% of the variance in the outcome variable (R² = .82, RMSE = 2.51).
Example reporting language
Frequently Asked Questions
What is the objective function in ridge regression?
The ridge regression objective function is RSS + λΣβⱼ² — the ordinary least squares residual sum of squares plus an L2 penalty term that shrinks coefficients toward zero as λ increases.
What does the ridge regression penalty actually do?
The penalty term (λΣβⱼ²) discourages large coefficient values. It doesn't remove predictors — it constrains how much influence any single predictor can have, which reduces variance in the coefficient estimates when predictors are correlated.
How do I run ridge regression with glmnet in R?
Load the glmnet package, build a predictor matrix with as.matrix(), and call glmnet(x, y, alpha = 0). Use cv.glmnet(x, y, alpha = 0) to select the optimal lambda via cross-validation before reporting final coefficients.
What is the ridge regression equation for the coefficient estimates?
Ridge coefficient estimates are given by β̂_ridge = (XᵀX + λI)⁻¹Xᵀy, where I is the identity matrix. The added λI term is what stabilizes the matrix inversion when predictors are collinear and XᵀX is close to singular.
Can ridge regression be used for variable selection?
No. Ridge shrinks coefficients toward zero but never sets them exactly to zero, so every predictor remains in the final model. If you need automatic variable selection, use lasso regression or elastic net instead.
Related Posts
Need this analysis done for your thesis or dissertation? I'll handle it in R, SPSS, or Minitab — with APA-formatted results delivered fast.
