Key takeaways
Lineardiscriminant analysis (LDA) in R is run with lda() from the MASS package. It finds the linear combination of predictors — the discriminant function — that maximizes between-group variance relative to within-group variance, then classifies new observations using the resulting LDA score and posterior probability. It assumes predictors are roughly normal within each class and share a common covariance matrix; when that second assumption fails, quadratic discriminant analysis (QDA) is the fallback.
Table of Contents
Linear Discriminant Analysis in R: The Complete lda() Guide
Linear discriminant analysis (LDA) in R is performed with the lda() function from the MASS package. Give it a formula and a data frame — lda(Species ~ ., data = train) — and it returns a discriminant function you can use for two things: classifying new observations into groups, or reducing several correlated predictors down to one or two dimensions you can actually plot.
%20in%20R%20Programming.png "Linear Discriminant Analysis (LDA) in R Programming")
What Is Linear Discriminant Analysis?
LDA is a supervised technique — meaning it needs labeled classes to learn from — that does two jobs at once: classification and dimensionality reduction. It was developed by Sir Ronald Fisher in 1936 as Fisher's Linear Discriminant (FLD), which only handled two classes. LDA is the generalization that extends Fisher's method to any number of classes.
The idea: find a line (or plane, in higher dimensions) that separates the classes as cleanly as possible by maximizing the ratio of between-class variance to within-class variance. Between-class variance measures how far apart the group means are; within-class variance measures how spread out each group is internally. A good discriminant function makes the first large and the second small.
The LDA Formula, Explained Simply
You don't need to run this by hand — lda() does it — but knowing what's happening under the hood helps you interpret the output. Given n observations with p predictors across K classes, LDA computes:
- SW — the within-class scatter matrix: how much each class's points spread around their own class mean.
- SB — the between-class scatter matrix: how far each class mean sits from the overall mean, weighted by class size.
LDA solves for the eigenvectors of SW-1SB. The eigenvector with the largest eigenvalue is the first linear discriminant (LD1) — the single direction that best separates your classes. Project your data onto it and you get the LDA score: a single number per observation that tells you where it sits relative to the class boundaries and how confidently it belongs to each class.
Assumptions of LDA
LDA can tolerate mild violations of these, but knowing them tells you when to reach for QDA instead:
- Normality — each predictor is approximately normally distributed within each class.
- Equal covariance — all classes share the same variance-covariance structure. This is the assumption LDA relies on that QDA relaxes.
- Independence — limited multicollinearity between predictors; heavily correlated features distort the discriminant function.
- Linear separability — a straight boundary can reasonably divide the classes. If the true boundary is curved, LDA will underperform QDA or a non-linear classifier.
Warning!
When the equal-covariance assumption fails, don't force LDA — switch to qda() from the same MASS package. It uses the same syntax but fits a separate covariance matrix per class.
How Many Discriminant Functions Do You Need?
With K classes, LDA computes at most K−1 discriminant functions. Three species of iris → at most 2 discriminants (LD1, LD2), which is why the earlier plot only needed two axes. With 5 classes you'd get up to 4 discriminants, but you rarely need all of them — check model$svd (the singular values) or the proportion-of-trace figures in the model summary to see how much separating power each discriminant actually carries. If LD1 alone explains 99% of the trace, LD2 through LDk-1 are adding almost nothing and can usually be dropped for visualization or reporting purposes.
Should You Standardize Predictors First?
lda() does not require pre-scaled predictors — it works directly on the raw variance-covariance structure. But if your predictors are on wildly different scales (say, income in dollars alongside age in years), scaling first makes the discriminant coefficients easier to compare and interpret, even though it won't change the classification result:
# Optional: scale numeric predictors before fitting
train_scaled <- train
train_scaled[ , 1:4] <- scale(train[ , 1:4])
model_scaled <- lda(Species ~ ., data = train_scaled)Related Posts
How to Perform LDA in R: Step-by-Step
Required R Packages
library(MASS) # lda(), qda()
library(caret) # createDataPartition(), confusionMatrix()
library(pROC) # roc()
library(irr) # kappam.fleiss()New to package installation? See How to Import and Install Packages in R. New to RStudio itself? Start with A Comprehensive Guide to RStudio.
Step 1 — Load the Data
This tutorial uses the built-in iris dataset: 150 flowers, 3 species (setosa, versicolor, virginica), 4 numeric predictors (sepal length/width, petal length/width).
data(iris)
str(iris)
summary(iris)Step 2 — Split Into Training and Test Sets
createDataPartition() from caret keeps the class proportions balanced across the split — important for classification tasks, unlike a plain random split.
set.seed(123)
train_index <- createDataPartition(iris$Species, p = 0.8, list = FALSE)
train <- iris[train_index, ]
test <- iris[-train_index, ]Step 3 — Fit the Model With lda()
# Fit the LDA model
model <- lda(Species ~ ., data = train)
print(model)The output has three parts worth reading closely:
Prior probabilities of groups:
setosa versicolor virginica
0.3333333 0.3333333 0.3333333
Group means:
Sepal.Length Sepal.Width Petal.Length Petal.Width
setosa 5.040 3.44 1.462 0.242
versicolor 5.953 2.78 4.230 1.320
virginica 6.617 2.97 5.590 1.997
Proportion of trace:
LD1 LD2
0.9916 0.0084 Prior probabilities confirm the classes are balanced (a third each) — lda() uses these as the baseline probability before looking at any predictor. Group means show setosa is consistently smaller across petal measurements, which is exactly why it's the easiest class to separate. Proportion of trace tells you LD1 alone captures roughly 99% of the between-class separation here — LD2 is doing almost no work, which is why a single-axis histogram of LD1 scores would already classify most flowers correctly on its own.
The coefficients (model$scaling) show each predictor's contribution to LD1: petal length and petal width load most heavily, sepal width the least — meaning petal measurements are driving almost all of the classification here, not sepal measurements.
Step 4 — Visualize the Separation
species_colors <- c("setosa" = "red", "versicolor" = "blue", "virginica" = "green")
plot(model, col = species_colors, main = "Visualization of Linear Discriminant Analysis")
LD1 cleanly separates setosa from the other two species; LD2 does most of the work separating versicolor from virginica, though you can see some overlap between those two — a visual cue that their classification accuracy will be slightly lower than setosa's.
Step 5 — Predict on New Data
# Predict class labels
pred_class <- predict(model, newdata = test, type = "class")
# Align factor levels for comparison
test_species <- as.character(test$Species)
test_species <- factor(test_species, levels = levels(pred_class$class))
table(pred_class$class, test_species)
# Predict posterior probabilities instead of hard labels
pred_prob <- predict(model, newdata = test, type = "response")
head(pred_prob, 5)Use type = "class" when you just need the predicted label, and type = "response" when you need the posterior probability for each class — useful when you want a confidence threshold rather than a hard classification.
Step 6 — Evaluate the Model
Accuracy — proportion of correct predictions:
accuracy <- sum(diag(table(pred_class$class, test_species))) / nrow(test)
accuracyConfusion matrix — where the misclassifications are happening, by class. A representative 80/20 split typically looks like this:
confusionMatrix(pred_class$class, test$Species)
# Reference
# Prediction setosa versicolor virginica
# setosa 10 0 0
# versicolor 0 9 1
# virginica 0 0 10
#
# Overall Accuracy : 0.9667All 10 setosa test flowers classify correctly every time — they're the easiest class, consistent with the group-means table above. The one error that shows up here is typical of LDA on this dataset: a versicolor flower misclassified as virginica, never the reverse for setosa. That single error costs about 3 percentage points of accuracy, which is why accuracy alone can be misleading on small test sets — one flower and one hard case account for the entire gap from perfect.
ROC curve — trade-off between true and false positive rates for a given class:
pred_prob1 <- pred_prob$posterior
roc_result <- roc(test$Species, pred_prob1[, "virginica"], levels = c("virginica", "setosa"))
roc_result
plot(roc_result)Cohen's kappa — agreement between predicted and true labels, adjusted for chance agreement. On the split above, kappa comes out to 0.95 — well above the 0.81+ threshold generally considered "almost perfect" agreement, and meaningfully more informative than the raw 96.7% accuracy figure alone, since kappa accounts for the fact that a 3-class problem has a much lower chance-agreement baseline (33%) than a 2-class one would:
kappam.fleiss(data.frame(pred_class, test$Species))Info!
Your exact numbers will vary slightly depending on the random seed and split ratio you use — that's expected. What should stay consistent across runs is where the errors happen: setosa near-perfect, occasional versicolor/virginica confusion, because that reflects real overlap in the underlying petal measurements, not sampling noise.
LDA vs Logistic Regression vs PCA
These three get confused constantly because they're all used near classification problems — but they answer different questions.
| Method | What it does | Key assumption | Use it when |
|---|---|---|---|
| LDA | Classifies AND reduces dimensions using class labels | Normal predictors, equal covariance across classes | You have labeled classes and want both classification and a low-dimensional projection |
| Logistic regression | Classifies by modeling class probability directly | No distributional assumption on predictors | Predictors aren't normal, or you mainly care about calibrated probabilities |
| PCA | Reduces dimensions using only variance, ignoring class labels | None re: classes — unsupervised | You have no labels, or want to explore structure before deciding on classes |
In practice: LDA tends to outperform logistic regression when classes are well-separated and roughly normal, because it uses more information (the full covariance structure) than logistic regression does. Logistic regression tends to be more robust when that normality assumption doesn't hold. If you're deciding between LDA and PCA specifically, see Factor Analysis vs PCA — the same supervised-vs-unsupervised logic applies to LDA vs PCA.
Real-World Applications of LDA
Beyond classroom datasets like iris, LDA shows up in a handful of recurring domains — worth knowing if you're framing a dissertation methodology section around why LDA fits your research question:
Bioinformatics and Gene Expression
LDA is used to classify tissue or patient samples (e.g., cancerous vs. healthy) based on gene expression profiles — often after a dimensionality-reduction step like PCA, since gene expression datasets routinely have far more features than observations. See our PCA in R guide if you need that step first.
Face and Pattern Recognition
Fisherfaces — one of the earliest practical face-recognition methods — is literally LDA applied to pixel intensities, finding the projection that best separates one person's face from another's while minimizing lighting and expression variation within each person's own images.
Credit Scoring and Finance
Banks have used discriminant analysis since the 1960s (Altman's Z-score for bankruptcy prediction is a direct application) to classify loan applicants or companies into risk categories based on financial ratios.
Marketing Segmentation
Given known customer segments, LDA identifies which behavioral or demographic variables actually separate the groups — useful when you want an interpretable answer ("income and frequency drive segment separation") rather than a black-box cluster label.
When LDA Hits the Small-Sample Problem
LDA needs to invert the within-class covariance matrix. When the number of predictors approaches or exceeds the number of observations per class — common in gene expression or text data — that matrix becomes singular or near-singular, and lda() either fails or produces unstable coefficients. Two practical fixes:
- Reduce dimensions first with PCA, then run LDA on the principal components instead of the raw features.
- Use a regularized variant — the
sparseLDAorpenalizedLDApackages shrink the coefficient estimates, trading a small amount of bias for a large reduction in variance when p is close to n.
Advantages and Disadvantages of LDA
Advantages
- Simple, interpretable coefficients — you can read off which predictors drive class separation.
- Computationally cheap relative to SVMs or neural networks, and scales well to large datasets.
- Does double duty as both a classifier and a dimensionality-reduction tool.
Disadvantages
- Sensitive to violations of its normality and equal-covariance assumptions.
- Purely linear — can't capture curved decision boundaries (QDA or kernel methods can).
- Degrades when the number of predictors approaches the number of observations (the small-sample-size problem), unless regularized.
Frequently Asked Questions
Is linear discriminant analysis supervised or unsupervised?
Supervised — LDA requires labeled classes in the training data to compute the discriminant function. This is the key difference from PCA, which needs no labels.
What packages do I need for LDA in R?
MASS for lda() and qda(), caret for data splitting and the confusion matrix, pROC for ROC curves, and irr for kappa statistics.
How is LDA different from logistic regression?
LDA assumes predictors are normally distributed within each class with equal covariance, then derives class probabilities from that distribution. Logistic regression makes no distributional assumption and models the probability of class membership directly. LDA tends to win when its assumptions hold; logistic regression is more robust when they don't.
What is Quadratic Discriminant Analysis (QDA)?
QDA relaxes LDA's equal-covariance assumption, letting each class have its own covariance matrix. It's more flexible and can capture curved boundaries, but needs more data to estimate reliably and is more prone to overfitting.
How do I predict on new data with an LDA model in R?
Use predict(model, newdata = new_data, type = "class") for hard class labels, or type = "response" for posterior probabilities per class.
How do you evaluate LDA model performance?
Accuracy, a confusion matrix (via confusionMatrix() from caret), an ROC curve for binary or one-vs-rest comparisons, and Fleiss' or Cohen's kappa for agreement adjusted for chance.
Does LDA use Mahalanobis distance?
Yes, conceptually. Classifying a new point under LDA is equivalent to assigning it to the class whose mean is closest in Mahalanobis distance, adjusted for the shared covariance matrix and prior class probabilities.
What is the LDA formula in simple terms?
LDA finds the direction (linear combination of predictors) that maximizes the ratio of between-class variance to within-class variance — solved as the eigenvectors of SW-1SB, where SW is the within-class scatter matrix and SB is the between-class scatter matrix.
Conclusion
LDA in R comes down to one function call — lda() from MASS — wrapped around a normal supervised-learning workflow: split your data, fit, predict, evaluate. The formula behind it is just a variance-maximization problem, and the assumptions worth checking are normality and equal covariance across classes. When those hold, LDA is hard to beat for interpretability; when they don't, QDA or logistic regression are the next steps, both covered in the comparison table above.
Need this analysis done for your thesis or dissertation? I'll handle it in R, SPSS, or Minitab — with APA-formatted results delivered fast.
%20in%20R%20Programming.png)