What Is Logistic Regression and How Does It Predict Binary Outcomes?

Logistic regression is one of the most widely used statistical methods for modeling a binary outcome—such as pass/fail, defective/acceptable, or yes/no—based on one or more predictor variables. Unlike linear regression, it does not assume a straight-line relationship with the outcome; instead, it models the probability that an event occurs, making it a cornerstone of quality control, risk analysis, and medical diagnostics.

What It Is

Logistic regression, as formalized in Hosmer & Lemeshow's Applied Logistic Regression, estimates the probability \( P(Y=1) \) that a binary event occurs, given predictor variables \( X_1, X_2, \dots, X_k \). Instead of predicting the outcome directly, it predicts the log-odds (also called the logit) of the event:

\[
\log\left(\frac{P}{1-P}\right) = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \dots + \beta_k X_k
\]

Here, \( P \) is the probability of the event, and \( \beta \) coefficients are estimated from data using maximum likelihood. The model is traceable to the standard framework described by Hosmer & Lemeshow, where the focus is on interpreting coefficients as odds ratios and assessing model fit via deviance.

How It Works / Steps

1. Define the binary outcome – Code the event of interest as 1 and the non-event as 0.
2. Select predictor variables – These can be continuous, categorical, or a mix.
3. Fit the model – Use maximum likelihood estimation to find \( \beta \) values that maximize the probability of observing the sample data.
4. Interpret coefficients – Each \( \beta_j \) represents the change in log-odds per one-unit increase in \( X_j \), holding other variables constant. Exponentiating gives the odds ratio \( e^{\beta_j} \), which tells you how many times more likely the event is per unit change.
5. Assess model fit – Use the deviance statistic (a measure of model fit relative to a saturated model) and likelihood-ratio tests to compare nested models.

A Worked Illustrative Example

Example data (illustrative only): Suppose a manufacturer tests 100 parts and records whether each part is defective (1) or not (0), along with a single predictor: processing temperature in °C. The fitted logistic model is:

\[
\log\left(\frac{P}{1-P}\right) = -4.20 + 0.08 \times \text{Temperature}
\]

• For a part processed at 50°C: log-odds = \(-4.20 + 0.08 \times 50 = -0.20\). Probability \( P = 1 / (1 + e^{0.20}) \approx 0.45 \).
• For a part processed at 60°C: log-odds = \(-4.20 + 0.08 \times 60 = 0.60\). Probability \( P = 1 / (1 + e^{-0.60}) \approx 0.65 \).

The odds ratio for a 1°C increase is \( e^{0.08} \approx 1.083 \), meaning each degree increase raises the odds of a defect by about 8.3%. This example is for illustration only; real coefficients require fitting to actual data.

Common Pitfalls

• Overfitting – Including too many predictors relative to the number of events can produce unstable estimates.
• Ignoring non-linearity – The logit is linear in predictors, but the relationship with the outcome is not; check for interactions or curvature.
• Misinterpreting odds ratios – An odds ratio is not a risk ratio; do not state "8.3% more likely" when you mean "8.3% higher odds," unless the event is rare.
• Sample size – Hosmer & Lemeshow recommend at least 10 events per predictor variable for reliable estimation.

---

To compute logistic regression coefficients, odds ratios, and deviance quickly without manual math, use the free tool at https://www.6sq.com/tools/logistic_regression/ — paste your binary data and get results in seconds.