Univariate Logistic Regression

Logistic Regression is used for classification problems, where the target is a category rather than a continuous number - most commonly a binary outcome like "yes/no" or "pass/fail." Univariate logistic regression uses a single input feature to make this prediction.

Why Not Just Use Linear Regression

Linear Regression can predict any number, including values below 0 or above 1, which doesn't make sense for a probability. Logistic Regression solves this using the sigmoid function, which squashes any input into a value between 0 and 1.

The Sigmoid Function

sigmoid(z) = 1 / (1 + e^(-z))

# Output is always between 0 and 1, interpreted as a probability

Implementing in Python

import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

df = pd.DataFrame({
    "HoursStudied": [1, 2, 3, 4, 5, 6, 7, 8],
    "Passed": [0, 0, 0, 1, 1, 1, 1, 1]
})

X = df[["HoursStudied"]]
y = df["Passed"]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

model = LogisticRegression()
model.fit(X_train, y_train)

print(model.predict(X_test))               # predicted classes (0 or 1)
print(model.predict_proba(X_test))          # predicted probabilities

Setting the Decision Threshold

By default, a predicted probability of 0.5 or higher is classified as "1" (pass), and anything below as "0" (fail) - though this threshold can be adjusted depending on the cost of different types of mistakes.

Despite having "regression" in its name, Logistic Regression is fundamentally a classification technique - it estimates the probability of belonging to a class, not a continuous numeric outcome.

You've Completed This Section