KNN Classifier

The K-Nearest Neighbors (KNN) Classifier predicts the class of a new data point by looking at the classes of its 'k' closest neighbors in the training data and taking a majority vote.

How KNN Works

1. Choose a value for k
2. Calculate the distance from the new point to every training point
3. Select the k closest points
4. Assign the class that appears most among those neighbors

Implementing in Python

import pandas as pd
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split

df = pd.DataFrame({
    "Height": [150, 155, 160, 165, 170, 175, 180, 185],
    "Weight": [50, 55, 60, 65, 70, 75, 80, 85],
    "Category": ["Light", "Light", "Light", "Medium", "Medium", "Medium", "Heavy", "Heavy"]
})

X = df[["Height", "Weight"]]
y = df["Category"]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

model = KNeighborsClassifier(n_neighbors=3)
model.fit(X_train, y_train)

print(model.predict(X_test))

Distance Metrics

KNN typically uses Euclidean distance to measure closeness, though Manhattan and Minkowski distances are also common choices depending on the nature of the data.

Coming Up Next

Next, you'll look at where KNN is actually used in real-world applications.