Perceptron Algorithm

The Perceptron Algorithm is the original learning procedure used to train a single perceptron - it works by repeatedly adjusting the weights whenever the model makes a wrong prediction.

Step-by-Step Algorithm

1. Initialize weights (and bias) to zero or small random values
2. For each training example:
     a. Compute the predicted output using current weights
     b. Compare prediction to the actual label
     c. If the prediction is wrong, update the weights
3. Repeat for a fixed number of epochs, or until predictions stop improving

The Update Rule

wi_new = wi_old + learning_rate * (actual - predicted) * xi

If the prediction is correct, the weights don't change at all. If it's wrong, the weights shift in the direction that would have made the correct prediction more likely.

Implementing in Python

from sklearn.linear_model import Perceptron

model = Perceptron(max_iter=1000, eta0=0.1, random_state=42)
model.fit(X_train, y_train)

print("Accuracy:", model.score(X_test, y_test))

Convergence

The Perceptron Convergence Theorem guarantees that if the data is linearly separable, the algorithm will find a set of weights that perfectly classifies it in a finite number of steps. If the data isn't linearly separable, it will never converge.

The perceptron algorithm only updates weights on mistakes - correctly classified points are left untouched, making it a fairly efficient, "mistake-driven" learning process.

Coming Up Next