Basics of Probability

Probability measures how likely an event is to occur, expressed as a number between 0 (impossible) and 1 (certain). It forms the mathematical foundation behind statistics, machine learning, and decision-making under uncertainty.

The Basic Formula

For equally likely outcomes, probability is calculated as:

Probability of an event = (Favourable outcomes) / (Total possible outcomes)

# Example: probability of rolling a 4 on a fair six-sided die
P = 1 / 6

Key Terms

• Experiment - an action with an uncertain outcome, like rolling a die.
• Sample Space - the set of all possible outcomes, like {1, 2, 3, 4, 5, 6}.
• Event - a specific outcome or set of outcomes you're interested in, like rolling an even number.

Independent vs Dependent Events

Two events are independent if one doesn't affect the other's probability, such as two separate coin flips. They're dependent if the outcome of one changes the probability of the other, such as drawing cards from a deck without replacement.

A Simple Python Example

import random

# Simulating 10,000 coin flips to estimate probability of heads
flips = [random.choice(["Heads", "Tails"]) for _ in range(10000)]
p_heads = flips.count("Heads") / len(flips)
print(p_heads)   # close to 0.5

You've Completed This Section

This wraps up the foundations of Feature Engineering and Probability - covering what features are, how to engineer them, their benefits, common techniques, and the basics of probability that underpin statistical reasoning in Data Science.