Bag of Words

The Bag of Words (BoW) model represents a piece of text as an unordered collection ("bag") of its words, along with how many times each word appears - completely ignoring grammar, word order, and sentence structure.

How Bag of Words Works

Given a set of documents, BoW builds a vocabulary of all unique words across them, then represents each document as a vector of word counts against that vocabulary.

from sklearn.feature_extraction.text import CountVectorizer

docs = ["data science is fun", "python for data science"]
vectorizer = CountVectorizer()
bow = vectorizer.fit_transform(docs)

print(vectorizer.get_feature_names_out())
print(bow.toarray())

Why It's Useful

• Simple and fast to compute
• Converts text into a numeric format that machine learning models can use directly
• Works reasonably well for tasks like spam detection and basic text classification

Limitations

Because BoW ignores word order, "not good" and "good not" produce the same representation - losing meaning that depends on sequence and context. It can also produce very large, sparse vectors when the vocabulary is big.