Skip to content
  • Text Classification with Naive Bayes · Data Science

Text Classification with Naive Bayes

LearnViaNotes @learnvianotes· 2 min read

Classify documents by their words: turn each text into word counts, then let Naive Bayes estimate which class makes those words most likely.

  1. Vectorise

    CountVectorizer builds a vocabulary and turns each document into a vector of word counts (bag of words).

  2. Train

    MultinomialNB learns how often each word appears in each class.

  3. Predict

    For a new text it picks the class with the highest probability, assuming words are independent — the “naive” part.

Python
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB

texts = ["the team won the cricket match", "stocks fell as markets closed lower",
         "a great goal in the final", "the bank raised interest rates",
         "fans cheered the winning shot", "investors sold shares today"]
labels = ["sports", "finance", "sports", "finance", "sports", "finance"]

vectorizer = CountVectorizer()
X = vectorizer.fit_transform(texts)          # sparse matrix: 6 documents × vocabulary
print(X.shape)                               # (6, 27)

model = MultinomialNB().fit(X, labels)
new = vectorizer.transform(["the match was a great win", "interest rates and shares"])
print(model.predict(new).tolist())           # ['sports', 'finance']

The notebook's real dataset

Python
from sklearn.datasets import fetch_20newsgroups
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.model_selection import train_test_split
from sklearn import metrics

data = fetch_20newsgroups(subset="all")                  # ~18,800 posts in 20 topics (downloads once)
X = CountVectorizer().fit_transform(data.data)
X_train, X_test, y_train, y_test = train_test_split(X, data.target, test_size=0.3, random_state=0)
model = MultinomialNB().fit(X_train, y_train)
y_pred = model.predict(X_test)
print(f"Accuracy: {metrics.accuracy_score(y_test, y_pred):.2f}")   # 0.85 in the notebook run
print(metrics.classification_report(y_test, y_pred, target_names=data.target_names))

Free for students and teachers

Write notes like this one

Cards for every idea, quizzes and flashcards, read aloud — on the web and your phone.

Something wrong with this note? Open it to report