Text Classification Tutorial

Introduction to Text Classification

Text classification is the process of assigning predefined categories to unstructured text data. This technique is widely used in various applications such as sentiment analysis, spam detection, topic labeling, and more. The primary goal of text classification is to facilitate the organization and retrieval of information.

How Text Classification Works

The process of text classification generally involves the following steps:

1. Data Collection: Gathering a dataset that contains text data and corresponding labels.
2. Data Preprocessing: Cleaning and preparing the data for analysis (e.g., removing punctuation, converting to lowercase).
3. Feature Extraction: Transforming text data into numerical features that can be used by machine learning algorithms (e.g., using TF-IDF or word embeddings).
4. Model Training: Using a machine learning algorithm to train a model on the labeled dataset.
5. Model Evaluation: Assessing the performance of the trained model using metrics such as accuracy, precision, recall, and F1-score.
6. Prediction: Using the model to classify new, unseen text data.

Setting Up Your Environment

To perform text classification, we will use the Natural Language Toolkit (NLTK) library in Python. First, ensure you have Python installed on your system. Then, install NLTK using pip:

After installing NLTK, you may also want to download some necessary datasets:

Data Preparation

Next, we need to prepare our dataset. For this example, we will create a simple dataset for spam detection:

                [
                    ("Congratulations! You've won a $1,000 Walmart gift card.", "spam"),
                    ("Hey, are we still meeting for lunch today?", "ham"),
                    ("Limited time offer! Click here to claim your prize.", "spam"),
                    ("Can you send me the report by tomorrow?", "ham")
                ]
                

In this dataset, "spam" refers to unwanted messages, while "ham" refers to legitimate messages.

Data Preprocessing

Before training our model, we need to preprocess the text data. This involves tokenization and removing stopwords:

from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize

# Sample text
text = "Congratulations! You've won a $1,000 Walmart gift card."

# Tokenization
tokens = word_tokenize(text)

# Removing stopwords
stop_words = set(stopwords.words('english'))
filtered_tokens = [word for word in tokens if word.lower() not in stop_words]

print(filtered_tokens)
                

The output will show the tokens after removing stopwords.

Feature Extraction

We will use the Term Frequency-Inverse Document Frequency (TF-IDF) method for feature extraction. This converts text data into a numerical format that can be used by machine learning algorithms:

from sklearn.feature_extraction.text import TfidfVectorizer

# Sample dataset
documents = [
    "Congratulations! You've won a $1,000 Walmart gift card.",
    "Hey, are we still meeting for lunch today?",
    "Limited time offer! Click here to claim your prize.",
    "Can you send me the report by tomorrow?"
]

# Initialize TF-IDF Vectorizer
vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(documents)

print(X.toarray())
                

This code will convert the sample text documents into a matrix of TF-IDF features.

Model Training

Now, we will train a classification model using the extracted features. We can use the Multinomial Naive Bayes classifier for this task:

from sklearn.naive_bayes import MultinomialNB
from sklearn.model_selection import train_test_split

# Labels
labels = ["spam", "ham", "spam", "ham"]

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, labels, test_size=0.25, random_state=42)

# Train the model
model = MultinomialNB()
model.fit(X_train, y_train)
                

Model Evaluation

After training the model, we need to evaluate its performance using the test set:

from sklearn.metrics import accuracy_score, classification_report

# Make predictions
y_pred = model.predict(X_test)

# Evaluate the model
accuracy = accuracy_score(y_test, y_pred)
report = classification_report(y_test, y_pred)

print("Accuracy:", accuracy)
print(report)
                

The accuracy score will give you a measure of how well the model performs, while the classification report provides detailed metrics.

Making Predictions

Finally, we can use our trained model to make predictions on new, unseen text data:

new_samples = [
    "Win a free iPhone now!",
    "Let's catch up over coffee."
]

# Transform the new samples using the same vectorizer
new_X = vectorizer.transform(new_samples)

# Predict categories
predictions = model.predict(new_X)

print(predictions)
                

This code will output the predicted categories for the new samples.

Conclusion

Text classification is a powerful technique that can be applied to various problems in natural language processing. By following this tutorial, you have learned the fundamental steps of text classification, including data preparation, preprocessing, feature extraction, model training, evaluation, and making predictions. With these skills, you can apply text classification to your own projects and enhance your understanding of machine learning.