Python Advanced - Machine Learning Pipeline with Scikit-learn

Creating end-to-end machine learning pipelines with Scikit-learn

Scikit-learn is a powerful and widely-used machine learning library in Python. It provides simple and efficient tools for data mining and data analysis, making it easy to create end-to-end machine learning pipelines. This tutorial explores how to create machine learning pipelines using Scikit-learn.

Installing Scikit-learn

To use Scikit-learn, you need to install it using pip:

pip install scikit-learn
            

Loading and Preparing Data

Here is an example of loading and preparing data using Pandas and Scikit-learn:

import pandas as pd
from sklearn.model_selection import train_test_split

# Load the dataset
data = pd.read_csv('path/to/your/dataset.csv')

# Split the data into features and target
X = data.drop('target_column', axis=1)
y = data['target_column']

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
            

Creating a Pipeline

Here is an example of creating a machine learning pipeline with Scikit-learn:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.ensemble import RandomForestClassifier

# Define the pipeline steps
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('pca', PCA(n_components=2)),
    ('classifier', RandomForestClassifier(n_estimators=100))
])

# Fit the pipeline on the training data
pipeline.fit(X_train, y_train)

# Predict using the pipeline on the test data
y_pred = pipeline.predict(X_test)
            

Evaluating the Model

Here is an example of evaluating the model in the pipeline:

from sklearn.metrics import accuracy_score, classification_report

# Calculate the accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.2f}")

# Print the classification report
report = classification_report(y_test, y_pred)
print(report)
            

Cross-Validation

Here is an example of performing cross-validation with the pipeline:

from sklearn.model_selection import cross_val_score

# Perform cross-validation
cv_scores = cross_val_score(pipeline, X, y, cv=5)

# Print the cross-validation scores
print(f"Cross-validation scores: {cv_scores}")
print(f"Mean cross-validation score: {cv_scores.mean():.2f}")
            

Hyperparameter Tuning

Here is an example of performing hyperparameter tuning with GridSearchCV:

from sklearn.model_selection import GridSearchCV

# Define the parameter grid
param_grid = {
    'pca__n_components': [2, 3, 4],
    'classifier__n_estimators': [50, 100, 200]
}

# Initialize GridSearchCV
grid_search = GridSearchCV(pipeline, param_grid, cv=5, scoring='accuracy')

# Fit the model
grid_search.fit(X_train, y_train)

# Get the best parameters
best_params = grid_search.best_params_
print(f"Best Parameters: {best_params}")

# Predict using the best model
best_model = grid_search.best_estimator_
y_pred_best = best_model.predict(X_test)

# Calculate the accuracy
accuracy_best = accuracy_score(y_test, y_pred_best)
print(f"Accuracy with best model: {accuracy_best:.2f}")
            

Feature Engineering

Here is an example of adding feature engineering steps to the pipeline:

from sklearn.preprocessing import PolynomialFeatures

# Define the pipeline steps
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('poly', PolynomialFeatures(degree=2)),
    ('pca', PCA(n_components=2)),
    ('classifier', RandomForestClassifier(n_estimators=100))
])

# Fit the pipeline on the training data
pipeline.fit(X_train, y_train)

# Predict using the pipeline on the test data
y_pred = pipeline.predict(X_test)

# Calculate the accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.2f}")
            

Saving and Loading Pipelines

Here is an example of saving and loading a pipeline:

import joblib

# Save the pipeline
joblib.dump(pipeline, 'pipeline.pkl')

# Load the pipeline
loaded_pipeline = joblib.load('pipeline.pkl')

# Predict using the loaded pipeline
y_pred_loaded = loaded_pipeline.predict(X_test)

# Calculate the accuracy
accuracy_loaded = accuracy_score(y_test, y_pred_loaded)
print(f"Accuracy with loaded pipeline: {accuracy_loaded:.2f}")
            

Summary

In this tutorial, you learned about creating end-to-end machine learning pipelines with Scikit-learn in Python. Scikit-learn provides powerful tools for building, evaluating, and deploying machine learning models. Understanding how to create pipelines, perform cross-validation, tune hyperparameters, engineer features, and save/load pipelines can help you streamline your machine learning workflows and improve model performance.