Build Your First Machine Learning Pipeline Using Scikit-learn

Machine learning is a powerful tool for making predictions from data. However, getting from raw data to a deployable model involves many steps. You need to preprocess the data, train a model, evaluate its performance, and more. Doing this manually every time is tedious and error-prone.

This is where machine learning pipelines come in. A pipeline bundles up the workflow into one easy-to-use object. You can use it to train models and make predictions in a standardized, repeatable way. Pipelines improve code organization and make it easier to deploy your model.

In this article, you‘ll learn how to build your first machine learning pipeline using the popular scikit-learn library in Python. We‘ll cover:

• The main steps in a typical ML workflow
• How to build a pipeline for a regression problem
• Evaluating the pipeline‘s performance
• Using the pipeline to make predictions on new data

We‘ll work through an example using a real-world dataset to solidify the concepts. Let‘s dive in!

The Machine Learning Workflow

Before we start building a pipeline, let‘s review the typical machine learning workflow. While the exact steps may vary, a typical workflow looks like this:

1. Preprocess the data
2. Split data into training and test sets
3. Train the model on the training data
4. Evaluate model performance on the test data
5. Retrain the model on the full dataset
6. Use the trained model to make predictions on new data

The preprocessing step is very important. Raw data is messy – it often has missing values, outliers, features on different scales, categorical variables, and so on. You need to clean and transform the data before a model can learn from it.

Some common preprocessing steps are:

• Imputing missing values
• Scaling features to a consistent range
• Encoding categorical variables as numbers
• Selecting relevant features
• Handling outliers

In scikit-learn, there are classes like SimpleImputer, StandardScaler, and OneHotEncoder to help with these tasks.

After preprocessing, you split the data into training and test sets. The model learns from the training set and its performance is evaluated on the unseen test set. This gives an estimate of how well it generalizes to new data.

With the data prepared, you train the model on the training set. There are many algorithms to choose from like linear regression, decision trees, neural networks, etc. The model learns patterns from the training data.

Next, you evaluate the trained model‘s performance on the test set. Metrics like accuracy, precision, recall, F1 score, or mean squared error tell you how well the model is doing. If performance is not satisfactory, you can try a different algorithm, tune the hyperparameters, or change the preprocessing.

If you‘re happy with the model‘s performance, you can retrain it on the full dataset to make use of all available data. Finally, you use the trained model to make predictions on new, unseen data. And that‘s it!

As you can see, there are many steps before we get to the "fun" part of making predictions. Doing all of this manually every time we want to build a model would be very tedious and time-consuming. This is where pipelines save the day.

Building an ML Pipeline

Let‘s see how to implement the workflow in a pipeline. We‘ll use scikit-learn since it has many useful classes for this. We‘ll work through an example using the Big Mart Sales dataset.

The dataset has sales data for 1559 products across 10 stores in different cities. The goal is to build a model to predict the sales of each product at a particular store. You can download the dataset from here.

Here‘s a sample of the data:

 Item_Identifier  Item_Weight  Item_Fat_Content  Item_Visibility    Item_Type  Item_MRP Outlet_Identifier  Outlet_Establishment_Year Outlet_Size Outlet_Location_Type  Outlet_Type  Item_Outlet_Sales
0           FDA15        9.300                LF         0.016047  Dairy             249.8092            OUT049                       1999      Medium               Tier 1  Supermarket Type1          3735.1380
1           DRC01        5.920                 R         0.019278  Soft Drinks        48.2692            OUT018                       2009      Medium               Tier 3  Supermarket Type2           443.4228
2           FDN15       17.500                LF         0.016760  Meat              141.6180            OUT049                       1999      Medium               Tier 1  Supermarket Type1          2097.2700
3           FDX07       19.200                 R         0.000000  Fruits and Vegetables           182.0950            OUT010                       1998         NaN               Tier 3      Grocery Store           732.3800
4           NCD19        8.930                LF         0.000000  Household         53.8614            OUT013                       1987        High               Tier 3  Supermarket Type1           994.7052

The first step is to load the data into a pandas DataFrame and take a look:

import pandas as pd

data = pd.read_csv("Train_BigMart.csv")
print(data.head())

There are a few issues we need to address in the preprocessing step:

• The Item_Fat_Content column has categories like "LF", "reg", and "low fat". We need to standardize these.
• The Item_Visibility has a min value of zero, which doesn‘t make sense. We‘ll impute zero values with the mean.
• The Item_Weight column has some missing values. We‘ll impute them with the mean weight.
• The Outlet_Size has missing values. We‘ll impute with the mode.
• Some columns like Item_Identifier and Outlet_Identifier are not useful for the model. We‘ll drop them.

To preprocess, we‘ll build a custom transformer to clean up the Item_Fat_Content and Item_Visibility columns. For the rest, we can use scikit-learn‘s ColumnTransformer with SimpleImputer and OneHotEncoder:

from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline

class FeatureCorrector(BaseEstimator, TransformerMixin):
    def fit(self, X, y=None):
        return self

    def transform(self, X, y=None):
        X.loc[X[‘Item_Visibility‘] == 0, ‘Item_Visibility‘] = X[‘Item_Visibility‘].mean()
        X.loc[X[‘Item_Fat_Content‘] == ‘LF‘ ,‘Item_Fat_Content‘] = ‘Low Fat‘
        X.loc[X[‘Item_Fat_Content‘] == ‘reg‘ ,‘Item_Fat_Content‘] = ‘Regular‘
        X.loc[X[‘Item_Fat_Content‘] == ‘low fat‘ ,‘Item_Fat_Content‘] = ‘Low Fat‘
        return X

preprocessor = ColumnTransformer(transformers=[
    (‘imputer‘, SimpleImputer(strategy=‘mean‘), [‘Item_Weight‘]),
    (‘imputer2‘, SimpleImputer(strategy=‘most_frequent‘), [‘Outlet_Size‘]),  
    (‘ohe‘, OneHotEncoder(handle_unknown=‘ignore‘), [‘Item_Fat_Content‘, ‘Outlet_Location_Type‘, ‘Outlet_Size‘, ‘Outlet_Type‘, ‘Item_Type‘])
])

prep_pipeline = Pipeline(steps=[
    (‘correct‘, FeatureCorrector()),
    (‘preprocessor‘, preprocessor)
])

data_cleaned = prep_pipeline.fit_transform(data)

Here‘s what we did:

1. Defined a custom FeatureCorrector transformer to fix the Item_Fat_Content and Item_Visibility columns. In the transform method, we replaced zeros in Item_Visibility with the mean, and standardized the categories in Item_Fat_Content.

2. Set up a ColumnTransformer to apply different preprocessing to different columns. We used SimpleImputer to fill in missing values in Item_Weight with the mean, and in Outlet_Size with the mode. OneHotEncoder handles the categorical columns.

3. Put the custom transformer and ColumnTransformer in a Pipeline. This is our preprocessing pipeline.

4. Applied the preprocessing pipeline to our data to get the cleaned data.

With the data preprocessed, we can split it into training and test sets:

from sklearn.model_selection import train_test_split

X = data_cleaned.drop(‘Item_Outlet_Sales‘, axis=1)
y = data_cleaned[‘Item_Outlet_Sales‘]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Now we‘re ready to train a model. We‘ll use a simple linear regression model first:

from sklearn.linear_model import LinearRegression

model = LinearRegression()
model.fit(X_train, y_train)

Let‘s evaluate its performance on the test set:

from sklearn.metrics import mean_squared_error, r2_score

preds = model.predict(X_test)
mse = mean_squared_error(y_test, preds)
r2 = r2_score(y_test, preds)

print(f"Mean Squared Error: {mse:.2f}")  
print(f"R^2 Score: {r2:.2f}")

This gives:

Mean Squared Error: 1807150.58
R^2 Score: 0.57

The R-squared value of 0.57 means the model explains 57% of the variance in the sales data. There‘s room for improvement.

Let‘s try a different model. We‘ll use a random forest regressor:

from sklearn.ensemble import RandomForestRegressor

model = RandomForestRegressor(n_estimators=100, random_state=42)  
model.fit(X_train, y_train)

preds = model.predict(X_test)
mse = mean_squared_error(y_test, preds)
r2 = r2_score(y_test, preds)

print(f"Mean Squared Error: {mse:.2f}")
print(f"R^2 Score: {r2:.2f}")

This gives:

Mean Squared Error: 1049742.86 
R^2 Score: 0.61

The random forest performs better with an R-squared of 0.61. We could continue experimenting with other algorithms, but let‘s say we‘re happy with this model for now.

The final step is to put everything together into a single pipeline that includes both the preprocessing and model training steps:

final_pipeline = Pipeline(steps=[
    (‘preprocessor‘, prep_pipeline),
    (‘model‘, RandomForestRegressor(n_estimators=100, random_state=42))
])

final_pipeline.fit(data, y)

And that‘s it! We‘ve built an end-to-end machine learning pipeline that takes in the raw data, preprocesses it, trains a model, and spits out predictions. We can use this pipeline to make predictions on new data:

test_data = pd.read_csv("Test_BigMart.csv") 
test_preds = final_pipeline.predict(test_data)

Some benefits of using pipelines are:

• All the preprocessing and model training steps are bundled together. We don‘t have to run them separately every time.
• Less chance of data leakage, i.e., the model seeing data during training that it shouldn‘t. The pipeline takes care of applying the same transformations to the training and test data.
• Easier to productionize the model. We can just save the entire pipeline object and use it for predictions in a production setting.

Wrapping Up

In this article, we learned how to build a machine learning pipeline using scikit-learn. We walked through an example using the Big Mart Sales dataset to predict sales. The key steps were:

1. Preprocess the data using scikit-learn transformers like SimpleImputer and OneHotEncoder
2. Split the data into training and test sets
3. Train and evaluate models
4. Put the preprocessing and model training steps together in a pipeline

Pipelines offer a clean and concise way to organize your machine learning workflow. They abstract away a lot of the repetitive work and let you focus on iterating quickly.

There are some best practices to keep in mind when building pipelines:

• Keep the data preprocessing and model training steps separate. This makes the pipeline more modular.
• Use custom transformers for specialized preprocessing that scikit-learn transformers don‘t cover.
• Experiment with different models in the pipeline to see which performs best.
• Use GridSearchCV or RandomizedSearchCV to find the best hyperparameters for your models.

In a real project, you would also spend more time on exploratory data analysis, feature engineering, and model evaluation. Pipelines help by providing a standard framework to plug these steps into.

I encourage you to try building pipelines on other datasets and experiment with different preprocessing techniques and models. With practice, constructing ML pipelines will become second nature.

I hope this article gave you a good starting point to build your own pipelines. Let me know in the comments if you have any other tips or best practices to share!