TPOT: Automating ML Pipelines with Genetic Algorithms in Python

Photo by author

# Introduction

Building a machine learning model involves a long series of manual decisions. There are many steps involved, such as cleaning the data, choosing the right algorithm, and tuning the hyperparameters to achieve good results. This trial and error process often takes hours or even days. However, there is a way around this problem using A tree-based pipeline optimization toolor tpot.

TPOT is a Python library that uses genetic algorithms to automatically find the optimal machine learning pipeline. It treats pipelines like populations in nature: it tries many combinations, evaluates their performance, and “breeds” the best ones over several generations. This automation allows you to focus on solving your problem while TPOT handles the technical details of model selection and optimization.

# How does TPOT work?

TPOT uses genetic programming (GP). It is a type of evolutionary algorithm inspired by natural selection in organisms. Instead of developing organisms, GPs develop computer programs or workflows to solve a problem. In the context of TPOT, the “programs” being developed are machine learning pipelines.

TPOT works in four main steps:

1. Create pipelines: It starts with a random population of machine learning pipelines, including preprocessing methods and models.
2. Assess fitness: Each pipeline is trained on the data and evaluated to measure performance.
3. Selection and Evolution: The best-performing pipelines are selected for “reintroduction” and to create new pipelines through crossover and mutation.
4. Repetition over generations: This process repeats for several generations until TPOT identifies the pipeline with the best performance.

The process is visualized in the diagram below:

Next, we’ll see how to configure and use TPOT in Python.

# 1. Installing TPOT

To install TPOT, run the following command:

# 2. Importing Libraries

Import the required libraries:

from tpot import TPOTClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# 3. Loading and distribution of data

We will use the popular IRIS dataset for this example:

iris = load_iris()
X, y = iris.data, iris.target

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

load_iris() Provides function properties X and labels y. train_test_split The function maintains a test set so you can measure final performance on unseen data. This creates an environment where pipelines will be evaluated. All pipelines are trained on the training section and internally validated.

Note: TPOT uses internal cross-validation during fitness evaluation.

# 4. Starting the TPOT

Start TPOT as follows:

tpot = TPOTClassifier(
    generations=5,
    population_size=20,
    random_state=42
)

You can control how long and how widely TPOT searches for a good pipeline. For example:

• Generations = 5 This means that TPOT will run five cycles of evolution. In each cycle, it generates a new set of candidate pipelines based on the previous generation.
• population_size = 20 This means there are 20 candidate pipelines in each generation.
• random_stat Ensures that results are reproducible.

# 5. Model training

Train the model by running this command:

tpot.fit(X_train, y_train)

When you run tpot.fit(X_train, y_train)TPOT started searching for the best pipeline. It creates a group of candidate pipelines, trains each one to see how well it performs (usually using cross-validation), and retains the top performers. After that, it mixes and slightly changes them to form a new group. This cycle repeats the number of generations you set. TPOT always remembers which pipeline has performed best so far.

Output:

# 6. Assessing accuracy

This is your final check on how the selected pipeline behaves on the observed data. You can calculate the accuracy as follows:

y_pred = tpot.fitted_pipeline_.predict(X_test)
acc = accuracy_score(y_test, y_pred)
print("Accuracy:", acc)

Output:

# 7. Exporting the best pipeline

You can export the pipeline to a file for later use. Note that we should import dump from JOBBLIB First:

from joblib import dump

dump(tpot.fitted_pipeline_, "best_pipeline.pkl")
print("Pipeline saved as best_pipeline.pkl")

joblib.dump() As stores the entire fit model best_pipeline.pkl.

Output:

Pipeline saved as best_pipeline.pkl

You can load it later as follows:

from joblib import load

model = load("best_pipeline.pkl")
predictions = model.predict(X_test)

This makes your model reusable and easy to deploy.

# wrap up

In this article, we saw how machine learning pipelines can be automated using genetic programming, and we also walk through a practical example of implementing TPOT in Python. For further search, please consult it Documents.

Kanwal Mehreen is a machine learning engineer and technical writer with a deep passion for data science and the intersection of AI with medicine. He co-authored the eBook “Maximizing Productivity with ChatGPT.” As a 2022 Google Generation Scholar for APAC, she champions diversity and academic excellence. He has also been recognized as a Teradata Diversity in Tech Scholar, a MITACS GlobalLink Research Scholar, and a Harvard Wicked Scholar. Kanwal is a passionate advocate for change, having founded the Fame Code to empower women in stem fields.