Iris Flower Classification: A Classic Machine Learning Problem

The iris flower dataset is one of the most famous datasets in machine learning. Despite its small size, it serves as an excellent introduction to classification problems. In this post, we‘ll explore the iris dataset in depth and walk through the process of building machine learning models to classify iris flower species based on their measurements.

The Iris Dataset

The iris flower dataset contains measurements of 150 iris flowers from three different species – setosa, versicolor, and virginica. There are 50 flowers from each species.

The dataset includes the following features for each flower:

• Sepal length in centimeters
• Sepal width in centimeters
• Petal length in centimeters
• Petal width in centimeters

Based on these four measurements, the goal is to predict the species of the iris flower. This is a classification task, since the target variable, species, is categorical rather than numeric.

The three iris species in the famous dataset. Image source: Wikimedia Commons

Exploratory Data Analysis

Before diving into building machine learning models, it‘s important to explore and visualize the iris dataset. This can reveal insights about the data distribution and relationships between the features.

Some things we can plot to visualize the iris data:

• Histograms of each feature to see the distribution
• Scatterplots of the features against each other to look for separability between the classes
• Box plots of the features for each species to compare the distributions

Example scatterplot showing the separability of the three iris species. The setosa flowers are distinct, while there is some overlap between versicolor and virginica.

From the exploratory plots, we can see that the three species have somewhat distinct distributions of petal and sepal sizes, especially for the setosa flowers. This suggests that a machine learning model should be able to learn to differentiate between the species based on these features.

Preprocessing the Data

With the iris dataset, we‘re lucky that it‘s already in good shape for machine learning. However, there are still a few preprocessing steps we should do:

1. Check for any missing values and handle them appropriately (the iris dataset has no missing values, but this is always good to check).

2. Convert the species names to numeric values since most machine learning models require numeric input. We can map setosa to 0, versicolor to 1, and virginica to 2.

3. Split the data into training and test sets. We‘ll use the training set to build the models and the test set to evaluate their performance on unseen data. A typical split is 80% for training and 20% for testing.

Building Classification Models

Now we‘re ready to train some machine learning models on the iris dataset! We‘ll try out several common classification algorithms:

Logistic Regression

Logistic regression is a classic model for binary classification, but it can be extended to multi-class problems like the iris dataset. It learns a linear decision boundary to separate the classes.

K-Nearest Neighbors (KNN)

The KNN algorithm makes predictions based on the majority class of the K closest training examples. It‘s a simple but powerful model that can learn complex decision boundaries.

Decision Trees

Decision trees learn a series of split rules to partition the feature space into regions corresponding to the different classes. They‘re easy to interpret but prone to overfitting.

Support Vector Machines (SVM)

SVMs try to find the hyperplane that maximally separates the classes. They can efficiently learn non-linear decision boundaries using the kernel trick.

To train and evaluate each model, we‘ll use the scikit-learn library in Python. Here‘s a quick example of training a logistic regression model:

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

lr_model = LogisticRegression()
lr_model.fit(X_train, y_train)

pred = lr_model.predict(X_test)
accuracy = accuracy_score(y_test, pred)
print(f‘Logistic Regression Accuracy: {accuracy:.3f}‘)

We simply create an instance of the LogisticRegression class, fit it to the training data, and then use it to make predictions on the test set. We can evaluate the model‘s performance by computing the accuracy, which is the fraction of correct predictions.

Comparing Model Performance

After training all the models, we can compare their performance on the test set. Some useful metrics for classification problems include:

• Accuracy: The overall fraction of correct predictions
• Precision: The fraction of true positive predictions among all positive predictions
• Recall: The fraction of true positives captured by the model
• F1 score: The harmonic mean of precision and recall

We can easily compute these metrics using scikit-learn. Here‘s an example classification report for the logistic regression model:

              precision    recall  f1-score   support

      setosa       1.00      1.00      1.00        19
  versicolor       0.94      0.89      0.91        18
   virginica       0.90      0.95      0.93        19

    accuracy                           0.95        56
   macro avg       0.95      0.95      0.95        56
weighted avg       0.95      0.95      0.95        56

This tells us that the model achieved 100% precision and recall for the setosa class, but had a bit more difficulty distinguishing between versicolor and virginica. The overall accuracy is 95%, which is quite good!

We can also visualize the models‘ decision boundaries to get a more intuitive sense of how they‘re separating the classes. The following plot shows the decision regions learned by a KNN model:

Example decision regions learned by a KNN classifier on the iris dataset. The model separates the feature space into regions for each iris species.

Optimizing Models with Hyperparameter Tuning

Most machine learning models have hyperparameters that control their behavior, such as the K in KNN or the regularization strength in logistic regression. To get the best performance, we need to search for the optimal hyperparameter values on our specific dataset.

A simple approach is grid search, which tries out all combinations of hyperparameters in a specified range. We can automate this in scikit-learn like so:

from sklearn.model_selection import GridSearchCV

param_grid = {‘C‘: [0.1, 1, 10], ‘penalty‘: [‘l1‘, ‘l2‘]}
lr_model = LogisticRegression()

grid_search = GridSearchCV(lr_model, param_grid, cv=5)
grid_search.fit(X_train, y_train)

print(f‘Best parameters: {grid_search.best_params_}‘) 
print(f‘Best score: {grid_search.best_score_:.3f}‘)

Here we define a grid of hyperparameters for logistic regression and do a search to find the best combination using 5-fold cross-validation. The best_params_ and best_score_ attributes give us the optimal hyperparameters and the corresponding model performance.

Tuning the hyperparameters can significantly improve the performance compared to the default values, so it‘s always worth trying, especially for complex models like SVMs or neural networks.

Limitations and Future Work

While the iris dataset is a great testbed for machine learning models, it‘s important to keep in mind its limitations:

• It‘s a very small dataset with only 150 examples. In practice, datasets can be orders of magnitude larger.
• The classes are relatively balanced and easy to separate. Real-world problems often have imbalanced classes and more subtle differences between them.
• The features were carefully measured in a controlled setting. Data in the wild is often noisy and incomplete.

Nonetheless, the same basic workflow we used here can be applied to tackle more complex classification problems. Some possible extensions to the iris classification task include:

• Trying out more advanced models like random forests, gradient boosting, or neural networks
• Doing feature selection to identify the most informative features
• Training an ensemble of different models and combining their predictions
• Deploying the trained model as a web application or API for others to use

Conclusion

In this post, we walked through the key steps of a classification problem using the classic iris flower dataset:

1. Exploring and visualizing the data
2. Preprocessing the data for machine learning
3. Training and evaluating different classification models
4. Tuning the hyperparameters to optimize performance

We saw that even with this simple dataset, we can build highly accurate models to predict the species of iris flowers. I encourage you to try out the code yourself and experiment with different models and techniques.

Classification is a fundamental task in machine learning with countless applications in areas like healthcare, finance, marketing, and more. I hope this post gave you a taste of the power and potential of machine learning for classification problems. Happy modeling!