Predicting Wind Speed with K-Nearest Neighbors in Python

Wind speed is a critical variable for many applications, from weather forecasting to renewable energy production. Accurate predictions of wind speed can help wind farm operators optimize power generation, enable more efficient scheduling of maintenance, and ultimately lower costs. In this post, we‘ll walk through how to build a machine learning model to predict wind speed using the k-nearest neighbors algorithm in Python.

Understanding K-Nearest Neighbors

K-nearest neighbors (k-NN) is a simple yet powerful supervised machine learning algorithm used for both classification and regression tasks. The core idea is to make predictions for a data point based on the known values of its "neighboring" points.

Here are the key steps in the k-NN algorithm:

1. Calculate the distance between the point you want to make a prediction for and all points in the training data. Common distance metrics include Euclidean, Manhattan, and Minkowski distance.

2. Identify the k training data points closest to your target point (the "nearest neighbors").

3. For regression, take the average value of the k nearest neighbors as the prediction. For classification, take a majority vote among the neighbors.

The k in k-NN refers to the number of neighboring points to consider, which is a key hyperparameter that needs to be tuned for best performance. In general, a larger k reduces noise and smooths out decision boundaries but risks underfitting, while a smaller k allows for more complex boundaries but risks overfitting.

Exploring the Wind Speed Dataset

For this tutorial, we‘ll use a dataset of weather measurements collected from various locations, with the goal of predicting wind speed based on other weather variables. Here‘s a sample of the raw data:


air_pressure air_temp avg_wind_direction max_wind_direction min_wind_direction rainfall max_wind_speed wind_speed
0 1019.69 25.12 177.46 215.10 68.98 0.00000 2.2582 1.508400
1 1018.02 29.30 182.09 254.51 73.69 0.00000 3.3358 2.443518
2 1018.25 28.87 180.91 259.75 79.97 0.00000 3.2592 2.303075
3 1017.11 29.32 176.69 196.50 70.54 0.00000 2.6672 1.920700
4 1016.84 29.54 190.29 259.03 79.97 0.00000 3.4215 1.680975


The dataset contains the following columns:

• air_pressure: atmospheric pressure in millibars
• air_temp: air temperature in °C
• avg_wind_direction: wind direction in degrees (averaged over 10 min intervals)
• max_wind_direction: max wind direction in degrees (over 10 min intervals)
• min_wind_direction: min wind direction in degrees (over 10 min intervals)
• rainfall: amount of rainfall in mm
• max_wind_speed: maximum wind speed in m/s (over 10 min intervals)
• wind_speed: average wind speed in m/s (over 10 min intervals) – this is our target variable to predict

Let‘s load the data into a Pandas DataFrame and take a look at the shape and basic statistics:


import pandas as pd

data = pd.read_csv(‘wind_speed_data.csv‘)

print(data.shape)
data.describe()


(10000, 8)
air_pressure air_temp … max_wind_speed wind_speed
count 10000.0000 10000.000000 … 10000.000000 10000.000000
mean 986.8746 27.788540 … 4.531227 2.926936
std 26.6621 4.864904 … 1.840736 1.396811
min 883.3900 16.900000 … 0.922754 0.257818
25% 964.5100 24.207500 … 3.067425 1.814450
50% 987.1000 27.515000 … 4.227300 2.647650
75% 1004.7275 30.617500 … 5.804100 3.859175
max 1069.4800 40.410000 … 10.471820 8.430425

The dataset contains 10,000 rows and 8 columns. There are no missing values, and the wind speeds range from about 0.26 to 8.43 m/s with an average of 2.93 m/s.

Let‘s visualize the distributions of the numeric features and their relationships with wind speed using seaborn:


import seaborn as sns
import matplotlib.pyplot as plt

sns.pairplot(data,
x_vars=[‘air_pressure‘, ‘air_temp‘, ‘avg_wind_direction‘,
‘rainfall‘, ‘max_wind_speed‘],
y_vars=[‘wind_speed‘],
kind=‘scatter‘, height=4, aspect=1);


From the pairplot, we can see that wind speed has the strongest positive linear relationship with max_wind_speed, while the other features have weaker and mostly non-linear relationships. This suggests max_wind_speed will likely be the most important predictor.

Preparing the Data for Modeling

Before training our k-NN model, we need to do some data cleaning and preprocessing:

1. The columns min_wind_direction and max_wind_direction have a lot of overlap with avg_wind_direction, so we‘ll drop them to avoid redundant features.

2. We‘ll scale all features to a consistent range (between 0 and 1) using scikit-learn‘s MinMaxScaler. This is important for k-NN since it relies on distance calculations, and we don‘t want some features to dominate just because they have larger numeric values.

3. We‘ll randomly split the data into training and test sets, using 80% for training and 20% for final model evaluation.

Here‘s the code to perform these steps:


from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split

data_input = data.drop([‘min_wind_direction‘,‘max_wind_direction‘], axis=1)

X = data_input.drop(‘wind_speed‘, axis=1)
y = data_input[‘wind_speed‘]

scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)

X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.2, random_state=42)


Training and Evaluating the K-NN Model

With our data prepared, we‘re ready to train the k-NN model using scikit-learn. The key hyperparameter we need to tune is n_neighbors, which controls the number of neighboring points the model considers.

Let‘s start by fitting k-NN models with a range of n_neighbors values and comparing their performance on the test set using mean absolute error (MAE), root mean squared error (RMSE), and R^2:


from sklearn.neighbors import KNeighborsRegressor
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

def evaluate_model(model, X_test, y_test):
y_pred = model.predict(X_test)
mae = mean_absolute_error(y_test, y_pred)
rmse = mean_squared_error(y_test, y_pred, squared=False)
r2 = r2_score(y_test, y_pred)

print(f"Mean Absolute Error: {mae:.3f}")
print(f"Root Mean Squared Error: {rmse:.3f}")
print(f"R^2: {r2:.3f}")

k_values = [1, 3, 5, 10, 20, 50]

for k in k_values:
print(f"\nk = {k}")
model = KNeighborsRegressor(n_neighbors=k)
model.fit(X_train, y_train)
evaluate_model(model, X_test, y_test)


k = 1
Mean Absolute Error: 0.345
Root Mean Squared Error: 0.610
R^2: 0.810

k = 3
Mean Absolute Error: 0.303
Root Mean Squared Error: 0.529
R^2: 0.844

k = 5
Mean Absolute Error: 0.292
Root Mean Squared Error: 0.509
R^2: 0.853

k = 10
Mean Absolute Error: 0.288
Root Mean Squared Error: 0.503
R^2: 0.855

k = 20
Mean Absolute Error: 0.287
Root Mean Squared Error: 0.505
R^2: 0.854

k = 50
Mean Absolute Error: 0.287
Root Mean Squared Error: 0.508
R^2: 0.853

The results show that model performance generally improves as we increase n_neighbors up to around 10, and then starts to plateau or slightly decline after that. The model with k=10 achieves the best test MAE of 0.288 m/s and R^2 of 0.855, so we‘ll consider that our final tuned model.

Making Predictions on New Data

Now that we have a trained and tuned k-NN model, we can use it to predict wind speed for new unseen data points. To do this, we simply need to pass a DataFrame of new data points (scaled using the same MinMaxScaler) to the model‘s predict method.

For example, let‘s predict the wind speed for a new data point with the following weather conditions:

• Air pressure: 990 mb
• Air temperature: 28°C
• Average wind direction: 180°
• Rainfall: 0 mm
• Max wind speed: 5 m/s


new_data = pd.DataFrame({‘air_pressure‘: [990],
‘air_temp‘: [28],
‘avg_wind_direction‘: [180],
‘rainfall‘: [0],
‘max_wind_speed‘: [5]})

new_data_scaled = scaler.transform(new_data)

final_model = KNeighborsRegressor(n_neighbors=10)
final_model.fit(X_train, y_train)

wind_speed_prediction = final_model.predict(new_data_scaled)[0]

print(f"Predicted Wind Speed: {wind_speed_prediction:.3f} m/s")


Predicted Wind Speed: 3.482 m/s

Our k-NN model with k=10 predicts a wind speed of about 3.48 m/s for the given weather conditions, which seems reasonable given the patterns we observed in the training data.

Pros and Cons of K-NN for Wind Speed Prediction

K-nearest neighbors has a few key strengths that make it a good choice for this wind speed prediction task:

• It‘s simple to understand and implement, with only one main tuning parameter (k).
• It naturally handles interactions between features without needing to specify them explicitly.
• It can fit highly non-linear patterns in the data.
• Decent performance with minimal tuning on this dataset.

However, k-NN also has some important limitations to be aware of:

• Predictions can be sensitive to the scale of the features, so careful normalization is required.
• It gets significantly slower as the training dataset gets larger, since it needs to calculate distances to all points.
• It tends to struggle with very high-dimensional data.
• It doesn‘t provide much insight into which features are most important.

Overall, k-NN can be an effective choice for small-to-medium regression problems like wind speed prediction when quick results are needed and interpretability is less critical. For large datasets or problems where inference speed is important, alternative algorithms like linear regression, decision trees, or neural networks may be preferred.

Next Steps

There are many ways we could further refine our k-NN wind speed prediction model:

• Engineer new features, such as rolling averages of wind speed/direction
• Experiment with other distance metrics besides Euclidean distance
• Perform more systematic hyperparameter tuning, e.g. with grid search cross-validation
• Try ensembling the k-NN model with other algorithms
• Collect data over a longer time period to capture seasonal wind speed patterns

I encourage you to try out some of these ideas and see how much you can improve the model‘s performance!

Conclusion

In this post, we walked through the process of building a k-nearest neighbors model to predict wind speed based on weather data. We covered the key steps of exploring the dataset, preparing the features, training and tuning the model, and interpreting the results.

The final model achieved strong performance, with a test MAE of 0.288 m/s and R^2 of 0.855. While k-NN has some limitations, it can be a powerful and intuitive algorithm for predictive modeling on small-to-medium sized datasets.

The complete code for this analysis is available on GitHub. I hope this has been a helpful introduction to k-NN regression in Python. Feel free to reach out with any questions!