Wheel-y Interesting Insights: How I Cycled to a Top 5% Finish in Kaggle‘s Bike Sharing Demand Competition

Introduction

As an avid cyclist and data scientist, I couldn‘t resist the opportunity to combine two of my passions in Kaggle‘s Bike Sharing Demand competition. The objective was to forecast hourly rental demand for a bike sharing system in Washington, D.C. based on historical usage patterns and weather data.

With over 3,000 participants battling it out, I knew I‘d have to bring my A-game to stand out from the peloton. Leveraging my background in artificial intelligence and machine learning, I set out to build a model that could accurately predict bike demand while uncovering actionable insights for the business.

After extensive feature engineering, model tuning, and lots of coffee, I managed to pedal my way to a top 5% finish on the leaderboard. In this post, I‘ll take you through my journey from data exploration to final submission, sharing the key techniques and lessons I learned along the way.

So put on your helmet, hop on your (stationary) bike, and let‘s go for a ride!

Exploratory Data Analysis

The first stage of any data science race is getting to know the course – in this case, the dataset. The competition provided hourly rental data from 2011-2012, with variables including:

• datetime: timestamp of each hour
• season: 1 = spring, 2 = summer, 3 = fall, 4 = winter
• holiday: whether the day is a holiday
• workingday: whether the day is neither a weekend nor holiday
• weather: 1 = Clear, 2 = Mist/Cloudy, 3 = Light Rain/Snow, 4 = Heavy Rain/Snow
• temp: temperature in Celsius
• atemp: "feels like" temperature in Celsius
• humidity: relative humidity
• windspeed: wind speed
• casual: number of non-registered user rentals initiated
• registered: number of registered user rentals initiated
• count: total number of rentals (casual + registered)

To get a feel for the data, I started by visualizing demand patterns across different time frames. Plotting average rentals by hour of day revealed the classic commuter pattern for registered users, with peaks around 8am and 5pm:

Meanwhile, casual riders ramped up more slowly, hitting their peak in the early evening. Weekends showed a much different pattern than weekdays:

Clearly, the model would need to account for these nuances in usage between rider types and day of week. Weather also had a clear impact, with temperature showing a positive correlation with rentals and humidity/windspeed showing negative:

Perhaps most interestingly, segmenting rentals by month over the two years showed significant growth in the system, especially among registered users:

This suggested that a feature capturing the overall trend could be important to the model. With these insights noted, I shifted gears to data preparation.

Data Cleaning & Feature Engineering

In any modeling project, a significant amount of work must happen behind the scenes to get the data in proper shape before fitting algorithms. Key steps included:

• Splitting the datetime into separate features for year, month, day, and hour to allow the model to detect trends at different granularities
• Creating a "day of week" feature to capture the weekday/weekend usage patterns
• One-hot encoding categorical features like season and weather to make them model-friendly
• Scaling the continuous features like temp and windspeed to put them on a consistent range
• Clipping outliers in the weather variables to the 5th/95th percentiles to avoid undue influence
• Log transforming the casual and registered rental counts, since the evaluation metric (RMSLE) operated in log space
• Creating interaction features between important variables (e.g. temp humidity, hour workingday)
• Engineering lag features to provide the previous 1, 2, and 24 hours of demand as inputs, since recent trends are likely predictive

Here‘s a peek at the code used to create some of the time-based features:

def extract_time_features(df):
    df[‘date‘] = df.datetime.apply(lambda x: x.split()[0])
    df[‘hour‘] = df.datetime.apply(lambda x: x.split()[1].split(‘:‘)[0]).astype(int)
    df[‘year‘] = df.datetime.apply(lambda x: x.split()[0].split(‘-‘)[0]).astype(int)
    df[‘month‘] = df.datetime.apply(lambda x: x.split()[0].split(‘-‘)[1]).astype(int)
    df[‘day‘] = df.datetime.apply(lambda x: x.split()[0].split(‘-‘)[2]).astype(int)
    df[‘dayofweek‘] = pd.to_datetime(df.date).dt.dayofweek
    return df

And here‘s the function used to create lag features:

def create_lag_features(df, cols, lags):
    for col in cols:
        for lag in lags:
            df[f‘{col}_lag{lag}‘] = df[col].shift(lag)
    return df

By the end of the feature engineering process, I had expanded the feature set from 12 to over 50 variables, providing ample fodder for the model fitting stage.

Modeling

With the data spruced up and ready to go, it was time for the main event – model building! I chose to focus on tree-based algorithms like random forests and gradient boosted machines (GBMs), which tend to perform very well on structured, tabular data like this.

To streamline model evaluation, I created a custom cross-validation loop that used the RMSLE metric for scoring (code simplified for brevity):

from sklearn.model_selection import KFold
from sklearn.metrics import mean_squared_log_error

def rmsle_cv(model, X_train, y_train, n_folds=5):
    kf = KFold(n_folds, shuffle=True, random_state=42).get_n_splits(X_train)
    rmse = np.sqrt(-cross_val_score(model, X_train, y_train, scoring="neg_mean_squared_log_error", cv = kf))
    return rmse

I then defined a hyperparameter grid for key settings like max_depth, min_samples_leaf, and n_estimators and used a grid search to find the optimal combination.

After trying a variety of algorithms, I found that a random forest with 200 trees, a max depth of 15, and a minimum of 20 samples per leaf produced the best cross-validated RMSLE score. Here‘s a look at the final model definition:

from sklearn.ensemble import RandomForestRegressor

rf = RandomForestRegressor(n_estimators=200, 
                           max_depth=15,
                           min_samples_leaf=20, 
                           n_jobs=-1)

I also experimented with neural networks, using a simple multi-layer perceptron, but found that they didn‘t outperform the random forest, so I stuck with the tree-based approach for parsimony.

To squeeze out every last drop of performance, I trained separate models for casual and registered rentals, then combined their predictions to get the total rental forecast. This allowed the models to focus on the nuances of each rider type.

Finally, I made predictions on the test set, inverse transformed the log predictions back to the original scale, and submitted my results to Kaggle. To my delight, my solution landed me in the top 5% of the 3,000+ participants – a podium finish!

Insights & Impact

While the modeling process was a blast, some of the most valuable outcomes were the insights generated for optimizing the bike share system. By analyzing the feature importances from the trained model, clear patterns emerged:

Weather variables like temperature, humidity, and windspeed had a major impact on casual ridership, but far less effect on registered users. This suggests that promotions and dynamic pricing could be used to incentivize rides during ideal weather conditions.

The overall growth rate of the system, especially among subscribers, indicates that continual investment may be needed to avoid supply shortages at peak hours. The model could be used to forecast demand months in advance to optimize inventory planning.

Weekday vs. weekend usage patterns were remarkably different, supporting a dynamic reallocation of bikes between stations on Mondays and Fridays. Registered user commute patterns could inform station placement near major office hubs.

Perhaps most excitingly, I prototyped an interactive dashboard in Streamlit that would allow system operators to visualize predicted demand across the city based on adjustable weather and date inputs:

Putting these powerful ML models in the hands of decision-makers, in an interpretable way, is the ultimate goal of data science. An interactive tool like this could streamline the day-to-day operations and surface high-ROI optimization opportunities.

Conclusion & Lessons Learned

Participating in Kaggle‘s Bike Sharing Demand competition was a fantastic way to exercise the full data science process end-to-end – from dirty data to insights and impact.

Some of my top takeaways for approaching predictive modeling challenges:

1. Impactful feature engineering is just as important (if not more so) than model selection. Really get to know your data and brainstorm ways to extract new signals.

2. Start with simple, interpretable models, then move to more complex ones if needed. You‘d be surprised how far a random forest can take you.

3. Ensembling and stacking models tends to outperform any individual model. Don‘t be afraid to combine predictions from diverse algorithms.

4. Data science doesn‘t end with model building – arguably the most important part is extracting insights from models and building tools to support data-driven decisions.

5. Have fun and learn as much as you can! Kaggle is an incredible resource to develop real-world data science skills, regardless of your competition ranking.

I hope my approach to the bike share forecasting challenge inspires you to join the race and accelerate your own data science journey. With a combination of technical chops, creative problem solving, and business acumen, you too can ride your way to the top of the leaderboard – or better yet, real-world impact.

Feel free to reach out if you have any questions or just want to geek out about data science. And be sure to connect with me on Kaggle – I‘ll see you on the next competition!