How to Build a Predictive Model in 10 Minutes or Less
Building an effective predictive model doesn‘t have to be a long, drawn-out process. With the right strategies and tools, you can go from raw data to submitted predictions in 10 minutes or less.
In this post, I‘ll share my proven methodology for rapid model building. Whether you‘re competing in a data science hackathon or just need to prototype a solution quickly, these tips will help you optimize each step in the modeling pipeline for speed.
The Typical Predictive Modeling Workflow
Let‘s start by breaking down the key steps in a typical predictive modeling project:
- Data preprocessing
- Descriptive analysis
- Modeling
- Evaluation
In an analysis of 40 data science competitions, I found that on average, data preprocessing consumed 40% of the total project time, descriptive analysis 50%, modeling 4%, and evaluation 6%.
The preprocessing and descriptive analysis phases are the most time-consuming. This makes sense – you need to get to know your data before you can model it effectively. However, there are some strategies we can employ to streamline these stages.
Rapid Data Preprocessing
With large datasets, cleaning and transforming the raw data can be a huge time sink. Here are two of my favorite techniques to make preprocessing more efficient:
1. Dummy-code missing values
Missing values can actually be informative in some contexts. Rather than getting rid of them outright, create a dummy variable indicating whether a value is missing or not. This allows your model to learn any signal that may exist in the missingness itself.
For example, if you‘re analyzing clickstream data and a lot of values are missing for mobile-specific fields, the fact that a value is missing likely means that observation came from a desktop device. By dummy-coding it, you make it easy for the model to pick up on that pattern.
2. Quick and dirty imputation
Of course, most algorithms can‘t handle missing values directly, so we still need to fill them in. But that doesn‘t mean you need to break out sophisticated imputation methods from the start.
I find that for numeric features, simply imputing the mean or median works fine as a first pass. For categorical features, replace missing values with a constant like "Missing" or "Unknown".
The goal is to do the minimal cleaning required to get your data to a model-ready state. You can always iterate and refine your imputation strategy later if needed.
Streamlined Descriptive Analysis
When you‘re working on a tight deadline, resist the urge to perform an exhaustive exploratory analysis. Your first pass should be focused on understanding the basic shape of the data and identifying any obvious opportunities or issues.
Specifically, I recommend looking for:
- Missing values
- High cardinality categorical features that may need special handling
- Extreme values or outliers
- Strong target encoding features (features where the mean of the target varies a lot based on the value of that feature)
If you spot a feature that looks especially promising or problematic, make a note of it. But don‘t go down the rabbit hole of plotting every variable and calculating tons of aggregations. Your goal at this stage is just to inform some basic preprocessing decisions and maybe generate some ideas for model features. The real insights will come from modeling.
Rapid Modeling
With your data cleaned and a basic understanding of what you‘re working with, it‘s time to build some models. The key to modeling quickly is to leverage algorithms and tools that are both performant and require minimal tuning out of the box.
My go-to algorithms for rapid modeling are:
- Gradient boosted decision trees: Specifically, implementations like XGBoost, LightGBM, and CatBoost. These are fast, scalable, and tend to work well with minimal tuning on structured/tabular data.
- Random forest: Another solid choice for tabular data. Not as fast or tunable as gradient boosting, but more easily parallelizable, making it a good option if you have a lot of cores available.
- Linear models: For very large datasets or datasets with all numeric features, linear models like logistic regression can be trained extremely quickly. They also tend to be more interpretable than tree-based models.
I usually have a "recipe" of model hyperparameters that are known to work well as a starting point. For example, with XGBoost I might use:
params = {
‘objective‘:‘binary:logistic‘,
‘eval_metric‘:‘logloss‘,
‘eta‘:0.02,
‘max_depth‘:4,
‘min_child_weight‘:1,
‘subsample‘:0.7,
‘colsample_bytree‘:0.7
}
I don‘t spend a lot of time hand-tuning hyperparameters in the early stages. As long as the model is performing reasonably well, I‘m happy. I can always fine-tune later if I have time.
Another way to speed up modeling is to leverage automated machine learning (AutoML) tools. Platforms like H2O, auto-sklearn, and TPOT can automatically search for the best model and hyperparameters for your dataset. You sacrifice some control, but it‘s a handy way to get a model spun up quickly without needing to code everything yourself.
Rapid Model Evaluation
For the initial evaluation, I like to use k-fold cross validation with 5 or 7 folds. This gives a more stable estimate than a single train/test split, but is still relatively quick.
Make sure to evaluate your model on a metric that aligns with the business objective. If you‘re not sure what metric to use, some good general choices are:
- Classification: AUC, F1 score, log loss
- Regression: RMSE, MAE, R-squared
In a competition setting, you‘ll also want to make sure you‘re using the same metric as what the leaderboard is scored on. Nothing is more frustrating than discovering your model is doing great on accuracy when the competition is using log loss!
Putting It All Together
To tie this all together, I recommend wrapping your modeling pipeline in a single function or script. This makes it push-button easy to ingest new data, run preprocessing, train a model, and generate predictions.
Here‘s some skeleton code for what that might look like in Python:
def rapid_model(train_path, test_path, submit_path, target):# Read in data train = pd.read_csv(train_path) test = pd.read_csv(test_path) submit = pd.read_csv(submit_path) features = [f for f in train.columns if f != target] # Preprocess data for df in [train, test]: df[‘missing_vals‘] = df.isna().sum(axis=1) df[features] = df[features].fillna(df[features].mean()) df[features] = df[features].fillna(‘Unknown‘) # Train model model = xgb.XGBClassifier(**params) model.fit(train[features], train[target]) # Predict and submit submit[target] = model.predict_proba(test[features])[:,1] submit.to_csv(submit_path, index=False) print("Done!")The goal is to be able to run this script the minute you get your hands on the data and generate a submission file in 10 minutes or less. This gives you a quick benchmark to work off of.
From there, you can start layering in improvements – more sophisticated preprocessing, hyperparameter tuning, feature engineering, ensembling, etc. The key is that you‘re always iterating off a foundation of a basically performant model.
Go Forth and Model!
Speed is a critical skill for data scientists, whether you‘re competing in a hackathon or trying to rapidly prototype solutions to business problems. The techniques covered in this post will help you get to an effective predictive model faster, without sacrificing performance.
The key things to remember are:
- Automate as much of the workflow as possible
- Use algorithms and tools that are known to be fast and effective
- Make simplicity and speed the priority in the early stages, then iterate
- Submit early and often!
Now the next time you‘re facing down a tight modeling deadline, you‘ll be able to confidently build a powerful predictive model in 10 minutes flat. Happy modeling!
