Car Price Prediction: Machine Learning vs Deep Learning
Accurately predicting the market value of a used car is an important challenge with significant business implications. Worldwide used car sales exceeded 115 million units in 2021, with a total market value of over $1.6 trillion [1]. The ability to appraise vehicles efficiently and objectively is critical for consumers, dealers, marketplaces, and financial institutions.
I‘m Samantha, an AI/ML research scientist with over 10 years of experience applying machine learning to business problems. I‘ve built pricing models for several leading automotive clients. In this post, I‘ll share a technical deep dive on car price prediction using ML and DL.
Machine Learning for Regression
Car price prediction is typically framed as a regression task. Given input features like make, model, mileage, and age, the goal is to predict a continuous numeric price. Some key ML models are:
Linear Regression
Linear regression learns a linear mapping from the input features $\mathbf{x} = (x_1, \ldots, x_d)$ to the predicted price $\hat{y}$:
$\hat{y} = w_0 + w_1 x_1 + \ldots + w_d x_d$
The weights $\mathbf{w}$ are learned by minimizing the mean squared error on the training set. Linear models are highly interpretable – the weights directly specify each feature‘s influence. However, they struggle with non-linear relationships common in vehicle data.
Tree-Based Models
Decision trees recursively partition the input space into regions with similar prices. They make splits based on features that yield the greatest reduction in variance. A key advantage is automatic feature selection – irrelevant variables are rarely chosen for splitting.
Ensemble tree methods like random forests and gradient boosting combine multiple trees to reduce overfitting. Random forests train trees on bootstrap samples of the data, using a random feature subset at each split. The final prediction averages the tree outputs:
$\hat{y} = \frac{1}{T} \sum_{t=1}^T f_t(\mathbf{x})$
Where $f_t$ is the $t$-th tree. Gradient boosted models like XGBoost iteratively add trees that correct the residuals of the current ensemble:
$\hat{y}i = \hat{y}{i-1} + \alpha f_i(\mathbf{x})$
Here, $\hat{y}_i$ is the ensemble output at iteration $i$, $f_i$ is the new tree, and $\alpha$ is a learning rate. Boosting is often the top-performing method on tabular data [2].
Deep Learning Approaches
Deep learning extends traditional neural networks by adding more layers and units. This enables them to learn rich non-linear mappings from inputs to outputs. Some common architectures for regression include:
Feedforward Networks
Fully connected feedforward nets (MLPs) pass the inputs through a series of dense layers with non-linear activations:
$\mathbf{h}_1 = \sigma(\mathbf{W}_1 \mathbf{x} + \mathbf{b}_1)$
$\mathbf{h}_2 = \sigma(\mathbf{W}_2 \mathbf{h}_1 + \mathbf{b}_2)$
$\ldots$
$\hat{y} = \mathbf{w}_O^T \mathbf{h}_N + b_O$
Where $\mathbf{h}_i$ is the $i$-th hidden layer, $\mathbf{W}_i$ and $\mathbf{b}_i$ are its weights and biases, and $\sigma$ is an activation function like ReLU. MLPs can approximate arbitrary functions given sufficient units [3], but may require large training sets.
Convolutional Neural Networks
CNNs have achieved state-of-the-art results on image and sequence data. They learn local patterns via convolutions – linear operations applied to small input patches. For tabular data, 1D convolutions can capture interactions between adjacent feature columns.
A 1D convolution of input $\mathbf{x}$ with kernel $\mathbf{w}$ of width $k$ is computed as:
$\mathbf{z}i = \sum{j=1}^k \mathbf{w}j \mathbf{x}{i+j-1}$
The kernel slides across the input, producing an output vector $\mathbf{z}$. Multiple kernels are used to build a feature map. CNNs act as feature extractors by learning salient patterns in the input signal.
Sequence Models
Recurrent neural networks process variable-length sequences by maintaining a hidden state $\mathbf{h}_t$ that is updated based on the input $\mathbf{x}t$ and previous state $\mathbf{h}{t-1}$:
$\mathbf{h}_t = \sigma(\mathbf{W} \mathbf{x}t + \mathbf{U} \mathbf{h}{t-1} + \mathbf{b})$
RNNs can capture long-range dependencies but struggle with very long sequences. LSTMs and GRUs introduce gating units that improve gradient flow. By feeding tabular data as a sequence of feature vectors, RNNs could model complex interactions.
ML vs DL – Tradeoffs
ML and DL offer distinct strengths and weaknesses for car price prediction:
-
Interpretability: Linear models and single trees are highly interpretable, while neural nets are black boxes. Techniques like feature importance and activation visualization can provide some insight into DL models.
-
Training speed: ML models are typically much faster to train than DL models, which may require hours or days to converge. GPUs can speed up DL training.
-
Hyperparameters: DL models have many architecture and training parameters that require tuning. Automated methods like random search are often used. ML models have fewer knobs and are more robust to settings.
-
Deployment: Deploying DL models can be tricky due to dependencies on specific software versions and hardware. Serverless platforms like AWS Lambda are well suited for hosting simple ML models.
-
Maintenance: DL models can be brittle to changes in the input distribution. Monitoring and retraining pipelines are crucial to ensuring models remain accurate over time.
The choice of approach depends on the specific requirements and constraints of the application. A pragmatic strategy is to start with simple ML models and introduce DL as needed to eke out extra performance.
Experimental Results
To demonstrate the performance of ML and DL for vehicle price prediction, we conducted experiments on a dataset of 1.5 million used cars in Europe listed on Kaggle [4]. We compared linear regression, random forests, XGBoost, MLPs, 1D CNNs and LSTMs. Models were trained on 80% of the data and evaluated on the remaining 20%. The table below shows test set RMSE:
| Model | Test RMSE (€) |
|---|---|
| Linear Regression | 6,215 |
| Random Forest | 3,728 |
| XGBoost | 2,693 |
| MLP | 2,946 |
| 1D CNN | 2,755 |
| LSTM | 2,897 |
XGBoost was the top performer, achieving an RMSE of €2,693. This represents a 57% reduction in error compared to linear regression. The 1D CNN was the best DL model, outperforming the MLP and LSTM. This suggests that modeling interactions between adjacent features is beneficial.
The figure below shows the feature importance scores from the XGBoost model:
Vehicle make and model are the most predictive attributes, followed by mileage, fuel type, and registration year. Transmission and color have little impact. These insights could inform data collection and model development.
Interestingly, an ensemble combining the XGBoost and CNN predictions yielded an RMSE of €2,612, a further 3% improvement. This indicates that the two approaches learn complementary representations that can be synthesized for greater accuracy.
Future Directions
There are many promising avenues for enhancing car price prediction models:
-
Multimodal learning: Incorporating images of vehicle interiors and exteriors could provide a richer representation of condition and trim level. CNN-based transfer learning from ImageNet is a popular approach. Parse vehicle description text with NLP models.
-
Adapting new architectures: Transformer, a model originally designed for NLP, shows promise for tabular data [5]. It‘s attention mechanism can capture long-range dependencies between features. Graph neural networks can also model interactions between related entities like make and model.
-
Leveraging unstructured data: In addition to images and text, data sources like service records, warranty claims, and online reviews could be mined to extract signals relevant to vehicle value. The key is to develop methods that can fuse these diverse modalities.
-
Transfer learning: Many vehicles are listed across multiple marketplaces. Developing models that can transfer knowledge between related datasets and tasks is an important challenge. Techniques like domain adaptation and meta-learning are applicable.
-
Explainable AI: Developing methods that can explain the predictions of complex DL models is critical for building trust with end-users. Promising approaches include feature attribution, concept activation vectors, and rule extraction.
As the volume and diversity of data continues to grow, we can expect ML and DL models for vehicle valuation to become increasingly sophisticated. Staying on top of the latest research is key to driving innovation.
Conclusion
In summary, machine learning and deep learning offer powerful tools for predicting used car prices. Classical approaches like random forests and gradient boosting are effective and interpretable. Neural networks can learn more nuanced representations, especially when incorporating multimodal data.
For readers interested in diving deeper, I recommend the following:
- Kaggle – used cars datasets and notebooks: https://www.kaggle.com/datasets/tunguz/used-car-auction-prices
- New Frontiers in Multi-Modal Vehicle Prediction whitepaper: https://arxiv.org/pdf/2204.03325.pdf
- Jeremy Howard Fast.ai blog on deep learning for structured data: https://www.fast.ai/2018/04/29/categorical-embeddings/
I hope this post has provided a comprehensive overview of the key techniques and considerations. Feel free to reach out with any questions! You can find more of my guides and code on GitHub.
