Leveraging Logistic Regression in R to Understand and Reduce Employee Attrition

Employee attrition is a billion-dollar problem. The Work Institute‘s 2020 Retention Report estimates that voluntary turnover costs U.S. businesses over $630 billion annually, with the average cost per employee ranging from 33% to 200% of their annual salary depending on seniority and role.

In today‘s tight labor market, the stakes have never been higher for HR leaders to retain top talent. Luckily, advances in machine learning and people analytics offer powerful tools to predict and prevent unwanted attrition.

In this guide, we‘ll walk through a step-by-step case study of how logistic regression analysis in R can uncover the key factors driving employee attrition. By following this framework, you‘ll learn how to translate raw HR data into actionable insights to boost retention.

Why Logistic Regression Excels at Predicting Attrition

Logistic regression is a classification algorithm that estimates the probability of a binary outcome occurring based on a set of independent variables. Since employee attrition is a binary event (stay vs leave), logistic regression is a natural fit.

Under the hood, logistic regression fits an S-shaped logistic function that outputs a probability between 0 and 1. Mathematically, it models the log-odds (logit) of the binary outcome as a linear combination of predictor variables:

$\log\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1x_1 + … + \beta_nx_n$

Where $p$ is the probability of attrition, $x_i$ are the predictor variables, and $\beta_i$ are the regression coefficients.

Compared to other binary classification techniques, logistic regression offers several advantages for attrition modeling:

1. Interpretability: Coefficients directly quantify each predictor‘s impact on the odds of attrition, aiding explainability
2. Efficiency: Requires fewer parameters vs neural networks, allowing fast training on small-to-medium HR datasets
3. Simplicity: Outputs a linear decision boundary, reducing risk of overfitting on high-dimensional problems
4. Extensibility: Easily enhanced via feature engineering, regularization, et al. to boost predictive power

With these benefits in mind, let‘s put logistic regression into practice on an employee attrition dataset using R.

Collecting and Preparing the Attrition Dataset

We‘ll be using the IBM HR Analytics Employee Attrition & Performance dataset available on Kaggle, containing 1470 employee records with 35 attributes:

attrition_data <- read.csv("WA_Fn-UseC_-HR-Employee-Attrition.csv")
dim(attrition_data)  
## [1] 1470  35

Key data preparation steps:

1. Check for missing values and outliers
2. Convert categorical variables like Attrition, BusinessTravel, Department to factors
3. Normalize numeric variables to comparable scales
4. Address class imbalance in Attrition (only 16% attrition rate)

# Convert Attrition to factor
attrition_data$Attrition <- as.factor(attrition_data$Attrition)

# Check class imbalance
table(attrition_data$Attrition) 
##  No  Yes 
## 1233  237

# Normalize numeric columns
norm_cols <- c("Age", "MonthlyIncome", "NumCompaniesWorked", "PercentSalaryHike", 
               "TotalWorkingYears", "TrainingTimesLastYear", "YearsAtCompany", 
               "YearsInCurrentRole", "YearsSinceLastPromotion", "YearsWithCurrManager")
norm_data <- as.data.frame(scale(attrition_data[, norm_cols]))

# Add normalized columns to original data 
attrition_data <- cbind(attrition_data[, -which(names(attrition_data) %in% norm_cols)], 
                        norm_data[, norm_cols])

To handle class imbalance, we can use techniques like:

• Oversampling the minority class (attrition)
• Undersampling the majority class (non-attrition)
• Synthesizing new minority examples with SMOTE (Synthetic Minority Oversampling Technique)

Here we‘ll use random oversampling to create a 50/50 class balance:

# Set random seed for reproducibility 
set.seed(123)

# Oversample minority class
attrition_over <- ovun.sample(Attrition ~ ., data=attrition_data, method="over", N=2466)$data
table(attrition_over$Attrition)
##  No Yes 
## 1233 1233

Training the Logistic Regression Model

With data prepped, we split into 70% training and 30% test sets and fit the logistic regression:

# 70/30 train/test split
index <- sample(seq_len(nrow(attrition_over)), size=floor(0.7*nrow(attrition_over))) 
train_data <- attrition_over[index, ]
test_data <- attrition_over[-index, ]

# Fit logistic regression 
model <- glm(Attrition ~ ., data=train_data, family="binomial")

To identify the most predictive features, we use stepwise selection with the Akaike Information Criterion (AIC):

step <- stepAIC(model, direction="both")
summary(step)

## Coefficients:
##                                  Estimate Std. Error z value Pr(>|z|)    
## (Intercept)                     -1.205e+00  1.819e-01  -6.625 3.47e-11 ***
## OverTime[T.Yes]                  1.391e+00  1.356e-01  10.256  < 2e-16 ***
## TotalWorkingYears               -7.867e-01  1.491e-01  -5.276 1.32e-07 ***
## MonthlyIncome                   -6.642e-01  9.781e-02  -6.791 1.12e-11 ***
## StockOptionLevel                -2.945e-01  6.804e-02  -4.328 1.51e-05 ***
## JobRole[T.Manufacturing Director] 6.557e-01  2.802e-01   2.340 0.019307 *  
## JobRole[T.Research Scientist]    5.932e-01  1.900e-01   3.121 0.001801 ** 
## JobRole[T.Sales Executive]       3.931e-01  1.653e-01   2.377 0.017452 *  
## JobLevel                         2.395e-01  1.150e-01   2.082 0.037320 *

The final model includes 8 significant predictors of attrition, with job role, overtime, years at company, monthly income, and stock options having the largest effects.

Evaluating Model Performance

To gauge real-world model performance, we score the test set and calculate key classification metrics:

# Score test set
preds <- predict(step, type="response", newdata=test_data)

# Convert probabilities to class labels    
preds_class <- ifelse(preds > 0.5, "Yes", "No")

# Calculate confusion matrix
conf_mat <- confusionMatrix(as.factor(preds_class), test_data$Attrition, positive="Yes")
conf_mat

## Confusion Matrix and Statistics
##           Reference
## Prediction   No  Yes
##        No  317   45
##        Yes  51  327
##                                           
##                Accuracy : 0.8703          
##                  95% CI : (0.8438, 0.8939)
##     No Information Rate : 0.4973          
##     P-Value [Acc > NIR] : < 2.2e-16       
##                                           
##                   Kappa : 0.7405          
##                                           
##  Mcnemar‘s Test P-Value : 0.6815          
##                                           
##             Sensitivity : 0.8790          
##             Specificity : 0.8614          
##          Pos Pred Value : 0.8651          
##          Neg Pred Value : 0.8757          
##              Prevalence : 0.5027          
##          Detection Rate : 0.4419          
##    Detection Prevalence : 0.5108          
##       Balanced Accuracy : 0.8702          
##                                           
##        ‘Positive‘ Class : Yes             
## 

The model achieves 87% accuracy on the test set, striking a good balance between sensitivity (88%) and specificity (86%). This means it correctly identifies 88% of actual attrition cases and 86% of non-attrition cases.

We can also visualize performance tradeoffs with an ROC curve:

roc_obj <- roc(test_data$Attrition, preds, plot=TRUE, col="blue", lwd=2, 
               main="ROC Curve for Attrition Model")
auc(roc_obj)
## Area under the curve: 0.945

The model achieves an excellent AUC of 0.945, or 94.5% probability that it will rank a randomly chosen attrition case higher than a non-attrition case. This gives us confidence in its predictive power.

Interpreting Drivers of Attrition

With a robust model in hand, we can now interpret the logistic regression coefficients to understand what factors have the biggest impact on attrition risk:

• Working overtime increases odds of attrition by 303% (exp(1.391) = 4.03), the strongest effect
• Each additional year at the company reduces odds of attrition by 54% (exp(-0.787) = 0.46)
• Every $1,000 increase in monthly income lowers odds of attrition by 48% (exp(-0.664) = 0.52)
• Higher stock option level and job level also decrease attrition risk
• Being a manufacturing director, research scientist, or sales exec increases odds of leaving vs other job roles

These insights suggest that reducing overtime, raising pay, providing stock options, and enabling career progression could help mitigate attrition.

Informing HR Retention Initiatives

Translating model insights into targeted retention programs is key to realizing business impact. Based on the drivers surfaced, HR could:

1. Implement overtime tracking and manager notifications to better distribute workload
2. Review compensation bands against market data and adjust as needed, especially for critical talent segments
3. Expand stock option grants to more employee levels
4. Analyze attrition within job families to identify and address pain points in career pathing
5. Conduct "stay interviews" and engagement surveys to get ahead of attrition risks

By measuring KPIs like quarterly attrition rate, compensation ratios, and engagement scores before and after each intervention, HR can quantify its impact and build a business case for future investments.

Advancing the State of the Art in Attrition Modeling

Looking ahead, AI and machine learning will continue to push the boundaries of attrition prediction and prevention:

• Neural network models like MLP, CNN, and LSTM show the potential to boost predictive accuracy, albeit at the cost of some interpretability. IBM researchers achieved 96% accuracy with simple MLP models on this same dataset.

• Survival analysis extends attrition modeling from if to when. Techniques like Kaplan-Meier curves, Cox regression, and accelerated failure time models can predict the time until an employee is likely to churn. This supports more proactive retention efforts.

• NLP can mine unstructured data like performance reviews, exit interviews, and Glassdoor posts for attrition signals. Researchers have used CNN and RNN models to predict flight risk from review text, raising accuracy 5-10% vs models based on structured data alone.

• Explainable AI (XAI) methods such as SHAP, LIME, and Integrated Gradients can make complex deep learning models more accessible to HR stakeholders. By quantifying each feature‘s impact on individual predictions, XAI promotes trust and adoption.

• Multi-task learning jointly predicts attrition alongside related outcomes like performance, engagement, and tenure. Google researchers improved attrition model ROC AUC by 1-5% by adding auxiliary tenure prediction tasks.

Whatever the approach, rigorous model validation, bias auditing, and privacy safeguards are critical to the responsible deployment of AI in HR. But with sound data ethics, the potential to transform employee experience and retention is immense.

Conclusion

As the battle for talent intensifies, leveraging data science to predict and prevent voluntary turnover has become table stakes for HR. Logistic regression offers an accessible yet powerful tool to surface attrition drivers and inform targeted retention strategies.

By following the framework laid out in this guide—from data prep to model training to insight translation—you‘re well on your way to putting machine learning into practice. Combined with domain expertise and empathy, these analytical approaches will help create the engaging employee experiences needed to thrive in the future of work.