Titanic Survival Prediction Using Machine Learning

The sinking of the RMS Titanic in 1912 was one of the deadliest maritime disasters in history. Over 1,500 passengers and crew members lost their lives when the ship sank on its maiden voyage after colliding with an iceberg. In this article, we‘ll use machine learning to analyze data about Titanic passengers and develop a model to predict which passengers were most likely to survive the disaster.

Machine learning has become an increasingly popular and powerful tool for data analysis and prediction. By training machine learning algorithms on historical data, we can uncover complex patterns and relationships to make informed predictions and decisions. The Titanic dataset is often used as an introductory example for machine learning and data science, as it is relatively small and easy to understand while still providing an interesting prediction problem.

Our goal will be to build a binary classification model that predicts whether a passenger survived (1) or did not survive (0) based on various features like age, gender, passenger class, and so on. We‘ll follow a typical machine learning workflow of data exploration, preprocessing, model training, and evaluation. By the end of this article, you‘ll have a good understanding of how to approach a machine learning problem and implement a solution in Python.

The Titanic Dataset

The dataset we‘ll be working with contains information on 891 of the Titanic passengers, along with labels indicating whether each passenger survived or not. Here are the main features:

• PassengerId: Unique identifier for each passenger
• Survived: Whether the passenger survived (1) or not (0)
• Pclass: Ticket class (1 = 1st, 2 = 2nd, 3 = 3rd)
• Name: Passenger name
• Sex: Passenger gender
• Age: Passenger age in years
• SibSp: Number of siblings/spouses aboard
• Parch: Number of parents/children aboard
• Ticket: Ticket number
• Fare: Passenger fare
• Cabin: Cabin number
• Embarked: Port of embarkation (C = Cherbourg, Q = Queenstown, S = Southampton)

We can load this data into a Pandas DataFrame and take an initial look:

import pandas as pd

df = pd.read_csv(‘titanic.csv‘)
df.head()

Exploratory Data Analysis

Before diving into machine learning, it‘s important to explore and visualize the data to gain insights. We can start by checking the data types and looking for missing values:

df.info()

We can see that Age, Cabin, and Embarked have missing values that we‘ll need to deal with later. The Name, Ticket, and Cabin features also likely won‘t be useful for predicting survival.

Let‘s look at the relationship between Pclass and Survived:

import seaborn as sns

sns.countplot(x=‘Pclass‘, hue=‘Survived‘, data=df)

It appears that a larger proportion of 1st class passengers survived compared to 2nd and 3rd class. We can examine other features like Sex and Age in a similar way. Visualizing relationships between features and the target variable helps inform our feature engineering and modeling choices.

Data Preprocessing

Based on our EDA, we‘ll preprocess the data with the following steps:

1. Drop unnecessary features: Name, Ticket, Cabin
2. Handle missing Age values by filling with the median
3. Handle missing Embarked values by filling with the mode
4. Convert Sex and Embarked to numeric via one-hot encoding
5. Confirm there are no missing values

Here‘s how we can implement these steps in Python:

df = df.drop([‘Name‘, ‘Ticket‘, ‘Cabin‘], axis=1)

df[‘Age‘] = df[‘Age‘].fillna(df[‘Age‘].median())
df[‘Embarked‘] = df[‘Embarked‘].fillna(df[‘Embarked‘].mode()[0])  

df = pd.get_dummies(df, columns=[‘Sex‘, ‘Embarked‘], drop_first=True)

print(df.isnull().sum())

Train-Test Split

Now our data is ready for machine learning. We‘ll split it into training and test sets, using the training set to build the model and the test set to evaluate performance on unseen data. This helps assess how well the model generalizes.

We‘ll use scikit-learn‘s train_test_split function to perform the split, with 80% of data for training and 20% for testing:

from sklearn.model_selection import train_test_split

X = df.drop(‘Survived‘, axis=1)
y = df[‘Survived‘]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Logistic Regression Model

Since we‘re predicting a binary outcome (survived or not), logistic regression is an appropriate model to start with. Logistic regression estimates the probability of the target variable belonging to a particular category. If the probability is >0.5, the sample is assigned to the positive class, otherwise it‘s assigned to the negative class.

We can quickly train and evaluate a logistic regression model with scikit-learn:

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

model = LogisticRegression()
model.fit(X_train, y_train)

train_preds = model.predict(X_train) 
print(f‘Training Accuracy: {accuracy_score(y_train, train_preds)}‘)

test_preds = model.predict(X_test)
print(f‘Testing Accuracy: {accuracy_score(y_test, test_preds)}‘)

The logistic regression model achieves 80% accuracy on the training set and 82% accuracy on the test set, which is a decent baseline performance.

Model Evaluation

While accuracy is an intuitive evaluation metric, it doesn‘t tell the whole story. We can get a more complete picture of model performance by looking at a confusion matrix, which shows the number of true positives, true negatives, false positives, and false negatives:

from sklearn.metrics import confusion_matrix

print(confusion_matrix(y_test, test_preds))

We can also examine other metrics like precision, recall, and F1 score:

from sklearn.metrics import classification_report

print(classification_report(y_test, test_preds))

These metrics provide additional insight into the types of errors the model makes and the trade-offs between precision and recall.

Improving the Model

There are several ways we could potentially improve our model performance:

• Feature engineering: Creating new features or transforming existing ones
• Hyperparameter tuning: Adjusting model settings like regularization strength
• Trying other algorithms: Decision trees, random forests, gradient boosting, etc.

Let‘s try tuning the regularization hyperparameter C in logistic regression using grid search cross-validation:

from sklearn.model_selection import GridSearchCV

param_grid = {‘C‘: [0.01, 0.1, 1, 10, 100]}
grid = GridSearchCV(model, param_grid, cv=5)
grid.fit(X_train, y_train)

print(f‘Best hyperparameters: {grid.best_params_}‘) 
print(f‘Best score: {grid.best_score_}‘)

It looks like using C=1 yields the best result. We can retrain the model with this setting and evaluate performance.

Other Machine Learning Models

Now let‘s train a few other popular classification algorithms and compare performance:

from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import GradientBoostingClassifier

models = {
    ‘Logistic Regression‘: LogisticRegression(C=1),
    ‘Decision Tree‘: DecisionTreeClassifier(),
    ‘Random Forest‘: RandomForestClassifier(),
    ‘Gradient Boosting‘: GradientBoostingClassifier()
}

for model_name, model in models.items():
    model.fit(X_train, y_train)
    test_preds = model.predict(X_test)
    print(f‘{model_name} Accuracy: {accuracy_score(y_test, test_preds)}‘)  

The decision tree-based ensemble methods perform best, with random forest and gradient boosting classifiers achieving around 85% accuracy on the test set.

Conclusions

In this article, we stepped through the process of using machine learning to predict Titanic passenger survival. We performed exploratory data analysis to understand the relationships between various features and the target variable. We preprocessed the data by removing unnecessary features, imputing missing values, and encoding categorical variables. We then trained a logistic regression model and evaluated its performance using metrics like accuracy, confusion matrix, precision, and recall.

We saw how we could improve performance through hyperparameter tuning and training other types of models. In the end, decision tree-based ensemble models like random forests and gradient boosting achieved the best results with 85% test set accuracy.

The key takeaways are:

1. EDA is a crucial first step to guide feature engineering and modeling choices
2. Data preprocessing is necessary to clean and encode features for modeling
3. Splitting data into train and test sets allows evaluating generalization ability
4. Different models have different strengths and should be compared empirically
5. Hyperparameter tuning can significantly impact model performance

Of course, an accuracy of 85% still means that our best model‘s predictions are wrong 15% of the time. Depending on the use case, this may or may not be an acceptable error rate. It‘s important to have a good understanding of a model‘s limitations and when it can be relied upon for making important decisions.

Further Work

There are many potential extensions to this work, including:

• Exploring other feature engineering options, like binning age groups or scaling fare values
• Dealing with outliers and examining their influence on the results
• Using additional evaluation techniques like ROC curve and area under the curve (AUC)
• Performing survival analysis with Kaplan-Meier estimation
• Applying more advanced methods like support vector machines, XGBoost, neural networks, etc.
• Gathering more data (if available) to see if model performance improves
• Deploying a final model to make predictions on new passenger data

I encourage you to try some of these ideas and see if you can build a better Titanic survival prediction model! Machine learning is a powerful tool, but one that requires careful thought and assumptions. Always think critically about your results and conclusions before putting models into production.

I hope this article helped explain the key concepts involved in using machine learning to predict Titanic passenger survival. Please leave a comment if you have any other questions, and happy coding!