Titanic Survivors: A Guide for Your First Data Science Project

If you‘re just getting started with data science and machine learning, you‘ve probably heard of the Titanic dataset. It‘s a popular beginner project that challenges you to predict passenger survival based on attributes like age, sex, and cabin class.

The Titanic dataset has several advantages for learning:

• It only has 12 columns, so it‘s small and manageable
• There‘s a clear goal: Predicting survival, encoded as a binary value (0 = died, 1 = survived)
• The features are easy to understand
• There are interesting historical insights to uncover

You can download the Titanic dataset from Kaggle, a data science community that hosts machine learning competitions. The dataset is split into two CSV files:

• train.csv: The training set, which you‘ll use to build your model. It includes a "Survived" column.
• test.csv: The test set for evaluating your final model‘s performance. It does not include "Survived".

Exploratory Data Analysis

The first step is getting to know the data. Using pandas and matplotlib in a Jupyter notebook, we can load the CSVs into DataFrames and start our analysis.

Let‘s look at the numeric columns first: Age, SibSp (# of siblings/spouses aboard), Parch (# of parents/children aboard), and Fare (ticket price). Plotting histograms of each column reveals their distributions:

Age is roughly normal, with a mean around 30 years old. The other columns are more skewed. There are a few outliers with very high Fares.

A correlation heatmap shows that SibSp and Parch have the strongest positive relationship. This makes sense, as families likely bought tickets together.

But how do these factors relate to survival? Comparing the averages between survived vs. died passengers, we see:

• Survivors tended to be younger (mean age of 28 vs. 31)
• Survivors had more parents/children with them (mean of 0.46 vs. 0.33)
• Survivors paid higher fares (mean of $48 vs. $23)

Already we‘re seeing that age, family structure, and economic class may predict survival. Let‘s continue by examining the categorical features.

Counting unique values reveals:

• Only ~38% of passengers survived
• The majority traveled in 3rd class
• About 65% were male
• Most passengers embarked at Southampton

To see survival rates across categories, we can group by the Survived column and aggregate with count().

This confirms our earlier hunches:

• 1st class passengers had the highest survival rate (~63%), followed by 2nd class (~47%) and 3rd class (~24%)
• Women survived at a much higher rate than men (~74% vs. ~19%)
• Passengers embarking at Cherbourg had the best odds, while those embarking at Southampton fared the worst

So far we‘ve established clear survival trends around age, sex, family size, and passenger class. Before building any models, let‘s do some feature engineering to extract more signal from the messier columns.

Feature Engineering

Columns like Name, Ticket, and Cabin need to be transformed to be useful for machine learning. With the Name column, we can extract titles and see if they correlate with survival.

Using string methods, we pull out titles like Mr., Miss., Col., etc. and make a new Title feature. Cross-tabulating this with Survived shows that certain titles had better survival odds:

• 100% of women with rare titles like Lady, Countess, and Mme survived
• Military titles like Col., Major, and Capt. had high survival rates
• Professionals like Dr. survived at high rates

For Ticket, we separate numeric from non-numeric and count how many cabins each passenger had (based on the number of cabins listed). Those with multiple cabins had a higher survival rate, likely because they belonged to first class.

With Cabin, the letter prefixes may indicate the deck, side of the ship, or other location factors. Extracting just the first letter, we see that certain letters (like B, D, and E) had much higher survival rates. Cabins starting with T had 0% survival. This suggests some areas of the ship were closer to lifeboats and escape routes.

Data Preprocessing

With our feature engineering complete, it‘s time to prepare the data for modeling:

1. Drop any remaining rows with null values
2. Select relevant features and drop noise like PassengerId
3. Transform categorical features into numeric via one-hot encoding
4. Impute missing numeric values with the column median
5. Normalize the Fare column with log to reduce the skew
6. Scale all features to a consistent range, like 0-1

After our transformations, we have a clean numeric matrix to feed into machine learning models.

Modeling and Evaluation

It‘s wise to start with simple, interpretable models before moving on to more complex techniques. A logistic regression classifier is a good baseline.

Using 5-fold cross validation, our logistic model achieves 80.8% accuracy. Not bad for a first attempt! But we also care about the precision, recall, and F1 scores for the survived class. Looking at the confusion matrix, we see the model is better at predicting death than survival. This makes sense given the imbalanced classes.

Next we try k-nearest neighbors, which looks at the k most similar data points to make a prediction. With k=5, cross-validated accuracy increases to 82.3%. The confusion matrix is more balanced between classes as well.

Finally, we test a support vector machine. SVMs can model complex decision boundaries and have less dependence on class sizes. Using an RBF kernel and gridsearch to find optimal hyperparameters, we hit 83.9% cross-validated accuracy – our best result yet.

Could we improve performance further? With neural networks, feature scaling, and ensembling, it‘s possible to achieve accuracy above 86% on this dataset. However, those techniques are beyond the scope of a beginner project.

For now, we use our tuned SVM to predict survival for the test set. This gives us the predictions to submit to the Kaggle competition.

Conclusions and Caveats

In roughly 1918 out of 2224 cases (86%), our model was able to correctly predict whether a passenger aboard the Titanic would live or die. Variables around social status, sex, and family structure proved to be the most informative.

However, there are important limitations to keep in mind:

• This dataset describes a single historical event. Generalizing patterns of survival to other nautical accidents is questionable.
• The dataset is imbalanced, with only 38% positive cases. This skews the model toward predicting death and makes metrics like accuracy misleading. In real projects, it‘s critical to consider the cost of false positives/negatives and optimize for the right metric.
• Many missing values required imputation. More granular age data, especially for children, would likely improve the model.
• There are aspects of each passenger‘s fate that can never be captured in data. Not every decision to board a lifeboat was coldly rational, and heroism or chance certainly played a role.

Despite these caveats, the Titanic dataset is a great way to get your feet wet with data science and appreciate how far machine learning has come in the past century.

With a bit of data wrangling, feature engineering, and model tuning, you can build a reasonably performant classifier for a complex real-world event. Just remember to validate your model and focus on the right performance metric!

For those who want to dive deeper, consider these extensions:

• Ensemble the SVM with other well-performing models
• Vary the feature set and see what affects model performance
• Balance the classes using oversampling, undersampling, or SMOTE
• Read up on random forests, gradient boosting, and neural networks

I hope this step-by-step walkthrough gives you the context and confidence to try the Titanic dataset yourself. For complete Python code, including more detailed EDA and a variety of model types, check out my Kaggle notebook.

Treat it as a learning opportunity and don‘t be afraid to experiment. Iterate quickly and allow yourself to make mistakes. That‘s how every data scientist starts their journey. Best of luck!