# Doing Data Exploration the Right Way: Insights from Analyzing GRE Scores and Graduate School Admissions

- Canonical: https://33rdsquare.com/doing-data-exploration-the-right-way-gre-scores-case-study/
- Published: 2024-09-03
- Author: Jordan Brown
- Categories: [Artificial Intelligence & Machine Learning & ChatGPT](https://33rdsquare.com/category/tech/ai/)

---

Data exploration is a crucial first step in any data science or machine learning project. Before diving into building complex models, it‘s important to thoroughly understand the data you‘re working with – its distribution, relationships between variables, data quality issues, and more. Proper data exploration can uncover valuable insights that inform feature engineering, model selection, and overall project strategy.

In this post, we‘ll walk through a case study of exploring a dataset of GRE scores and graduate school admissions to illustrate key concepts and best practices of data exploration. Through a combination of visualizations and statistical analysis, we‘ll seek to answer the question: how important are GRE scores and other factors in determining an applicant‘s chance of being admitted to graduate school?

## The Data

The dataset we‘ll be working with contains information on 400 applicants to graduate school, including:

- GRE Scores (out of 340)
- TOEFL Scores (out of 120)
- University Rating (out of 5)
- Statement of Purpose Strength (out of 5)
- Letter of Recommendation Strength (out of 5)
- Undergraduate GPA (out of 10)
- Research Experience (either 0 or 1)
- Chance of Admit (ranging from 0 to 1)

Let‘s read in the data and take a high-level look at it:

![DataFrame Head](https://i.imgur.com/l3fwFcF.png)

We can see that the dataset contains the expected columns, with a mix of continuous, discrete, and binary variables. We‘ll want to keep the different data types in mind as we explore each variable.

## Checking for Missing Data

Before diving into analysis and visualization, it‘s important to check for missing or null values in the dataset. Missing data can throw off calculations and visualizations, so it‘s best to identify and handle it early.

Luckily, running `df.isnull().sum()` shows that there are no missing values in this dataset:

```
Serial No.    0
GRE Score     0
TOEFL Score   0
University Rating  0
SOP           0
LOR           0
CGPA          0
Research      0
Chance of Admit    0
dtype: int64
```

## Exploring Relationships between GRE Scores and Chance of Admit

One of the key questions we want to answer is: how much do GRE scores affect an applicant‘s chance of being admitted to graduate school? Let‘s start by visualizing the relationship between these two variables.

![GRE vs Chance of Admit \- Scatter](https://i.imgur.com/DQYwbiu.png)

The scatter plot shows a generally positive, linear relationship between GRE score and chance of admit – applicants with higher GRE scores tend to have a higher chance of admission. However, there is quite a bit of variance, suggesting other factors are at play as well.

We can quantify the strength of the relationship by calculating the correlation coefficient:

```
df.corr()[‘Chance of Admit ‘][‘GRE Score‘]

0.802108
```

The correlation of 0.802 confirms a strong positive relationship between GRE scores and chance of admit. In fact, looking at the full correlation matrix shows that GRE score has the second highest correlation with chance of admit after undergraduate GPA:

![Correlation Heatmap](https://i.imgur.com/rXXcYYl.png)

To further visualize the distribution of each variable and how they relate, we can use a joint plot with marginal histograms:

![GRE vs Chance of Admit \- Joint Plot](https://i.imgur.com/VAFSVmS.png)

The plot reveals that both GRE scores and chance of admit are roughly normally distributed, with most scores between 300-340 and most admission chances between 0.5-0.9.

So in summary, GRE scores are strongly related to chance of graduate school admission, but they don‘t tell the whole story. Let‘s bring in some of the other variables to see how they affect the relationship.

## Factoring in Research Experience and University Rating

Two other factors that seem likely to impact chance of admit are whether the applicant has research experience and the rating of the university they are applying to. Let‘s visualize how these variables interact with GRE scores and chance of admit.

First, we‘ll use color coding to examine research experience:

![GRE vs Chance of Admit by Research Experience](https://i.imgur.com/lBv8I6r.png)

The plot reveals that applicants with research experience (orange points) have a higher chance of admission on average compared to those without (blue points). However, those with research experience also tend to have higher GRE scores, so the effect is somewhat confounded.

We can attempt to detangle the effects by looking at the difference in average chance of admit for applicants with and without research experience, controlling for GRE score range:

```
dfresearch_high =‘s_bucket‘] <= 340) & (df[‘GRE Score_bucket‘] >= 320)]
print("Average chance of admit for 320-340 GRE scores:")
print("With research experience: ", research_high[‘Chance of Admit ‘].mean())
print("Without research experience: ", no_research_high[‘Chance of Admit ‘].mean())

Average chance of admit for 320-340 GRE scores:
With research experience:  0.820489
Without research experience:  0.738714
```

Among applicants with high GRE scores between 320-340, those with research experience have about an 8% higher chance of admission on average. This suggests that research experience does provide a modest boost, but high GRE scores are still important.

We can do a similar analysis factoring in university rating:

![GRE vs Chance of Admit by University Rating](https://i.imgur.com/7FtXCRy.png)

Not surprisingly, applicants to the highest rated universities (green points) have the highest chance of admit on average, followed by those with slightly lower ratings (orange and purple points).

To quantify the impact of university rating, we can perform ANOVA to test whether the average chance of admit is significantly different across the rating categories:

```
import statsmodels.api as sm
from statsmodels.formula.api import ols

model = ols(‘Q("Chance of Admit ") ~ C(Q("University Rating"))‘, data=df).fit()
anova_table = sm.stats.anova_lm(model, typ=2)
anova_table

                     sum_sq    df         F        PR(>F)
C(Q("University Rating"))  17.618079   4.0  884.032581  9.816847e-232
Residual               0.785491  395.0        NaN           NaN
```

The very low p-value suggests that university rating does have a significant effect on chance of admit, confirming our visual analysis.

## Building Predictive Models

While visualizing relationships is very useful for understanding the data, we often want to go a step further and build models to predict one variable based on others. In this case, we might want to predict chance of admit based on GRE scores and other factors.

Let‘s start with a simple linear regression model using only GRE scores as the independent variable:

```
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score, mean_squared_error

X = df[[‘GRE Score‘]].values
y = df[‘Chance of Admit ‘].values

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

model = LinearRegression()
model.fit(X_train, y_train)
print("Train R^2 Score : ", model.score(X_train, y_train))
print("Test R^2 Score : ", model.score(X_test, y_test))

Train R^2 Score :  0.619305
Test R^2 Score :  0.681422
```

The R^2 score of 0.68 on the test set indicates that GRE scores alone explain about 68% of the variance in chance of admit. Not bad for a single variable model, but let‘s see if we can improve it by adding more predictors.

We‘ll use the random forest model this time to capture potentially non-linear relationships:

```
from sklearn.ensemble import RandomForestRegressor

features = [‘GRE Score‘, ‘TOEFL Score‘, ‘University Rating‘, ‘SOP‘,
            ‘LOR‘, ‘CGPA‘, ‘Research‘]
X = df[features].values
y = df[‘Chance of Admit ‘].values

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

model = RandomForestRegressor(n_estimators=100, max_depth=3)
model.fit(X_train, y_train)
print("Train R^2 Score : ", model.score(X_train, y_train))
print("Test R^2 Score : ", model.score(X_test, y_test))

Train R^2 Score :  0.985801
Test R^2 Score :  0.818950
```

With all the variables included, the random forest model achieves an impressive 0.82 R^2 score on the test set. The model fit is more complex with multiple variables, so there are risks of overfitting, but techniques like cross-validation and limiting the tree depth can help mitigate that.

To get a sense of which features the model finds most important, we can plot the feature importances:

![Random Forest Feature Importances](https://i.imgur.com/wM3SYrm.png)

The model confirms that undergraduate GPA and GRE scores are the most important factors, but university rating, research experience and other factors also contribute to the predictions.

Finally, let‘s test the model on some new hypothetical applicants:

```
test_data = [[337, 118, 4, 4.5, 4.5, 9.65, 1],
             [324, 107, 4, 4.0, 4.5, 8.87, 1],
             [316, 104, 3, 3.0, 3.5, 8.22, 1],
             [322, 110, 3, 3.5, 2.5, 8.67, 0],
             [314, 103, 2, 2.0, 3.0, 8.21, 0]]

model.predict(test_data)

array([0.845, 0.76 , 0.636, 0.577, 0.434])
```

The model predicts fairly high chances of admission for applicants with high stats, and lower chances for those with lower stats, especially from universities with lower ratings and without research experience. Of course, the exact predictions would vary depending on the model used and training data, but this gives a sense of how a predictive model could be applied to new data points.

## Conclusions and Takeaways

Through our exploratory analysis, we‘ve seen that GRE scores are an important factor in graduate school admissions, with a strong positive correlation with chance of admit. However, GRE scores alone don‘t tell the whole story. Other factors like undergraduate GPA, university rating, and research experience also play a significant role.

Building predictive models allowed us to quantify the impact of multiple variables and make predictions on new data points. While more complex models were able to capture nuances in the data, it‘s important not to overfit and to choose models carefully.

This analysis provides a framework for exploring academic admissions data, but the same principles of visualizing relationships, statistical testing, and predictive modeling can be applied to all sorts of datasets.

Some key takeaways and best practices for data exploration include:

- Always check for data quality issues like missing values before analysis
- Use a variety of visualizations to understand distributions and relationships
- Quantify correlations and use techniques like ANOVA to determine statistically significant factors
- Start with simple models and add complexity gradually to avoid overfitting
- Carefully split data into train and test sets and evaluate model performance on held-out data
- Look for possible interaction effects and confounding factors
- Keep in mind the limitations of your data and scope of your conclusions

Data exploration is a key skill for any data scientist or analyst to build. Taking the time to deeply understand your data upfront will make the rest of the data science pipeline much smoother and lead to more reliable, valuable insights in the end.

---

Source: [Doing Data Exploration the Right Way: Insights from Analyzing GRE Scores and Graduate School Admissions](https://33rdsquare.com/doing-data-exploration-the-right-way-gre-scores-case-study/)