Predicting Soccer Player Positions with Keras Multi-Class Classification

Soccer is the world‘s most popular sport, played and watched by billions across the globe. The rise of sports analytics has led to large amounts of data being collected on professional players and their attributes and performance. This explosion of soccer data opens up exciting opportunities to apply data science and machine learning techniques to gain insights.

In this article, we‘ll walk through a machine learning project focused on utilizing a dataset of FIFA soccer player stats to build a neural network model that can classify the position a player plays based on their abilities and characteristics. We‘ll leverage the popular Keras deep learning library to construct our multi-class classification model.

Let‘s dive in and see how we can teach an AI system to recognize different soccer positions from data!

The FIFA Player Dataset

EA Sports‘ FIFA video game series is one of the most popular sports video game franchises. The game developers put significant effort into scouting and collecting detailed data on professional soccer players from around the world in order to realistically replicate them in-game. This data covers player physical attributes, abilities, past performance and more.

Luckily for data science enthusiasts, this rich dataset is made publicly available, allowing for interesting machine learning and analytics projects. For this article, we‘ll be using the data for the FIFA 19 edition of the game which contains over 18,000 players from numerous leagues worldwide. The dataset has 89 attributes for each player, including:

• Player information like name, age, nationality, club, etc.
• Physical attributes like height, weight, acceleration, sprint speed
• Technical skill attributes like ball control, dribbling, passing, shooting
• Mental attributes like aggression, vision, composure
• Goalkeeper specific attributes like GK diving, GK handling, GK reflexes
• Player personality traits like attack/defense work rate

The target variable we‘re interested in is the player‘s position, which is classified into 4 main categories:

• Forward (FW): Strikers, Wingers, Center Forwards
• Midfielder (MF): Center Midfielders, Attacking Midfielders, Defensive Midfielders, Wide Midfielders
• Defender (DF): Center Backs, Full Backs, Wing Backs
• Goalkeeper (GK)

With this dataset, we can build a model to predict a player‘s most suitable position based on their abilities and style of play. This could be useful for identifying talent, scouting players, assembling teams with the right balance of skills, and more. It‘s a great example of a multi-class classification machine learning problem.

Data Preprocessing

Before we can feed the data into a machine learning model, we need to clean it up and put it in the proper format. Key preprocessing steps include:

• Removing rows with missing values for the target "Position" variable
• Dropping columns that aren‘t relevant to our specific modeling task
• Converting categorical string values for position to numeric labels
• Scaling numeric features to similar range to avoid issues in model training

Here‘s a example of how we filter the dataset to only the columns we need and assign numeric labels to the positions:

df = df[["Position", ‘Finishing‘, ‘HeadingAccuracy‘, ‘ShortPassing‘, ‘Volleys‘, ‘Dribbling‘,       
‘Curve‘, ‘FKAccuracy‘, ‘LongPassing‘, ‘BallControl‘, ‘Acceleration‘,       
‘SprintSpeed‘, ‘Agility‘, ‘Reactions‘, ‘Balance‘, ‘ShotPower‘,       
‘Jumping‘, ‘Stamina‘, ‘Strength‘, ‘LongShots‘, ‘Aggression‘,       
‘Interceptions‘, ‘Positioning‘, ‘Vision‘, ‘Penalties‘, ‘Composure‘,       
‘Marking‘, ‘StandingTackle‘, ‘SlidingTackle‘, ‘GKDiving‘, ‘GKHandling‘,       
‘GKKicking‘, ‘GKPositioning‘, ‘GKReflexes‘]]

forward_player = ["ST", "LW", "RW", "LF", "RF", "RS","LS", "CF"]
midfielder_player = ["CM","RCM","LCM", "CDM","RDM","LDM", "CAM", "LAM", "RAM", "RM", "LM"]  
defender_player = ["CB", "RCB", "LCB", "LWB", "RWB", "LB", "RB"]

df.loc[df["Position"] == "GK", "Position"] = 0
df.loc[df["Position"].isin(defender_player), "Position"] = 1
df.loc[df["Position"].isin(midfielder_player), "Position"] = 2 
df.loc[df["Position"].isin(forward_player), "Position"] = 3

We use sklearn‘s StandardScaler to scale the numeric features. Finally, we split the data into train and test sets, using an 80/20 split.

Multi-Class Classification with Neural Networks

Our FIFA player position prediction is a multi-class classification problem, since we have 4 distinct position labels we want our model to predict. Some popular machine learning models for multi-class classification include:

• Logistic Regression
• Decision Trees & Random Forests
• Support Vector Machines
• K-Nearest Neighbors
• Neural Networks

We‘ll opt for a neural network model for this problem, as they are very flexible, powerful models that can pick up on complex non-linear patterns in the data. The specific neural network type we‘ll use is a multi-layer feed forward network, which takes in input data, passes it through multiple hidden layers of neurons to learn hierarchical representations, and outputs a prediction.

Keras is a popular Python deep learning library that makes defining and training neural networks simple. It provides an intuitive, object-oriented API to build models by stacking layers and offers a choice of training backends, including TensorFlow.

Building the Keras Model

Keras offers two main approaches to building models – a Sequential API for stacking layers and a Functional API for more complex architectures. For our straightforward feed forward network, the Sequential model is a great choice.

Here‘s the code to define our network architecture:

model = Sequential()
model.add(Dense(60, input_shape = (33,), activation = "relu"))
model.add(Dense(15, activation = "relu")) 
model.add(Dropout(0.2))
model.add(Dense(4, activation = "softmax"))

model.compile(Adam(lr = 0.01), "categorical_crossentropy", metrics = ["accuracy"])

Let‘s break this down:

• We instantiate a Sequential model
• The first layer is a fully-connected (Dense) layer with 60 neurons. It expects 33 input features and uses a ReLU activation function
• The second hidden layer has 15 neurons, also using ReLU
• We use Dropout for regularization, which will randomly disable 20% of neurons during training to prevent overfitting
• The final output layer has 4 neurons (one per class) and a softmax activation, which squashes the outputs to a probability distribution
• We compile the model specifying the Adam optimizer with a learning rate of 0.01, the categorical cross-entropy loss often used for multi-class problems, and accuracy as an evaluation metric

With our model architecture defined, we can go ahead and train it on our data!

Training the Model

To train the model, we simply call fit() on it, passing in our feature matrix X and label vector y, the number of epochs (complete passes through the data) to train for, as well as the validation data to evaluate performance on after each epoch.

history = model.fit(X_train, y_train,
                    epochs = 10, batch_size=128,  
                    validation_data=(X_valid, y_valid))

We use 10 epochs and a batch size of 128. After training, the model reached 88% validation accuracy, not bad! We can plot the learning curves to see how the model‘s performance evolved during training.

Evaluating Performance

With our trained model, we can now evaluate its performance on the held-out test set. We‘ll look at:

• Overall accuracy score
• Confusion matrix breaking down predictions for each class
• Classification report with precision, recall, F1 score for each class

First we generate predictions on the test data:

y_pred = model.predict(X_test)  
y_pred_classes = np.argmax(y_pred, axis = 1)

Accuracy:

from sklearn.metrics import accuracy_score

accuracy_score(y_test, y_pred_classes)
>>> 0.868

Our model reaches 86.8% test accuracy, which aligns with the validation accuracy seen during training.

Confusion Matrix:

from sklearn.metrics import confusion_matrix

confusion_matrix(y_test, y_pred_classes)  
>>>
array([[230,  14,   2,   0],
       [ 22, 682,  50,   1],
       [  2,  57, 636,  25],
       [  1,   2,  31, 202]])

The confusion matrix provides a detailed breakdown of our classification results. We can observe some key takeaways:

• The model does best at predicting the Midfielder and Defender classes, which had the most number of training examples
• It occasionally confuses Defenders and Midfielders or Midfielders and Forwards since players in these neighboring positions can have overlapping skillsets
• Goalkeepers are easy to distinguish based on their unique attributes and are rarely misclassified

Classification Report:

from sklearn.metrics import classification_report

print(classification_report(y_test, y_pred_classes))
>>>
              precision    recall  f1-score   support

           0       0.90      0.93      0.92       246
           1       0.90      0.90      0.90       755
           2       0.88      0.89      0.89       720
           3       0.89      0.86      0.87       236

    accuracy                           0.90      1957
   macro avg       0.89      0.89      0.89      1957 
weighted avg       0.90      0.90      0.90      1957

The report shows high precision and recall scores of close to 90% across all 4 classes, validating that the model is able to accurately identify the different player positions. The slightly lower scores for Forwards aligns with what we saw in the confusion matrix, where that class had the most misclassifications, likely due to similarities with Midfielders.

Conclusion and Next Steps

In this article, we walked through an end-to-end multi-class classification model on the FIFA 19 player dataset. We used a Keras feed forward neural network to predict a player‘s position based on their skill and ability attributes. The model was able to achieve strong results, with accuracy close to 90% across positions.

There are a number of ways we could expand on this work:

• Expanding to use the full dataset across multiple years of the game
• Experiment with other algorithms like random forests or gradient boosting machines
• Adding more granular position classes beyond the 4 high-level categories
• Deploying the trained model into a web app or tool that lets you input a player‘s stats and get a position prediction

The complete code for this analysis is available on GitHub: https://github.com/Siddharth1698.

I hope this was a helpful overview of applying neural networks to a multi-class classification task on an interesting real-world dataset! Feel free to connect with me on LinkedIn or GitHub. Happy coding!