- \n
- Splitting the data in test\/train sets and cross validation folds<\/li>\n
- Setting up the model structure (what tidymodels refers to as a recipe)<\/li>\n
- Creating preprocessing steps<\/li>\n
- Compiling the preprocessing and model structure together into a single workflow<\/li>\n
- Tuning the KNN model<\/li>\n
- Identifying the best tuned model<\/li>\n
- Evaluating the model on the test set<\/li>\n
- Saving the entire worflow and model to be used later on with new data<\/li>\n
- Making predictions with the saved workflow and model on new data<\/li>\n<\/ul>\n
I’ve provided a number of tidymodels tutorials on this blog so feel free to search the blog for those. Additionally, all of my tidymodels tutorials and templates are available in a GitHub Repo<\/a><\/span><\/strong>. Alternatively, if Python is more you jam, I did do a previous blog comparing the workflow in tidymodels to Scikit Learn, which can be found HERE<\/a><\/span><\/strong>.<\/p>\n
This entire tutorial and data are available on my GitHub page<\/a><\/span><\/strong> if you’d like to code along.<\/p>\n
Load & Clean Data<\/strong><\/p>\n
For this tutorial I’m going to use the 2021 FIFA Soccer Ratings, which are available on Kaggle<\/a><\/span><\/strong>. There are several years of ratings there, but we will concentrate on 2021 with the goal being to estimate a player’s contract value based on the various FIFA ratings provided and the position they play.<\/p>\n
First, I load the tidyverse and tidymodels libraries and read in the data. I’m going to drop goalies so that we only focus on players who play the field and, since the data has a large number of columns, I’m going to get only the columns we care about: Information about the player (playerID, name, height, weight, club, league, position, and contract value) and then the various FIFA ratings.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-7.51.00\u202fAM-1024x584.png\")
<\/a><\/p>\nThe only column with missing data that impacts our analysis is the team position column, as this is a feature in the data set. Additionally, there are 207 players with a contract value of 0 (all 195 players missing a team position have a 0 contract value). So, perhaps these are players who weren’t on a club at the time the ratings were assigned.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-7.53.31\u202fAM-1024x561.png\")
<\/a><\/p>\nI’m going to remove these players from our analysis data set, but I’m going to place them in their own data set because we will use them later to make estimates of their contract value.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-7.56.37\u202fAM-1024x163.png\")
<\/a><\/p>\nVisualizing the Data<\/strong><\/p>\n
Let’s take a look at a visual of the contract value for all of the players in our data set. Since this is severely right skewed, we will plot it on the log scale.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-7.58.06\u202fAM.png\")
<\/a><\/p>\nWe also notice there are a variety of positions (way more than I would have guessed in soccer!).<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-7.59.12\u202fAM-811x1024.png\")
<\/a><\/p>\nFinally, we want to explore how playing position might influence contract value.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.00.43\u202fAM-770x1024.png\")
<\/a><\/p>\ntidymodels set up<\/strong><\/p>\n
Now that we have our data organized we want to set up a tidymodels workflow to estimate the contract value of every player using a K-nearest neighbor model.<\/p>\n
First, we split the data in train and test splits and then further split the training data into 5 cross validation folds so that we can tune the model to find the best number of K-neighbors.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.04.47\u202fAM.png\")
<\/a><\/p>\nNext, we specify that we want to use a KNN model and the outcome variable as regression, since it is continuous (as opposed to classification).<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.06.17\u202fAM.png\")
<\/a><\/p>\nNow that the model is specified we need to set up our preprocessing steps. This is one thing that tidymodels is super useful for. Normally, we’d need to do all this preprocessing before fitting the model and, as in the case of KNN and most machine learning models, we’d need to standardize the variables in the training set and then store the means and standard deviations of those variables so that we can use them to standardize the test set or future data. In tidymodels, we set up our preprocessing workflow and store it and it contains all that information for us! We can simply load it and use it for any downstream analysis we want to do. There are three preprocessing steps we want to add to our workflow:<\/p>\n
- \n
- log transform the dependent variable (contract value)<\/li>\n
- Dummy code the positions<\/li>\n
- Standardize (z-score) all off the numeric variable in the model (the ratings)<\/li>\n<\/ol>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.07.02\u202fAM-1024x221.png\")
<\/a><\/p>\nWe can view the preprocessing steps using the prep()<\/strong> function.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.12.32\u202fAM-1024x504.png\")
<\/a><\/p>\n<\/p>\n
With the model specified and the preprocessing recipe set up we are ready to compile everything into a single workflow.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.14.27\u202fAM.png\")
<\/a><\/p>\nWe can look at the workflow to make sure everything makes sense before we start fitting the model.
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.15.15\u202fAM.png\")
<\/a><\/p>\nNext, we tune the model using the cross validated folds that we set up on our training data set. We are trying to find the optimal number of k-neighbors that minimizes the RMSE of the outcome variable (the log of contract value).<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.17.10\u202fAM-1024x771.png\")
<\/a><\/p>\nWe see the results are stored in a list for every one of our 5 cross validation splits. We can view the model metrics for each of the folds.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.17.54\u202fAM-1024x420.png\")
<\/a><\/p>\nThe model with 10 neighbors appears to have the lowest RMSE. So, we want to pull that value directly from this table so that we can store it and use it later when we go to fit the final model.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.19.25\u202fAM.png\")
<\/a><\/p>\nFitting the Final Model Version 1: Using finalize_workflow()<\/strong><\/p>\n
Version 1 that I’ll show for fitting the final model uses the finalize_workflow()<\/strong> function. I like this approach if I’m building my model in a local session and then want to see how well it performs on the test session (which is what this function does). This isn’t the approach I take when I need to save everything for downstream use (which we will cover in Version 2).<\/p>\n
First, fit the best model using the finalize_workflow()<\/strong> function.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.22.32\u202fAM-1024x171.png\")
<\/a><\/p>\n![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.23.28\u202fAM.png\")
<\/a><\/p>\nThen, we get the model predictions for this final workflow on the the test data set.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.24.17\u202fAM.png\")
<\/a><\/p>\nWe can plot the predicted contract value compared to the actual contract value and calculate the test set RMSE.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.26.01\u202fAM-1024x855.png\")
<\/a> ![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.26.11\u202fAM.png\")
<\/a><\/p>\nFitting the Final Model Version 2: Re-sepcify the model on the training set, save it, and use it later
\n<\/strong><\/p>\nIn this second version of fitting the final model, we will take the best neighbors from the tuning phase, re-specify the model to the training set, reset the workflow, and then save these components so that we can use them later.<\/p>\n
First, re-specify the model and notice I set the neighbors<\/strong> argument to best_neighbors<\/strong>, which we saved above when we tuned the model.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.28.40\u202fAM-1024x110.png\")
<\/a><\/p>\nWe then use this finalized\/re-specified model to reset the workflow. Notice it is added to the add_model() <\/strong>function however the recipe, with our preprocessing steps, does not change.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.28.54\u202fAM.png\")
<\/a><\/p>\nWith the workflow set up we now need to do the final two sets:<\/p>\n
- \n
- Fit the model to the entire data set and extract the recipe using extract_recipe()<\/strong> since we will need to save this to preprocess any new data before making predictions.<\/li>\n
- Fit the model to the final data and extract the model itself, using extract_fit_parsnip()<\/strong> so we can use the model to make future predictions.<\/li>\n<\/ol>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.31.08\u202fAM-1024x114.png\")
<\/a>![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.33.09\u202fAM.png\")
<\/a><\/p>\nNow we save the recipe and model as .rda <\/strong>files. We can then load these two structures and use them on new data!<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.35.10\u202fAM.png\")
<\/a><\/p>\nWe have 207 players with 0 contract value, of which 195 had no team position. So, that leaves us with 12 players that we can estimate contract value for with our saved model.<\/p>\n
First, we use the bake()<\/strong> function to apply the saved recipe to the new data so that we can preprocess everything appropriately.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.39.29\u202fAM.png\")
<\/a><\/p>\nWith the new data preprocessed we can now predict the log of contract value with our saved model.<\/p>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.39.56\u202fAM-1024x429.png\")
<\/a>
\nWe can add these predictions back to the new data set, exponentiate the predicted contract value to get it back to the normal scale and then create a visual of our estimates for the 12 players.<\/p>\n - Fit the model to the final data and extract the model itself, using extract_fit_parsnip()<\/strong> so we can use the model to make future predictions.<\/li>\n<\/ol>\n
- Fit the model to the entire data set and extract the recipe using extract_recipe()<\/strong> since we will need to save this to preprocess any new data before making predictions.<\/li>\n
![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.42.12\u202fAM.png\")
<\/a> ![\"\"](\"https:\/\/optimumsportsperformance.com\/blog\/wp-content\/uploads\/2024\/09\/Screenshot-2024-09-30-at-8.42.22\u202fAM-1024x867.png\")
<\/a><\/p>\n