{"id":2264,"date":"2022-03-16T15:32:34","date_gmt":"2022-03-16T15:32:34","guid":{"rendered":"http:\/\/optimumsportsperformance.com\/blog\/?p=2264"},"modified":"2022-11-08T03:38:06","modified_gmt":"2022-11-08T03:38:06","slug":"fitting-saving-and-deploying-tidymodels-with-cross-validated-data","status":"publish","type":"post","link":"https:\/\/optimumsportsperformance.com\/blog\/fitting-saving-and-deploying-tidymodels-with-cross-validated-data\/","title":{"rendered":"Fitting, Saving, and Deploying tidymodels with Cross Validated Data"},"content":{"rendered":"

I’ve talked about {tidymodels} previously when I laid out a {tidymodels} model fitting template<\/a>, <\/strong>which serves as a framework to wrap up the 10 series screen cast we did on {tidymodels} for Tidy Explained<\/a><\/strong><\/span>.<\/span><\/span><\/p>\n

During all 10 of our episodes, within my model fitting template, and pretty much every single tutorial I’ve seen online, people follow the same initial steps, which are to split the data into a training and testing set and then split the training data into cross validation sets.<\/p>\n

This approach is fine when you have enough data to actually perform a training and testing split. But, there are times where we don’t really have enough data to do this, meaning we are fitting a model to a small training set and then hoping<\/em> it picks up all of the necessary information in order to generalize well to external data.<\/p>\n

In these instances, we may prefer to use all of our available data, split it into cross validation sets, fit and test the model, and then save the model workflow so that it can be deployed later on and used in production.<\/p>\n

To cover this issue, I’ve put together a template for taking a data set, creating cross validation folds, fitting the model, and then saving the model. The code has both a regression and random forest classification model on the mtcars<\/em><\/strong> data set. I’ll only show the regression example below, but all code is available on my GITHUB<\/a><\/strong><\/span> page.<\/p>\n

Load Packages & Data<\/strong><\/p>\n

\r\n### load packages\r\nlibrary(tidymodels)\r\nlibrary(tidyverse)\r\n\r\n############ Regression Example ############\r\n### get data\r\ndf <- mtcars\r\nhead(df)\r\n\r\n<\/pre>\n
Create Cross\u00a0 Validation Folds & Specify Linear Model<\/strong><\/p>\n
\r\n### cross validation folds\r\ndf_cv <- vfold_cv(df, v = 10)\r\ndf_cv\r\n\r\n### specify linear model\r\nlm_spec <- linear_reg() %>%\r\n  set_engine("lm") %>%\r\n  set_mode("regression")\r\n<\/pre>\n
Create the Model Recipe and Workflow<\/strong><\/p>\n
To keep things simple, I wont do any pre-processing of the data. I’ll just set the recipe with the regression model I am fitting.<\/p>\n
\r\n### recipe\r\nmpg_rec <- recipe(mpg ~ cyl + disp + wt, data = df)\r\nmpg_rec\r\n\r\n### workflow\r\nmpg_wf <- workflow() %>%\r\n  add_recipe(mpg_rec) %>%\r\n  add_model(lm_spec)\r\n<\/pre>\n
Control Function to Save Predictions<\/strong><\/p>\n
To save our model predictions using only cross-validated folds, we need to set a control function that will be passed as an argument when we fit our model. Without this argument, we can fit the model using the cross-validated folds but we wont be able to extract the predictions.<\/p>\n
\r\n### set a control function to save the predictions from the model fit to the CV-folds\r\nctrl <- control_resamples(save_pred = TRUE)\r\n\r\n<\/pre>\n
Fit the Model<\/strong><\/p>\n
\"\"<\/a><\/strong><\/p>\n
Evaluate model performance<\/strong><\/p>\n
\r\n### view model metrics\r\ncollect_metrics(mpg_lm)\r\n<\/pre>\n
\"\"<\/a><\/p>\n
Unnest the .predictions column from the model fit and look at the predicted mpg versus actual mpg
\n<\/strong><\/p>\n
\r\n### get predictions\r\nmpg_lm %>%\r\n  unnest(cols = .predictions) %>%\r\n  select(.pred, mpg)\r\n<\/pre>\n
\"\"<\/a><\/p>\n
Fit the final model and extract the workflow<\/strong><\/p>\n
If we are happy with our model performance and the workflow that we’ve built (which contains our pre-processing steps) we can fit final model to the data set.<\/p>\n
To do this, we use the function fit()<\/strong> and pass it our data set and then we use extract_fit_parsnip()<\/strong> to extract the workflow that you’ve created. Then save the workflow as an RDA file to be loaded and used at a later time.<\/p>\n
\r\n## Fit the final model & extract the workflow\r\nmpg_final <- mpg_wf %>% \r\n  fit(df) %>%\r\n  extract_fit_parsnip()\r\n\r\nmpg_final\r\n\r\n## Save model to use later\r\n# save(mpg_final, file = "mpg_final.rda")\r\n<\/pre>\n
\"\"<\/a><\/p>\n
To access all of the code for this template and see an example with a random forest classifier go to my GITHUB<\/a><\/span><\/strong> page.<\/p>\n","protected":false},"excerpt":{"rendered":"
I’ve talked about {tidymodels} previously when I laid out a {tidymodels} model fitting template, which serves as a framework to wrap up the 10 series screen cast we did on {tidymodels} for Tidy Explained. During all 10 of our episodes, within my model fitting template, and pretty much every single tutorial I’ve seen online, people […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[47,45],"tags":[],"class_list":["post-2264","post","type-post","status-publish","format-standard","hentry","category-model-building-in-r","category-r-tips-tricks"],"_links":{"self":[{"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/posts\/2264","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/comments?post=2264"}],"version-history":[{"count":4,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/posts\/2264\/revisions"}],"predecessor-version":[{"id":2272,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/posts\/2264\/revisions\/2272"}],"wp:attachment":[{"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/media?parent=2264"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/categories?post=2264"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/optimumsportsperformance.com\/blog\/wp-json\/wp\/v2\/tags?post=2264"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}