Author Archives: Patrick

TidyX 84: Workflowsets in tidymodels

We’ve been working on developing machine learning models using {tidymodels} over the past several weeks. Sometimes, though, you need to build a variety of models on your data. This week, Ellis Hughes and I explore {workflowsets}.

Workflow sets allow you to set specific recipes for different model types and then run a variety of models simultaneously. Topics we cover:

  • Setting up recipes
  • Specifying multiple models
  • Creating a workflowset
  • Fitting all models within the workflowset
  • Choosing the optimal model
  • Two different approaches to fitting the optimal model on your test data set

To watch our screen cast, CLICK HERE.

To access the code and data (courtesy of kaggle), CLICK HERE.

If you like what we are doing and would like to buy us a coffee or beer, visit our Patreon page HERE.

TidyX 83: Naive Bayes Classifier using {tidymodels}

This week, Ellis and I add another classification model to our {tidymodels} series. Here, we walk through the framework of creating a Naive Bayes Classifier for classification purposes.

To watch our screen cast, CLICK HERE.

To access our code, CLICK HERE.

If you like what we are doing with the screen cast and want to support our work and buy us a beer or coffee, check out our Patreon page!

TidyX 82: Random Forest Classifier using {tidymodels}

This week, Ellis Hughes and I extend our {tidymodels} series by building a random forest classifier on the Palmer Penguins data set.

Some things we cover:

1. Continuing to refine our {tidymodels} frame work
2. Different approaches to setting up a tuning grid
3. Finalizing your workflow
4. Plotting ROC Curves for multi-class classification problems

To watch our screen cast, CLICK HERE.

To access our code, CLICK HERE.

TidyX 81: tidymodels logistic regression

This week, Ellis Hughes and I start exploring classification algorithms in {tidymodels}. We set up a logistic regression using NHL data to forecast whether a team will make the playoffs or not. Like all of our {tidymodels} episodes, we discuss:

  1. Initializing the model
  2. Splitting the data into training and test sets
  3. Creating cross-validation folds of the training data
  4. Setting up a model recipe
  5. Creating a model workflow
  6. Building and evaluating the model on the cross validation folds
  7. Fitting the model to the test data
  8. Evaluating the model predictions

To view our screen cast, CLICK HERE.

To access our code, CLICK HERE.

Also, if you enjoy our screen casts and find the useful, we have created a patreon page.

R Tips & Tricks: Excel Within Column Iteration in R (part 2)

Earlier this week I shared a method for doing within column iteration in R, as you might do in excel. For example, you want to create a new column that requires a starting value from a different column (or a 0 value) and then requires new values within that column to iterate over the prior values. The way I handled this was to use the accumuate() function available in the {tidyverse} package.

The article got some good feedback and discussion on Twitter. For example, Thomas Mock, provided some examples of using the {slider} package to handle window functions, see HERE and HERE. The package looks to be very handy and easy to use. I’m going have to play around with it some more.

Someone else asked, “how might we do this in a for() loop?”

It’s a good question. Sometimes you might need to use base R or sometimes the for() loop might be easier. So, let’s walk through an example:

Simulate Data

First, we need to simulate a basic data set:

library(tidyverse)

df <- tibble(
  id = 1:5,
  val = c(5,7,3,4,2)
)

df

Setting Up the Problem

Let’s say we want to create a new value that applies a very simple algorithm:

New Value = observed + 2 * lag(new value)

Putting the above data in excel the formula and answer looks like this:

Notice that the first new value starts with our initial observation (5) and then begins to iterate from there.

Writing the for() loop

for() loops can sometimes be scary but if you sequentially think through what you are trying to do you can often come up with a reasonable solution. Let’s step through this one:

  1.  We begin outside of the for() loop by creating two elements. We create N which simply gives us a count of the number of observations in our val column and we create new_val which is nothing more than a place holder for the new values we will create. Notice that the new_val place holder starts with the first element of the df$val column because, remember, we need to begin the new column with our first value observation in the val column. After that, I simply concatenate a bunch of NA values that will be populated with the new values that the for() loop will produce. Notice that I have NA repeat for N-1 times. This is important, as represents the number of observations in the val column and since we’ve already put a place holder in for the first observation we need to remove one of the NA’s to ensure the new_val column will be the same length as the val column.
  2. Next, we create our loop. I specify that I want to iterate over all “i” iterations from 2 to N. Why 2? Because the first value is already specified, as discussed above. Inside the for() loop, for each iteration that the loop runs it will store the new value, “i” in the new_val vector we created above. The equation that we specified earlier is within the for loop and I use “i” to index the observations. For example, for the second observation, what the for() loop is doing is saying, df$val[2] + new_val[2 – 1]*2, and for the third time through the loop it says, df$val[3] + new_val[3 – 1]*2, etc. until it goes through all N observations. Everything in the brackets is simply specifying the row indexes.
## We want to create a new value
# New Value = observed + 2 * lag(new value)
# The first value for the new value is the first observation in row one for value

N <- length(df$val)
new_val <- c(df$val[1], rep(NA, N-1))

for(i in 2:N){

  new_val[i] <- df$val[i] + new_val[i - 1]*2
  
}

 

Once the loop is done running we can simply attach the results to our data frame and see what it looks like:

Same results as the excel sheet!

Wrapping this into a function

After seeing how the for() loop works, you might want to wrap it up into a function so that you don’t need to do the first steps of creating an element for the number of iterations and vector place holder. Also, having it in a function might be useful if you need to frequently use it for other data sets.

We simply wrap all of the steps into a single function that takes an input of the data frame name and the value column that has your most recent observations. Run the function on the data set above and you will obtain the same output.

iterate_column_func <- function(df, val){
  
  N <- length(df$val)
  new_val <- c(df$val[1], rep(NA, N-1))
  
  for(i in 2:N){
    
    new_val[i] <- df$val[i] + new_val[i - 1]*2
    
  }
  
  df$new_val <- new_val
  return(df)
}

iterate_column_func(df, val)


Applying the function to multiple subjects

What if we have more than one subject that we need to apply the function to?

First, we simulate some more data:

df_2 <- tibble(
  subject = as.factor(rep(1:10, each = 5)),
  id = rep(1:5, times = 10),
  val = round(runif(n = 50, min = 10, max = 20), 0)
)

df_2

Next, I’m going to make a slight tweak to the function. I’m going to have the output get returned as a single column data frame.

iterate_column_func <- function(x){
  
  N <- length(x)
  new_val <- c(x[1], rep(NA, N-1))
  
  for(i in 2:N){
    
    new_val[i] <- x[i] + new_val[i - 1]*2
    
  }
  
  new_val <- as.data.frame(new_val)
  return(new_val)
}

Now, I’m going to apply the custom function to my new data frame, with multiple subjects, using the group_modify() function in {tidyverse}. This function allows us to apply other functions to groups of subjects, iterating over them and producing a data frame as a result.

 

new_df <- df_2 %>%
  group_by(subject) %>% 
  group_modify(~iterate_column_func(.x$val)) %>%
  ungroup()

Then, I simply bind this new data to the original data frame and we have our new_val produced within individual.

df_2 %>%
  bind_cols(new_df %>% select(-subject)) %>% as.data.frame()

Conclusion

And there you go, within column iteration in R, just as you would do in excel. Part 1 covered an approach in {tidyverse} while Part 2 used for() loops in base R to accomplish the same task.

The full code for this article is available on my GitHub page.