Category Archives: Strength & Conditioning

The Effects of High vs. Low Protein Intake on Body Composition — Simulating a Previous Study

This week I was listening to Layne Norton’s Podcast Interview with Bill Campbell on training and nutrition research. I found it to be an interesting listen. Layne has been in this space for a long time (he and I started out on the same internet forum back in the early 2000’s, Avant Labs — RIP Patrick Arnold). Layne always does a nice job at trying to combat a lot of the nonsense that takes place in the health and fitness space (with internet influencers pedaling whatever the latest supplement or training program is). I also have enjoyed much of the work that Bill Campbell has put out (we even published a paper together many years ago).

In this podcast, the two get into a discussion about protein intake and Bill makes the statement that you can overeat your calories and as long as the surplus is coming from protein the evidence seems to suggest that you’d be hard pressed to put on excess body fat and, in fact, gain some muscle. Layne is skeptical of this and the two discuss a few potential hypotheses of why this may be, ultimately concluding that Layne would do a bit of a deep dive into the literature — I’ll be interested to see what he finds.

During the discussion, Bill referenced one of his papers, Effects of high vs. low protein intake on body composition and maximal strength in aspiring female physique athletes engaging in an 8-week resistance training program. I decided to pull the paper and give it a read. In the process of reading it, I decided to pull out my laptop and write a simulation of aspects of the study (primarily calories and fat free mass) to better understand the results, explore uncertainty (it was only 17 subjects) and create a slightly different model (Bayesian hierarchical model) to show how we can understand more about where the variance is coming from and what we can learn from that.

I wont hash out al of the code here. If you want the full coded simulation you can download it from my GITHUB page and knit it from there if you want to also see it in an HTML form.

With that said, let’s dive in…

Introduction

  • Individuals participating in a regimented resistance training program require more protein than that of sedentary individuals. Recent research has suggested that requirements may be as high as 2.2 g/kg/day (1g/lb/day) in a young bodybuilder population.
  • However, longitudinal studies are lacking and many of the studies conducted up to this point have some methodological flaws (e.g., unsupervised training programs). Additionally, there is a paucity of such research in female populations.

Study Aim: The purpose of this study was to investigate the effects of higher versus lower daily protein intakes on body composition changes in aspiring female physique athletes following a supervised daily undulating periodized resistance training program.

Conclusions up front

NOTE: If you don’t want to read the nitty-gritty down below the primary take away from the study is on protein, calories, and increases in lean mass and decreases in fat mass:

  1. A high-protein diet of 2.4 grams protein/kg/day (1.1g/lb/day) increased Fat Free Mass compared to a low-protein diet of 1.2 grams protein/kg/day (0.5g/lb/day)
  2. These findings are similar to those of prior studies — though many of these studies have limitations such as the duration of time the study can be run, the population studied, the inability to control other aspects of diet or sometimes training, and the issue with sample size (most higher level trainees are not looking to be a part of a study where they could potentially end up in a control group and be forced to train and eat sub-optimally for 8-12 weeks)
  3. Importantly, the high-protein group also saw increased calorie intake compared to the low-protein group. Despite this, they still lost more body fat and gained more muscle mass. So, calories went up (with the excess of calories coming from protein) and body composition results improved!! It was hypothesized that this may be due to the increased calories from additional protein being used to help build lean tissue (as opposed to being stored as adipose tissue) and the higher levels of protein potentially having a positive influence on the energy expenditure side of the energy balance equation.

Goal of this simulation

  • The aim of this simulation is to try and recreate some of the analysis and better understand the observed data from the 17 participants.
  • The goal of this simulation is not to say that the study is wrong or incorrect or point out flaws. Generally, I am in favor of high protein diets (I’m a meathead at heart) and the results from this study are part of a bigger body of literature on protein intake, which despite study limitations (every study has some) generally seems to favor higher levels of protein for those with strength and physique based goals.

Methods

Design

  • Parallel group, repeated measures design
  • Subjects were randomized to a high protein or low protein diet group in conjunction with a supervised resistance training program for 8-weeks
  • Participants were healthy aspiring female physique athletes who were required to have been resistance training for the prior three months or longer and needed to be able to deadlift 1.5x body weight

Diet Groups

  • High Protein Group (n = 8) = 2.4 grams protein/kg/day (1.1g/lb/day)
  • Low Protein Group (n = 9) = 1.2 grams protein/kg/day (0.5g/lb/day)
  • NOTE: Subjects tracked all food intake but no restrictions were placed on dietary carbohydrate or fat intake.

Workouts

  • 8-week training program (28 sessions)
  • 2 upper body and 2 lower body sessions per week
  • Lower body consisted of 5 exercises per session
  • Upper body consisted of 6 exercises per session
  • Sets and reps varied throughout the program and included ranges such as 3-5 reps, 9-11 reps, 14-16 reps
  • Subjects self-selected loads that would allow them to complete the appropriate number of reps within the specified rep range allowing for one rep in reserve
  • Subjects also performed a high intensity interval program consisting of 30-sec sprints with 2min of rest between sets using their modality of choice (treadmill, sprinting, cycle ergometer, rowing machine, etc)

Statistics

  • Dependent Variables
    • Body Comp (fat-free mass, fat mass, body fat%)
    • Max strength (back squat and deadlift)
    • Resting metabolic rate
  • Nutrition intake data was analyzed with independent sample t-tests
  • Data for the dependent variables were analyzed with a 2-group (high vs moderate protein) * 2-time point (pre- and post-training) between-within factorial ANOVA with repeated measures

Results

  • This result section leaves a lot to be desired. It is literally a paragraph of about 30 words that basically say, “see the tables”. The authors then talk about the results in the discussion section as they pertain to prior research. However, the discussion section should be for the researchers to talk about their interpretation of the research (what they found, why they think the results were the way they were, how the results compare with other studies, what potential research should be done in the future to better understand the phenomenon, limitations of the study, etc). So, we don’t have a clear explanation of the observed data in the results section.
  • To try and better understand the data, we will recreate the analysis for Kcals (Table 1) and then build a simulation to better understand the analysis for Fat Free Mass (Table 2).

KCals

  • Kcals were analyzed with an independent t-test
  • Table 1 of the paper provides the pre- and post-mean Kcals for both groups and indicates a statistically significant effect.
  • We can easily calculate the change score for both groups (the change from pre to post). But, to calculate the standard deviation of the change score, which we need when conducting an independent t-test, we need to use a little bit of math to help us. We will estimate the standard deviation of the change scores for both groups using the pre- and post-test standard deviations provided in the table and then take a guess that the correlation between pre- and post-tests is relatively high, say, `r = 0.85`.

We will investigate the Kcals in two different ways:

  1. Calculate a t-test by hand
  2. Calculate a Cohen’s D Effect Size Statistic to help us understand the magnitude of the effect

T-test

Cohen’s D Effect Size

  • From the t-test we conclude that the high protein group consumed, on average, 498 more Kcals (+/- 121.3) than the low protein group. It’s important to note that most of these extra calories came from protein and, although the high protein group increased their calories they actually lost body fat and increased muscle mass (as we will see below) relative to the low protein group!
  • Cohen’s D tells us that the magnitude of the effect ranges from medium to large, indicating that the high protein group saw a rather large increase in calories relative to the low protein group

Analyzing Fat Free Mass

  • This is where things get interesting. We need the actual raw data to run a between-within factorial ANOVA with repeated measures. Soooo…that means we will have to simulate the study (since we don’t have raw data), which is tricky because we don’t know enough about the sources of variance between and within the groups. So, we will need to make some assumptions.
  • Looking at Fat-Free Mass from Table 2, we see that the author’s produced a Cohen’s d effect size for the change in fat-free mass from pre- to post-testing for both the high- and low-protein groups. Let’s first calculate those using the function we built above.

High Protein Group Effect Size

Low Protein Group Effect Size

  • Using the data from Table 2 of the paper, we easily re-create the point estimate for Cohen’s d. The authors did not produce the confidence interval around the effect statistic, but that shouldn’t stop us! We can see from our outputs above, in both cases the 95% Confidence Interval around the observed Cohen’s d ranges from negative to positive. This indicates that, while the author’s did find a statistically significant effect for the change in Fat-Free Mass, there is some uncertainty around the magnitude of this change. The high-protein group did have a standard deviation that was nearly 2x larger than that observed for the low-protein group. So, we don’t really know how the individuals in the high-protein group looked. Perhaps a few of them saw a really large effect, a few of them saw an effect more similar to that of the low-protein group, and a few of them saw a worse effect? We simply don’t know with the data being presented as it is. A change looks like it took place but their analysis doesn’t allow us to tease out the within-subject variability. Perhaps some people saw better results (for any number of reasons) than others. We’d love to know this!

Simulating the Fat Free Mass variable

  • We will turn to simulation to get a grasp of what might be taking place in this study. We know that the author’s find a statistically significant effect, but we want to try and put into perspective how certain we should be (after-all, it is only 17 subjects).

  • Our simulation looks similar to the results presented in the study for FFM. We see that the high-protein group increased their FFM be 2.2kg, on average, while the low-protein group only increased by 0.85 kg, on average.
  • From the plots we can see thas that in the simulated data, a few of the subjects in the high-protein group had very dramatic increases in FFM, while some of the others saw smaller improvements and one appeared to stay the same. The simulated low-protein group shows two of the subjects actually decreased their FFM. one stayed relatively the same, and the others saw small increases in FFM.

Model

  • Next, we take our simulated data set and we construct a model. The authors used a between-within factorial ANOVA with repeated measures. Instead, of doing this, I’m going to build a hierarchical mixed model so that we can better explore the the sources of variance in the data. We will use a Bayesian hierarchical model so that we can get the full posterior distribution. Our fixed effects will be group, time, and a group*time interaction and our random effects will consisted of a random intercept slope across time points for each subject.

  • The model recovers the parameters that we used for our simulation (the ground truth) as it should, as this is how we built the model.
  • The group*time interaction indicates the difference in the slope (change) between the two groups from pre to post. Here, -1.3 indicates that the low protein group experienced 1.3kg less improvement in FFM than the high-protein group.
  • In Table 2 we see that the high-protein group increased FFM, on average, by 2.0kg and the low-protein group by 0.6kg, on average, which is a -1.4kg difference between the groups (`0.6 – 2.0 = -1.4`), which is very closer to our interaction of -1.3

We can get credible intervals for the mixed effects from the model

  • Here, the 90% credible interval for the group*time interaction ranges from -2.7 to 0.09. So, we have a rather wide range of uncertainty in the credible interval, where it may be plausible that there is no difference in the two groups during the post-test FFM. There is 90% confidence that the change in FFM differs between the low and high group from -2.8 kg to nearly a 0.1 kg increase in FFM for the low-protein group (compared to the high protein group).

We can also plot the posterior distribution for the group*time interaction

Random Effects

  • From our model output, the random effects (Error Terms) tell is that the between subject standard deviation is 2.8 and the within subject standard deviation (Residual) is 1.1. We also see that the random effect for the time_point slope is 0.95.
  • We can get a table of the regression equation for each subject, incorporating their random intercept and slopes (notice how Intercept and time_pointpost differ for each)

  • We can plot each subjects random effects intercept posterior distribution (see GITHUB code)
  • We can plot each subjects random slope posterior distribution (see GITHUB code)
  • Finally, we can get predicted values from the model for the pre- and post-test FFM for each subject, based on their group.
    • We will use `posterior_epred()` to do this, which provides us with uncertainty around the average value, helping us to visualize the uncertainty around our model estimates. This is different than if we were to use `posterior_preidct()`, which also incorporates observation-level noise and is good simulating raw data on a *hypothetical* new person who we are trying to make an inference on.

Repeating the simulation

  • Okay, that is all interesting. We can see how the simulation returned the results of the ground truth coefficients that we set up. We can also see how the Bayesian mixed model was useful in helping us say more than just *”This group was better than that group”* by allowing us to extract posterior draws, visualize our uncertainty and, if we wanted, make more substantive statements about the individual subjects in our study.
  • The next thing we need to consider is the notion that this was one study with 17 subjects. If we pulled 17 new subjects, perhaps the results would look different? So, let’s set up a `for()` loop to play out 1000 simulated studies of the same data. We will keep the ground truth coefficients for the fixed effects the same as above — but of course you can allow them to vary if you’d like. In the loop, with each study, we will allow our subject intercepts and our model error to vary since we wouldn’t expect those to be the same if we selected a new group of 17 subjects. For each simulation we will simply extract the group*time interaction coefficient, since that is what we care about from the study.

NOTE: For purposes of time, we will use {lme4} to calculate the mixed models so that we don’t have to run 1000 Markov Chain Monte Carlo Simulations (as we would if we did a Bayesian mixed model like above).

  • In simulating this 1000 times we see that the median value is around where we specified it, 0.6 (the ground truth). However, we see considerable variability in this effect over the 1000 simulations. This could be because of the amount of variability that I have in the subject intercepts and the model error. Someone who does this type of research all of the time might have a better/more plausible prior assumption of what type of variability we would expect for these parameters. The point, however, is that one study is simply a snapshot of a small population that we hope to learn from and infer something meaningful to the broader population. Had the study been done again with a different group of 17 female physique competitors we might have gotten totally different results.
  • Overall, the effect does still favor the high-protein group with 68% of the distribution below 0, indicating a positive effect on fat free mass for high-protein relative to low protein.

Validity, Reliability, & Responsiveness — A few papers on measurement in sport science

I had the pleasure of speaking at the National Strength and Conditioning Association‘s (NSCA) National Conference this summer and while there I made it a point to attend the Sport Science & Performance Technology Special Interest Group meeting as well.

One thing that immediately stood out to me was the number of questions raised specific to what types of technologies to purchase (e.g. “Which brand of force plates should we buy?”, “Does anyone have a list comparing and contrasting different technologies so that we can determine what would be best for us?”, etc.).

While these are fine questions, I do feel they are a bit like putting the cart before the horse. Before thinking about what technology to purchase, we should spend a good bit of time gaining clarity on the question(s) we are attempting to answer. Once we have a firm understanding of the question we can then begin the process of evaluating whether a technology exists that can help us collect the necessary data to explore that question. In fact, this was the main crux of my lecture at the conference, as I spoke about using the PPDAC Framework in practice (I wrote an article about this framework a couple of years ago).

A force plate, a GPS unit, or an accelerometer won’t solve all of our problems. In fact, depending on our question, they might not solve any of our problems! Moreover, as sport scientists we need to concern ourselves not only with the research question but, also whether the desired technology is useful within our ecological setting. Just because something worked in a controlled lab environment or was valid in a different sport does not mean it will be useful for our sport, or in our setting, or with our athletes, or given our unique constraints.

So, I decided to share a few resources pertaining to measurement theory concepts such as validity, reliability, and responsiveness/sensitivity for those working in the sport science space who are interested in more critical approaches to evaluating the technology we use in practice.

Additionally, for those interested, several years ago I wrote a full R code blog for the last paper above (Swinton et al) ,which can he accessed HERE.

Happy reading!

The High Performance Hockey Podcast Interview

This week, I had the great pleasure of being interviewed by my good friend and colleague Anthony Donskov for his High Performance Hockey Podcast.

Anthony has done a tremendous job for the sports science and strength and conditioning community in his teaching, writing, and podcasting. He brings a wealth of knowledge from both the applied strength coach realm all the way through to his PhD work.

In this podcast interview, Anthony and I discuss:

  • Data analysis
  • The PPDAC Framework for conducting research
  • My criticisms of applied sport science
  • The challenge of measuring hard things and things that matter in applied sport.

Check out the podcast HERE.

R Tips & Tricks: Normalizing test dates & calculating test differences

A friend of mine was downloading some force plate data from the software provider so that he could evaluate test data in a few of his athletes during return to play. The issue he was running into was that the different athletes all had different numbers of tests and different start and end testing times. The software exports the test outputs by date and he was wondering how he could normalize the dates to numeric values (e.g. Test 1, Test 2, etc.) so that he could model the date (since we can’t really use a Date in a regression model).

I’ll be the first to admit that working with dates and times can be an incredible pain in the butt. For reference, I covered the topic of converting Catapult GPS practice duration strings to actual training minutes, HERE. To help him out, I provided a few different solutions depending on the research question. I also add some code for calculating changes in test performance between tests and from each test to baseline.

The full code is available on my GITHUB page.

Loading Packages & Simulating Data

## load packages ----------------------------------------------
library(tidyverse)
library(lubridate)

## Simulate data ----------------------------------------------
set.seed(78)
dat <- tibble(
  
  athlete = rep(c("Tom", "Bob", "Franklin"), times = c(5, 10, 3)),
  test_dates = c(
    seq(as.Date("2023-01-01"), as.Date("2023-01-5"), by = "days"),
    seq(as.Date("2023-02-15"), as.Date("2023-02-24"), by = "days"),
    as.Date(c("2023-01-19", "2023-01-30", "2023-02-26"))
  ),
  jump_height = round(rnorm(n = 18, mean = 28, sd = 2.5), 1)
  
)

dat


We can see that Tom has 5 tests, Bob has 10, and Franklin has only 3. Additionally, Tom and Bob tested every day, consecutively, while Franklin was less compliant and has larger time frames between his tests.

Create a test number

First, let’s normalize the Dates so that they are numeric. Basically, instead of dates we want a value indicating whether the test was test 1, or test 5, or test N. We will do this by creating a row_number() id/counter for each individual athlete.

### Create a test number ------------------------------------------
dat <- dat %>%
  group_by(athlete) %>%
  mutate(test_day = row_number())

dat

Calculating the time between tests

Alternatively, we may not just want to know the test number of each test but we may want to determine the amount of days between each test.

The code to do this is a bit ugly looking so let’s unpack it.

  1. Since we are dealing with dates we use the difftime() function which takes an argument for the two times you are looking to calculate the difference between. Here, we are trying to calculate the difference in time (days) between one date and the date preceding it for each individual athlete.
  2. The difftime() function will produce a to time variable. If we want to make this numeric we need to convert it to a character so we do that with the as.character() function.
  3. Once the variable is a character we use the as.numeric() function to convert it to a numeric value.
  4. Finally, since the first value for each athlete will be an NA, since there is no date preceding the first test, we use the coalesce() function to fill in a 0 value for each of the NA’s, to indicate that this was the first test and thus there was no time between it and any other test.
### Calculate the time between tests -------------------------------
dat <- dat %>%
  group_by(athlete) %>%
  mutate(time_btw_tests = coalesce(as.numeric(as.character(difftime(test_dates, lag(test_dates)))), 0))

dat

Notice that Tom and Bob have 1 day between all of their tests while Franklin’s second test was 11 days after his first and his third test was 27 days after his second.

Calculate the difference in jump height from one test to the next

### Calculate difference in jump height from one day to the next -------------------
dat <- dat %>%
  group_by(athlete) %>%
  mutate(test_to_test_diff = jump_height - lag(jump_height))

dat

Here, we use the lag() function to calculate the difference in one value from the value before it within in the same column. Since we grouped by athlete, which is what we want, their first test will always have an NA, in this new column, since there was no test preceding it.

Calculating the difference in jump height from the baseline test

Finally, we might also be interested to evaluate the performance on each test relative to the athlete’s baseline test. To do this we simply subtract jump_height from the jump_height indexed in row one for each athlete.

### Calculate difference in jump height from each test to the baseline test -------------

dat <- dat %>%
  group_by(athlete) %>%
  mutate(test_to_baseline_diff = jump_height - jump_height[1])

dat

Wrapping Up

Dates and times are always tricky to deal with. Most of the sports technology providers will proved data as dates (or unix timestamps) meaning that we have to do some cleaning of the data to codify the dates as numeric values representing the test number or the days between tests (depending on the research question). Additionally, using lag functions can be helpful for calculating he difference from one test to the next or from each test to the baseline.

The entire code is available on my GITHUB page.

If you have any data cleaning issues that you are dealing with from various sports science technologies, feel free to reach out!

Can I please be introduced to the Non-Applied Sport Scientist?

A recent discussion on Twitter spurred some thoughts that I had with respect to titles and roles in sport and in particular the title/role of Applied Sport Scientist.

@ScientistSport posed the following question:

It’s an interesting question to ponder. Given that sport science was originally born out of physiologists attempting to study human performance in Olympic sport athletes (which then eventually bled into team sport athletes) the question makes sense. Moreover, it seems like people generally think of sport science as something directed at helping the team “train better” – monitoring training loads, testing strength, power and conditioning, and even entering into the discussion of return to play following injury. Such a role has led many teams to employ an Applied Sport Scientist.

Titles in sport are weird. What does an Applied Sport Scientist do? What is the description of the role? More importantly, is there a Non-Applied Sport Scientist? If so, what are they doing?

Generally, when I’ve been introduced to the Applied Sport Scientist at a team when I’ve found is they are an assistant strength coach or assistant athletic trainer that has been tasked with turning on GPS units, conducting force plate jumps with the players, and coordinating the reports from the team’s Athlete Management System (AMS).

No doubt these are important tasks and critical to helping the staff plan and manage the team’s training! But, why is this a science role? What’s scientific about it? Is the individual ensuring data quality and integrity is being maintained before it is stored in the AMS? Is the individual conducting scientific inquiry of the data within the AMS to understand the measurements being made and determining if the measures are valid, reliable, or responsive? More importantly, how is the individual using the abundance of data being collected to answer larger questions that are relevant to the entire organization?

Perhaps the role shouldn’t be called Applied Sport Scientist? Maybe it should be Data Collection Coordinator or something more descriptive of the task at hand? Titles matter! They define what we do and how we do it. Again, if there is an Applied Sport Scientist is there a Non-Applied Sport Scientist? Maybe the latter is the one doing the real scientific work – identifying the pertinent research questions, planning applied science studies, structuring and establishing best practice data collection methods, analyzing data, and communicating the results to the end users.

What should the role of an Applied Sport Scientist be?

While some may feel like my argument is a bit pedantic here is why it matters.

The aim of the Applied Sport Scientist or the Sport Science Department should be to answer questions across the entire sporting organization. This shouldn’t simply be limited to matters of strength and conditioning. Rather, the goal should be to apply the scientific method to any and all questions in sport – training, return to play, performance evaluation, player acquisition, team tactics, etc. – and work at the intersection of such topics to provide analysis that helps the key stakeholders make decisions. A few colleagues and I wrote a paper about the parallels between Business Intelligence and Sports Science a few years ago <CLICK HERE>.

Science isn’t just a title; it is a framework and process for asking and answering questions. Or, as David Salsburg states, in his brilliant book The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century, “Science, we are often taught, is measurement. We make careful measurements and use them to find mathematical formulas that describe nature.” Consequently, someone that is given the title Applied Sport Scientist should actually have scientific training. The concept of framing a question, collecting data, doing basic statistics, knowing basic physiology and biomechanics, understanding how to run a simple reliability study, etc., are things that should be fundamental skills for this individual. Calling someone a Sport Scientist who doesn’t have these skills – even though they might be a really smart person and they might know a good deal about whatever technology they are using – is like calling me a strength coach. Sure, I can write a program and I can train and coach people. But, that’s probably not why you would hire me. Just as the strength coach can collect data and print reports, but you aren’t hiring them to conduct scientific investigations. You’re also not hiring the Physical Therapist to run the nutrition program.

Being smart and hardworking are important qualities in sport and everyone can help out in various areas of the organization. But titles should matter because they in some way define roles and responsibilities. The best organizations find the right people, with the right skill sets, to work together and create a super team.

As I like to say, Success boils down to four things:

  1. Knowing what you know.
  2. Working to be really good at what you know.
  3. Knowing what you don’t know.
  4. Knowing enough about what you don’t know to ask the right questions to get people in who can help you out with that thing.