Isenmann (2019) - Ecdysteroids as anabolic agents

Study Overview

Reference: Isenmann (2019) - Ecdysteroids as non-conventional anablic agent - Performance enhancement by ecdysterone supplementation in humans - Archives of Toxicology.

• Aim: To evaluate the effect of long-term administration of an ecdysterone-containing dietary supplement with a special focus on the increase in performance during resistance training in humans.
• Methods
  • 46 healthy male subjects (6 drop outs) with 1-year barbell training
  • 4 Supplement dose groups:
    • Ec1 — 2 capsules of Ecdysterone plus training
    • Ec2 — 8 capsules of Ecdysterone plus training
    • PL - 2 placebo capsules plus training
    • CO — Control group took two capsules of Ecdysterone but did not train
    • NOTE: Capsules were reported to contain 100 mg of ecdysterone
  • Training Program
    • PL, Ec1, and Ec2 groups trained in a 10-week program
    • 3 sessions/week
      • Each session had 6 barbell exercises
      • 3 sets x 12 reps
      • Intensity began at 70% of 1RM and then increased 2.5-5kg when possible
  • Outcome measures (Pre/Post)
    • Body composition (bio-electrical impedance)
    • CMJ
    • 1-RM Back Squat
    • 1-RM Bench Press
    • Serum and urine samples
    • Liver & Kidney Toxicity
  • Statistics
    • 2x4 (time*group) ANCOVA with Bonferroni adjustment
    • A paired t-test was also used to evaluate the individual time and group effects
• Results
  • Anthropometric Measures
    • Ec1 and Ec2 increased body weight significantly
    • Significant change in bodyweight between Ec2 and CO
    • A time effect for muscle mass in the Ec1 and Ec2 groups
    • Difference between PL and Ec2
  • Power & Strength
    • All three groups increased their jump height
    • No group*time effect difference between groups for CMJ
    • All three training groups increased 1-RM squat. However, no significant difference between the groups and no time*group effect
    • All three training groups increased their performance
  • Supplement analysis
    • Extraction of the amount of supplement in the 100mg capsules revealed that each capsule only had 6mg of ecdysterone

My Thoughts

• This is an intervention study. The authors set up 4 different supplemental groups and three of the groups trained while the fourth group did not. The paper does suffer from some severe limitations.
• Small sample of subjects with a low training age (1-year of barbell training)
  • Four groups with 10-12 subjects per group is very under-powered for the large number of comparisons that they did (body comp, performance, serum hormone levels, kidney biomarkers)
• They mention there were 46 subjects but that 6 that were dropped out of the analysis due to missed sessions. However, when they provide the group numbers they provide a total count of 46 subjects. We don’t know which group had the 6 drop outs.
• No dietary control
• Bioelectrical impedance is horribly noisy
• Heavy focus on within group change versus between group change
  • This is a massive limitation in sport and exercise science research. The estimand that we care about isn’t the within group change, it’s the between group change. Intervention studies are asking the question, “Is the change in this group, doing this thing, larger that the other group, doing that other thing.”
• The visualizations of the dynamite plunger plots is useless for understanding the spread of the distribution. We don’t know what the error bar represents — is that standard error? Is it standard deviation? Is it a confidence interval? None of that is clear.
• We don’t have a complete factorial design
  • We have three training groups and one non-training supplement group. What we don’t have is a non-training placebo group or a non-training high-dose group. Thus, it makes it hard to tease out the effects of supplementation, training, and their interaction
  • A better 2x2 factorial design would be:
    • Placebo without training
    • Placebo with training
    • Ecdysterone without training
    • Ecdysterone with training
    • With this we could estimate the effect of training, the effect of supplementation, and the interaction of training and supplementation
• They don’t discuss a clear randomization procedure (randomized within matched blocks, was allocation concealed, what were the baseline characteristics across groups) outside of saying that the groups were matched for performance and body comp
• They don’t explain anything about what covariates are in their ANCOVA, which is strange because there are no covariates in their data (I guess they could use pre-test values). This is probably an ANOVA with repeated measures and they stated it incorrectly. They could have instead used a mixed model. For example:

\(Y_ij = B0 + B1*Time_j + B2*Group_i + B3*(Time_j * Group_i) + u_i + e_ij\)

    * Where:
        * i = individual subject
        * j = the measurement occasion (pre or post)
        

• The supplement itself is problematic given that it is reported to have 100mg and really only had 6mg/capsule? This changes the dosage of the study.

Developing our analysis

• The author’s provided a supplement file for the paper, which had group means and standard deviations for the various things tested. So, we can attempt to simulate the data to try and better understand the observed effects.

• As noted above, this paper suffers from a number of limitations. The change in 1RM bench press strength was some of the most dramatic changes in the study. The author’s provided the visual for those results in Figure 4. Oddly, the author’s did not include the control group in their planned comparison. They simply focused on the three training groups – which is odd.

• There are a two ways we could try and design our simulation:

  1. The table in the supplement gives us pre and post test values for each group along with their standard deviations. We can conduct the t-tests used in this paper from these summary statistics.

  2. We could simulate the groups themselves using the differences in the data and perform the t-tests.

  3. We could attempt to simulate each individual’s pre- and post-test and construct a mixed model to build a more interesting analysis that what was presented in the paper.

Why not try each?!

T-tests using summary statistics

• Get the summary statistics from the supplemental data table

## sample sizes
placebo_n <- 12
ec1_n <- 12
ec2_n <- 10

## Pre-test values
placebo_pre_mean <- 92.71
placebo_pre_sd <- 13.46
ec1_pre_mean <- 82.92
ec1_pre_sd <- 15.73
ec2_pre_mean <- 88.75
ec2_pre_sd <- 13.03

## Post-test values
placebo_post_mean <- 96.04
placebo_post_sd <- 12.15
ec1_post_mean <- 92.50
ec1_post_sd <- 13.73
ec2_post_mean <- 97.25
ec2_post_sd <- 10.30

pre_post_data <- data.frame(
  group = c("placebo", "ec1", "ec2"),
  n = c(placebo_n, ec1_n, ec2_n),
  pre_mean = c(placebo_pre_mean, ec1_pre_mean, ec2_pre_mean),
  pre_sd = c(placebo_pre_sd, ec1_pre_sd, ec2_pre_sd),
  post_mean = c(placebo_post_mean, ec1_post_mean, ec2_post_mean),
  post_sd = c(placebo_post_sd, ec1_post_sd, ec2_post_sd)
)

knitr::kable(pre_post_data)

group	n	pre_mean	pre_sd	post_mean	post_sd
placebo	12	92.71	13.46	96.04	12.15
ec1	12	82.92	15.73	92.50	13.73
ec2	10	88.75	13.03	97.25	10.30

• First calculate the mean difference from pre to post for each group

pre_post_data <- pre_post_data %>%
  mutate(diff_mean = post_mean - pre_mean)

pre_post_data

##     group  n pre_mean pre_sd post_mean post_sd diff_mean
## 1 placebo 12    92.71  13.46     96.04   12.15      3.33
## 2     ec1 12    82.92  15.73     92.50   13.73      9.58
## 3     ec2 10    88.75  13.03     97.25   10.30      8.50

• Next we get the standard deviation of the differences from the paper itself.

## Standard Deviation of Differences
pre_post_data <- pre_post_data %>%
  mutate(diff_sd = c(3.74, 2.79, 4.44))

pre_post_data

##     group  n pre_mean pre_sd post_mean post_sd diff_mean diff_sd
## 1 placebo 12    92.71  13.46     96.04   12.15      3.33    3.74
## 2     ec1 12    82.92  15.73     92.50   13.73      9.58    2.79
## 3     ec2 10    88.75  13.03     97.25   10.30      8.50    4.44

• Recreate Figure 4 from the paper

pre_post_data %>%
  mutate(group = factor(group, levels = c("placebo", "ec1", "ec2"))) %>%
  ggplot(aes(x = group, y = diff_mean)) +
  geom_col(color = "black",
           fill = "black") +
  geom_errorbar(aes(ymin = diff_mean, ymax = diff_mean + diff_sd),
                width = 0.3,
                linewidth = 0.5) 

Calculate t-tests

• Now that we have summary statistics for each group we can compute our t-tests

• We have three between group comparisons to make:

  1. Placebo to Ec1
  2. Placebo to Ec2
  3. Ec1 to Ec2

• We will write two t-test functions to help us do this:

  1. A paired t-test function that provides us with the within-group difference (which they talked extensively about in the paper, but isn’t really the estimand we care about.
  2. A between group t-tests to evaluate whether the differences between groups were significant and we need to make the Bonferroni correlation for the p-value for multiple comparisons

## Paired t-test function
paired_t_test <- function(diff_mean, diff_sd, n_subjects, alpha = 0.05){
  
  se <- diff_sd / sqrt(n_subjects)
  t <- diff_mean / se
  deg_freedom <- n_subjects - 1
  
  # Confidence interval
  tail_value <- 1 - alpha
  crit_val <- abs(qt(alpha / 2, df = deg_freedom))
  moe <- crit_val * se
  lower_ci <- diff_mean - moe
  upper_ci <- diff_mean + moe
  
  p <- 2 * pt(-abs(t), df = deg_freedom)
  
  outcome <- if(p <= alpha){"Results are statistically significant"} else {
    "Results are not statistically significant"
  }
  
  data.frame(
    t_statistic = t,
    lower_ci = lower_ci,
    upper_ci = upper_ci,
    p_value = p,
    outcome = outcome
  )
}


## Between Group t-test Function
between_group_t_test <- function(n1, mean1, sd1,
                   n2, mean2, sd2,
                   n_comparisons,
                   alpha = 0.05) {
  
  ## Summary statistics
  mean_diff <- mean2 - mean1
  se_diff <- sqrt(sd1^2/n1 + sd2^2/n2)
  df <- (sd1^2/n1 + sd2^2/n2)^2 / ((sd1^2/n1)^2/(n1-1) + (sd2^2/n2)^2/(n2-1))

  ## t-test
  t <- mean_diff / se_diff
  p <- 2 * pt(-abs(t), df)
  
  ## Confidence intervals
  crit_val <- qt(1 - alpha/2, df)
  moe <- crit_val * se_diff
  lower_ci <- mean_diff - crit_val * se_diff
  upper_ci <- mean_diff + crit_val * se_diff
  
  ## Bonferroni adjustment for multiple comparisons
  sig <- alpha / n_comparisons
  
  ## Is the difference statistically significant?
  outcome <- if(p <= sig){ "Statistical signifiance achieved" } else {"Difference is not statistically significant"}

  data.frame(
    mean_diff = mean_diff,
    t_statistic = t,
    lower_ci = lower_ci,
    upper_ci = upper_ci,
    p_value = p,
    #p_adjusted = pmin(p * n_comparisons, 1)
    p_adjusted = outcome
  )
}

Paired t-tests

## get data from tables
pre_post_data

##     group  n pre_mean pre_sd post_mean post_sd diff_mean diff_sd
## 1 placebo 12    92.71  13.46     96.04   12.15      3.33    3.74
## 2     ec1 12    82.92  15.73     92.50   13.73      9.58    2.79
## 3     ec2 10    88.75  13.03     97.25   10.30      8.50    4.44

## Placebo
paired_t_test(diff_mean = 3.33, diff_sd = 3.74, n_subjects = 12, alpha = .05)

##   t_statistic  lower_ci upper_ci    p_value
## 1    3.084347 0.9537174 5.706283 0.01039037
##                                 outcome
## 1 Results are statistically significant

## EC1
paired_t_test(diff_mean = 9.58, diff_sd = 2.79, n_subjects = 12, alpha = .05)

##   t_statistic lower_ci upper_ci      p_value
## 1    11.89466 7.807319 11.35268 1.273573e-07
##                                 outcome
## 1 Results are statistically significant

## EC2
paired_t_test(diff_mean = 8.50, diff_sd = 4.44, n_subjects = 12, alpha = .05)

##   t_statistic lower_ci upper_ci     p_value
## 1    6.631726 5.678959 11.32104 3.70139e-05
##                                 outcome
## 1 Results are statistically significant

Between Group Comparisons

## get data from tables
pre_post_data

##     group  n pre_mean pre_sd post_mean post_sd diff_mean diff_sd
## 1 placebo 12    92.71  13.46     96.04   12.15      3.33    3.74
## 2     ec1 12    82.92  15.73     92.50   13.73      9.58    2.79
## 3     ec2 10    88.75  13.03     97.25   10.30      8.50    4.44

## Placebo to EC1
between_group_t_test(n1 = 12, 
       mean1 = 3.33, 
       sd1 = 3.74, 
       
       n2 = 12, 
       mean2 = 9.58, 
       sd2 = 2.79, 
       
       n_comparisons = 3, 
       alpha = 0.05
  )

##   mean_diff t_statistic lower_ci upper_ci      p_value
## 1      6.25    4.640069 3.443363 9.056637 0.0001516146
##                         p_adjusted
## 1 Statistical signifiance achieved

## Placebo to EC2
between_group_t_test(n1 = 12, 
       mean1 = 3.33, 
       sd1 = 3.74, 
       
       n2 = 10, 
       mean2 = 8.50, 
       sd2 = 4.44, 
       
       n_comparisons = 3, 
       alpha = 0.05
  )

##   mean_diff t_statistic lower_ci upper_ci     p_value
## 1      5.17    2.918998 1.444731 8.895269 0.009269929
##                         p_adjusted
## 1 Statistical signifiance achieved

## EC1 to EC2
between_group_t_test(n1 = 12, 
       mean1 = 9.58, 
       sd1 = 2.79, 
       
       n2 = 10, 
       mean2 = 8.50, 
       sd2 = 4.44, 
       
       n_comparisons = 3, 
       alpha = 0.05
  )

##   mean_diff t_statistic  lower_ci upper_ci   p_value
## 1     -1.08  -0.6672218 -4.538249 2.378249 0.5150359
##                                    p_adjusted
## 1 Difference is not statistically significant

Simulating individual participant changes

set.seed(2026)
change_dat <- data.frame(
  group = c(rep(c("placebo", "ec1"), each = 12), rep("ec2", times = 10)),
  change = c(rnorm(n = 12, mean = 3.33, sd = 3.74),
             rnorm(n = 12, mean = 9.58, sd = 2.79),
             rnorm(n = 10, mean = 8.5, sd = 4.44))
)

head(change_dat)

##     group     change
## 1 placebo  5.2770031
## 2 placebo -0.7080435
## 3 placebo  3.8507506
## 4 placebo  3.0130395
## 5 placebo  0.8367678
## 6 placebo -6.0801730

• With this simulated data we can make much more useful plots of the data than what the author’s provided in their paper (the dynamite plunger plots don’t reveal the data enough)

change_dat %>%
  ggplot(aes(x = change, fill = group)) +
  geom_density(alpha = 0.5) +
  geom_vline(xintercept = 0,
             color = "black",
             linetype = "dashed",
             linewidth = 1.2) +
  xlim(-15, 25)

• This provides a look at all of the data, rather than just summary statistics (like the bar plots in the previous section). We can see that with such small samples in each group the data looks far from normal (that doesn’t mean the assumption that this data would be normal in the population isn’t valid, however). We can also see that the two Ecdysteroid groups had greater improvement than the Placebo group. We can also see the overlap between the groups.
• Since we have a change score for each participant we can conduct an anove on this data.

fit_aov <- aov(change ~ group, data = change_dat)
summary(fit_aov)

##             Df Sum Sq Mean Sq F value   Pr(>F)    
## group        2  637.1   318.6   28.73 8.76e-08 ***
## Residuals   31  343.8    11.1                     
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

• The omnibus is statistically significant so now we proceed with our t-tests with Bonferroni correction to evaluate which groups had the significant differences.

pairwise.t.test(x = change_dat$change, g = change_dat$group, p.adjust.method = "bonferroni")

## 
##  Pairwise comparisons using t tests with pooled SD 
## 
## data:  change_dat$change and change_dat$group 
## 
##         ec1     ec2    
## ec2     0.63    -      
## placebo 4.1e-06 2.5e-07
## 
## P value adjustment method: bonferroni

• We obtain findings similar to our t-tests above

Simulating each individual’s pre- and post-test

• We will make this a simple simulation by generating data using the group level characteristics provided in the paper.
• This simulation does not have a correlation between pre- and post-test because we are generating the data as random draws from the group level parameters in the paper. It is an assumption we will make in order to keep this first simulation simple.
• Also note that these simulations won’t generate exact findings from the paper because we are generating random observations.

set.seed(46729)
subject_id <- as.factor(1:34)
group <- c(rep(c("placebo", "ec1"), each = 12), rep("ec2", times = 10))
pre_test <- c(rnorm(n = 12, mean = 92.71, sd = 13.46),
              rnorm(n = 12, mean = 82.82, sd = 15.73),
              rnorm(n = 10, mean = 88.75, sd = 13.03))
change <- c(rnorm(n = 12, mean = 3.33, sd = 3.74),
            rnorm(n = 12, mean = 9.58, sd = 2.79),
            rnorm(n = 10, mean = 8.50, sd = 4.44))
post_test <- pre_test + change


dat <- data.frame(
  subject_id = subject_id,
  group = group,
  pre_test = pre_test,
  post_test = post_test,
  change = change
) %>%
  mutate(group = factor(group, levels = c("placebo", "ec1", "ec2")))

• Calculate the summary stats in our data

dat %>%
  group_by(group) %>%
  summarize(across(.cols = pre_test:change,
                   list(mean = mean, sd = sd))) %>%
  knitr::kable()

group	pre_test_mean	pre_test_sd	post_test_mean	post_test_sd	change_mean	change_sd
placebo	91.74161	17.50019	94.68062	18.16336	2.939010	3.847647
ec1	75.46931	15.22229	85.40079	16.03926	9.931476	2.267836
ec2	90.64306	10.93764	98.01132	13.97776	7.368258	5.031488

• Create some plots of the data

## Distribution of each group
dat %>%
  ggplot(aes(x = change, fill = group)) +
  geom_density(alpha = 0.5) +
  geom_vline(xintercept = 0,
             color = "black",
             linetype = "dashed",
             linewidth = 1.2) +
  xlim(-15, 25)

## Individual participants
dat %>%
  dplyr::select(-change) %>%
  pivot_longer(cols = c(pre_test, post_test)) %>%
  mutate(name = factor(name, levels = c("pre_test", "post_test"))) %>%
  ggplot(aes(x = name, y = value, group = subject_id)) +
  geom_line(linewidth = 1.2) +
  geom_point(size = 3,
             shape = 21,
             color = "black",
             fill = "white") +
  facet_wrap(~group)

ANOVA

• Re-calculate the ANOVA from above

fit_aov <- aov(change ~ group, data = dat)
summary(fit_aov)

##             Df Sum Sq Mean Sq F value   Pr(>F)    
## group        2  299.5  149.76   10.38 0.000354 ***
## Residuals   31  447.3   14.43                     
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

pairwise.t.test(x = dat$change, g = dat$group, p.adjust.method = "bonferroni")

## 
##  Pairwise comparisons using t tests with pooled SD 
## 
## data:  dat$change and dat$group 
## 
##     placebo ec1    
## ec1 0.00026 -      
## ec2 0.03155 0.37551
## 
## P value adjustment method: bonferroni

ANCOVA

• We can use this data to calculate an ANCOVA where we adjust the post-test values for pre-test values

fit_ancova <- lm(post_test ~ pre_test + group, 
                 data = dat)

summary(fit_ancova)

## 
## Call:
## lm(formula = post_test ~ pre_test + group, data = dat)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -6.6207 -2.3988 -0.3101  2.0461  6.7944 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) -2.14721    4.27323  -0.502  0.61900    
## pre_test     1.05544    0.04505  23.431  < 2e-16 ***
## groupec1     7.89461    1.70371   4.634 6.54e-05 ***
## groupec2     4.49015    1.61380   2.782  0.00924 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 3.767 on 30 degrees of freedom
## Multiple R-squared:  0.9536, Adjusted R-squared:  0.949 
## F-statistic: 205.7 on 3 and 30 DF,  p-value: < 2.2e-16

emmeans(fit_ancova, pairwise ~ group)

## $emmeans
##  group   emmean   SE df lower.CL upper.CL
##  placebo   88.3 1.12 30     86.0     90.6
##  ec1       96.2 1.18 30     93.8     98.6
##  ec2       92.8 1.21 30     90.3     95.2
## 
## Confidence level used: 0.95 
## 
## $contrasts
##  contrast      estimate   SE df t.ratio p.value
##  placebo - ec1    -7.89 1.70 30  -4.634  0.0002
##  placebo - ec2    -4.49 1.61 30  -2.782  0.0244
##  ec1 - ec2         3.40 1.75 30   1.943  0.1442
## 
## P value adjustment: tukey method for comparing a family of 3 estimates

Mixed Model

• We can build a repeated measures mixed model.
• We first need to transform the data into a long format so that we can represent the repeated measures properly.
• We also will include the time*group interaction that the authors discussed in their paper

dat_long <- dat %>%
  pivot_longer(cols = c(pre_test, post_test),
               names_to = "time",
               values_to = "bench_press") %>%
  mutate(time = factor(time, levels = c("pre_test", "post_test")))

dat_long %>%
  head()

## # A tibble: 6 × 5
##   subject_id group   change time      bench_press
##   <fct>      <fct>    <dbl> <fct>           <dbl>
## 1 1          placebo 10.4   pre_test        104. 
## 2 1          placebo 10.4   post_test       114. 
## 3 2          placebo  0.405 pre_test        107. 
## 4 2          placebo  0.405 post_test       107. 
## 5 3          placebo -0.209 pre_test         57.7
## 6 3          placebo -0.209 post_test        57.5

fit_lmer <- lmer(bench_press ~ time * group + (1|subject_id),
                 data = dat_long)

display(fit_lmer)

## lmer(formula = bench_press ~ time * group + (1 | subject_id), 
##     data = dat_long)
##                        coef.est coef.se
## (Intercept)             91.74     4.52 
## timepost_test            2.94     1.10 
## groupec1               -16.27     6.39 
## groupec2                -1.10     6.71 
## timepost_test:groupec1   6.99     1.55 
## timepost_test:groupec2   4.43     1.63 
## 
## Error terms:
##  Groups     Name        Std.Dev.
##  subject_id (Intercept) 15.43   
##  Residual                2.69   
## ---
## number of obs: 68, groups: subject_id, 34
## AIC = 459.4, DIC = 484.7
## deviance = 464.0

• Make plots of the fixed effects

as.data.frame(fixef(fit_lmer)) %>%
  setNames("Coef") %>%
  rownames_to_column(var = "Variable") %>%
  filter(Variable != "(Intercept)") %>%
  left_join(
      as.data.frame(se.fixef(fit_lmer)) %>%
          setNames("SE") %>%
          rownames_to_column(var = "Variable") %>%
          filter(Variable != "(Intercept)")
  ) %>%
  ggplot(aes(x = Coef, y = Variable)) +
  geom_vline(xintercept = 0,
           color = "black",
           linetype = "dashed",
           linewidth = 1.2) +
  geom_errorbar(aes(xmin = Coef - 1.96*SE,
                    xmax = Coef + 1.96*SE),
                width = 0,
                linewidth = 1.2) +
  geom_point(shape = 21,
             size = 4,
             color = "black",
             fill = "white") +
  ggtitle("Mixed Model Fixed Effects Coefficients with 95% CI")

## Joining with `by = join_by(Variable)`

• Print Random Effects

as.data.frame(ranef(fit_lmer)) %>%
  rename(subject_id = grp) %>%
  ggplot(aes(x = condval, y = subject_id)) +
  geom_vline(xintercept = 0,
           color = "black",
           linetype = "dashed",
           linewidth = 1.2) +
  geom_errorbar(aes(xmin = condval - condsd,
                    xmax = condval + condsd),
                width = 0,
                linewidth = 1.2) +
  geom_point(shape = 21,
             size = 4,
             color = "black",
             fill = "white") +
  ggtitle("Mixed Model Subject Random Effects Coefficients with SD")

• Get each subject’s individual model (random intercept but fixed effects are for the population)

coef(fit_lmer)$subject_id %>%
  as.data.frame() %>%
  rownames_to_column(var = "subject_id") %>%
  head()

##   subject_id (Intercept) timepost_test groupec1  groupec2
## 1          1   107.05792       2.93901 -16.2723 -1.098548
## 2          2   105.54614       2.93901 -16.2723 -1.098548
## 3          3    56.65295       2.93901 -16.2723 -1.098548
## 4          4   110.84135       2.93901 -16.2723 -1.098548
## 5          5   105.00734       2.93901 -16.2723 -1.098548
## 6          6    97.58544       2.93901 -16.2723 -1.098548
##   timepost_test:groupec1 timepost_test:groupec2
## 1               6.992466               4.429247
## 2               6.992466               4.429247
## 3               6.992466               4.429247
## 4               6.992466               4.429247
## 5               6.992466               4.429247
## 6               6.992466               4.429247

Conclusions

• Overall, this wasn’t a good paper. It had significant methodological flaws and we can’t really say that Ecdysteroids do much of anything based on this data.

• We took several approaches to redoing the analysis from the paper and generating simulated data, offering us various potential options to conduct future analysis.

• Some things we might look to do to expand our approach here:

  1. Simulate new data with a desired correlation between the pre- and post-test values.
  2. Simulate other potential parameters (e.g., sex, nutritional intake, etc) to create a framework for new data collection and research methods.
  3. Seek out data from prior studies in the scientific literature and create a Smallest Effect Size of Interest and conduct TOST (Two One-Sided Test) analysis.
  4. Redo the analysis using Bayesian mixed models to obtain posterior distributions and make probabilistic statements about the data.