{"id":2338,"date":"2022-04-15T05:03:14","date_gmt":"2022-04-15T05:03:14","guid":{"rendered":"http:\/\/optimumsportsperformance.com\/blog\/?p=2338"},"modified":"2022-11-08T03:37:28","modified_gmt":"2022-11-08T03:37:28","slug":"deriving-a-confidence-interval-from-an-estimate-and-a-p-value","status":"publish","type":"post","link":"https:\/\/optimumsportsperformance.com\/blog\/deriving-a-confidence-interval-from-an-estimate-and-a-p-value\/","title":{"rendered":"Deriving a Confidence Interval from an Estimate and a p-value"},"content":{"rendered":"

Although most journals require authors to include confidence intervals into their papers it isn’t mandatory for all journal (merely a recommendation). Additionally, there may be occasions when you are reading an older paper from a time when this mandate\/recommendation was not enforced. Finally, sometimes abstracts (due to word count limits) might only present p-values and estimates, in which case you might want to quickly obtain the confidence intervals to help organize your thoughts prior to diving into the paper. In these instances, you might be curious as to how you can get a confidence interval around the observed effect when all you have is a p-value.<\/p>\n

Bland & Altman wrote a short piece of deriving confidence interval from only an estimate and a p-value in a 2011 paper in the British Medical Journal:<\/p>\n

Altman, DG. Bland, JM. (2011). How to obtain the confidence interval from a p-value. BMJ<\/em>, 343: 1-2.<\/a><\/strong><\/span><\/p>\n

Before going through the approach, it is important to note that they indicate a limitation of this approach is that it wont be as accurate in smaller samples, but the method can work well in larger studies (~60 subjects or more).<\/p>\n

The Steps<\/strong><\/span><\/p>\n

The authors’ list 3 easy steps to derive the confidence interval from an estimate and p-value:<\/p>\n

    \n
  1. Calculate the test statistic for a normal distribution from the p-value.<\/li>\n
  2. Calculate the standard error (ignogre the minus sign).<\/li>\n
  3. Calculate the 95% CI using the standard error and a z-critical value for the desired level of confidence.<\/li>\n
  4. When doing this approach for a ratio (e.g., Risk Ratio, Odds Ratio, Hazard Ratio), the formulas should be used with the estimate on the log scale (if it already isn’t) and then exponentiate (antilog) the confidence intervals to put the results back to the normal scale.<\/li>\n<\/ol>\n

    Calculating the test statistic<\/strong><\/em><\/p>\n

    To calculate the test statistic use the following formula:<\/p>\n

    z = -0.862 + sqrt(0.743 – 2.404 * log(p.value))<\/em><\/p>\n

    Calculating the standard error<\/strong><\/em><\/p>\n

    To calculate the standard error use the following formula (remember that we are ignoring the sign of the estimate):<\/p>\n

    se = abs(estimate) \/ z<\/em><\/p>\n

    If we are dealing with a ratio, make sure that you are working on the log scale:<\/p>\n

    se = abs(log(estimate)) \/ z<\/em><\/p>\n

    Calculating the 95% Confidence Limits<\/strong><\/em><\/p>\n

    Once you have the standard error, the 95% Confidence Limits can be calculated by multiplying the standard error by the z-critical value of 1.96:<\/p>\n

    CL.95 = 1.96 * se<\/em><\/p>\n

    From there, the 95% Confidence Interval can be calculated:<\/p>\n

    low95 = Estimate – CL.95<\/em>
    \nhigh95 = Estimate + CL.95<\/em><\/p>\n

    Remember, if you are working with rate statistics and you want to get the confidence interval on the natural scale, you will need to take the antilog:<\/p>\n

    low95 = exp(Estimate – CL.95)<\/em>
    \nhigh95 = exp(Estimate + CL.95)<\/em><\/p>\n

     <\/p>\n

    Writing a function<\/strong><\/span><\/p>\n

    To make this simple, I’ll write everything into a function. The function will take three arguments, which you will need to obtain from the paper:<\/p>\n

      \n
    1. p-value<\/li>\n
    2. The estimate (e.g., difference in means, risk ratio, odds ratio, hazard ratio, etc)<\/li>\n<\/ol>\n

      The function will default to log = FALSE<\/strong> but if you are working with a rate statistic you can change the argument to log = TRUE<\/strong> to get the results on both the log and natural scales. The function also takes a sig_digits<\/strong> argument, which defaults to 3 but can be changed depending on how many significant digits you need.<\/p>\n

      \r\nestimate_ci_95 <- function(p_value, estimate, log = FALSE, sig_digits = 3){\r\n  \r\n  if(log == FALSE){\r\n    \r\n    z <- -0.862 + sqrt(0.743 - 2.404 * log(p_value))\r\n    z\r\n\r\n    se <- abs(estimate) \/ z\r\n    se\r\n    \r\n    cl <- 1.96 * se\r\n    \r\n    low95 <- estimate - cl\r\n    high95 <- estimate + cl\r\n    \r\n    list('Standard Error' = round(se, sig_digits),\r\n         '95% CL' = round(cl, sig_digits),\r\n         '95% CI' = paste(round(low95, sig_digits), round(high95, sig_digits), sep = ", "))\r\n    \r\n  } else {\r\n    \r\n    if(log == TRUE){\r\n      \r\n      z <- -0.862 + sqrt(0.743 - 2.404 * log(p_value))\r\n      z\r\n      \r\n      se <- abs(estimate) \/ z\r\n      se\r\n      \r\n      cl <- 1.96 * se\r\n      \r\n      low95_log_scale <- estimate - cl\r\n      high95_log_scale <- estimate + cl\r\n      \r\n      low95_natural_scale <- exp(estimate - cl)\r\n      high95_natural_scale <- exp(estimate + cl)\r\n      \r\n      list('Standard Error (log scale)' = round(se, sig_digits),\r\n           '95% CL (log scale)' = round(cl, sig_digits),\r\n           '95% CL (natural scale)' = round(exp(cl), sig_digits),\r\n           '95% CI (log scale)' = paste(round(low95_log_scale, sig_digits), round(high95_log_scale, sig_digits), sep = ", "),\r\n           '95% CI (natural scale)' = paste(round(low95_natural_scale, sig_digits), round(high95_natural_scale, sig_digits), sep = ", "))\r\n      \r\n    }\r\n    \r\n  }\r\n  \r\n}\r\n<\/pre>\n
      \u00a0Test the function out<\/strong><\/span><\/p>\n
      The paper provides two examples, one for a difference in means and the other for risk ratios.<\/p>\n
      Example 1<\/strong><\/em><\/p>\n
      Example 1 states:<\/p>\n
      “the abstract of a report of a randomised trial included the statement that “more patients in the zinc group than in the control group recovered by two days (49% v 32%,P=0.032).” The difference in proportions was Est = 17 percentage points, but what is the 95% confidence interval (CI)?<\/p><\/blockquote>\n
      \r\nestimate_ci_95(p_value = 0.032, estimate = 17, log = FALSE, sig_digits = 1)\r\n<\/pre>\n
       <\/p>\n
      \"\"<\/a><\/p>\n
       <\/p>\n
      Example 2
      \n<\/strong><\/em><\/p>\n
      Example 2 states:<\/p>\n
      “the abstract of a report of a cohort study includes the statement that \u201cIn those with a [diastolic blood pressure] reading of 95-99 mm Hg the relative risk was 0.30 (P=0.034).\u201d What is the confidence interval around 0.30?”<\/p><\/blockquote>\n
      Here we change the argument to log = TRUE<\/strong> since this is a ratio statistic needs to be on the log scale.<\/p>\n
      \r\nestimate_ci_95(p_value = 0.034, estimate = log(0.3), log = TRUE, sig_digits = 2)\r\n<\/pre>\n
      \"\"<\/a><\/p>\n
      Try the approach out on a different data set to confirm the confidence intervals are calculated properly<\/strong><\/span><\/p>\n
      Below, we build a simple logistic regression model for the PimaIndiansDiabetes<\/strong> data set from the {mlbench} package.<\/p>\n