Outcomes and statistical power in adult critical care randomized trials.

Abstract

RATIONALE:

Intensive care unit (ICU)-based randomized clinical trials (RCTs) among adult critically ill patients commonly fail to detect treatment benefits.

OBJECTIVES:

Appraise the rates of success, outcomes used, statistical power, and design characteristics of published trials.

METHODS:

One hundred forty-six ICU-based RCTs of diagnostic, therapeutic, or process/systems interventions published from January 2007 to May 2013 in 16 high-impact general or critical care journals were studied.

MEASUREMENT AND MAIN RESULTS:

Of 146 RCTs, 54 (37%) were positive (i.e., the a priori hypothesis was found to be statistically significant). The most common primary outcomes were mortality (n = 40 trials), infection-related outcomes (n = 33), and ventilation-related outcomes (n = 30), with positive results found in 10, 58, and 43%, respectively. Statistical power was discussed in 135 RCTs (92%); 92 cited a rationale for their power parameters. Twenty trials failed to achieve at least 95% of their reported target sample size, including 11 that were stopped early due to insufficient accrual/logistical issues. Of 34 superiority RCTs comparing mortality between treatment arms, 13 (38%) accrued a sample size large enough to find an absolute mortality reduction of 10% or less. In 22 of these trials the observed control-arm mortality rate differed from the predicted rate by at least 7.5%.

CONCLUSIONS:

ICU-based RCTs are commonly negative and powered to identify what appear to be unrealistic treatment effects, particularly when using mortality as the primary outcome. Additional concerns include a lack of standardized methods for assessing common outcomes, unclear justifications for statistical power calculations, insufficient patient accrual, and incorrect predictions of baseline event rates.

• Authors: Michael O Harhay, Jason Wagner, Sarah J Ratcliffe, Rachel S Bronheim, Anand Gopal, Sydney Green, Elizabeth Cooney, Mark E Mikkelsen, Meeta Prasad Kerlin, Dylan S Small, Scott D Halpern
• Journal: American journal of respiratory and critical care medicine (Am J Respir Crit Care Med) Vol. 189 Issue 12 Pg. 1469-78 (Jun 15 2014) ISSN: 1535-4970 [Electronic] United States
• PMID: 24786714 (Publication Type: Journal Article, Research Support, N.I.H., Extramural, Research Support, N.I.H., Intramural)

Topics

• Adult
• Critical Care
• Data Interpretation, Statistical
• Humans
• Intensive Care Units
• Logistic Models
• Odds Ratio
• Outcome Assessment, Health Care (methods, statistics & numerical data)
• Poisson Distribution
• Randomized Controlled Trials as Topic (methods, statistics & numerical data)
• Research Design (statistics & numerical data)