Multiplicity Adjustments in Adaptive Design

Abstract

There are a number of available statistical methods for adaptive designs, among which the combination method of Bauer and Kohne's (1994) is well known and widely used. In this work, we revisit the the Bauer-Kohne method in three ways: overall FWER control for single-hypothesis in a two-stage adaptive design, overall FWER control for two-hypothesis in a two-stage adaptive design, and overall FDR control for multiple-hypothesis in a two-stage adaptive design. We first take the Bauer-Kohne method in a more direct manner to have more flexibility in the choice of the early rejection and acceptance boundaries as well as the second stage critical value based on the chosen combination function. Our goal is not to develop a new method, but focus primarily on developing a comprehensive understanding of two-stage designs. Rather than tying up the early rejection and acceptance boundaries by considering the second stage critical value to be the same as that of the level á combination test, as done in the original Bauer-Kohne method, we allow the second-stage critical value to be determined from prefixed early rejection and acceptance boundaries. An explicit formula is derived for the overall Type I error probability to determine the second stage critical value from these stopping boundaries not only for Fisher's combination function but also for other types of combination function. Tables of critical values corresponding to several different choices of early rejection and acceptance boundaries and these combination functions are presented. A dataset from a clinical study is used to apply the different methods based on directly computed second stage critical values from pre fixed stopping boundaries and discuss the outcomes in relation to those produced by the original Bauer-Kohne method. We then extend the Bauer-Kohne method to two-hypothesis setting and propose a stepwise-combination method for a two-stage adaptive design. In particular, we modify Holm's step-down procedure (1979) and suggest a step-down combination method to control the overall FWER at a desired level á. In many scientific studies requiring simultaneous testing of multiple null hypotheses, it is often necessary to carry out the multiple testing in two stages to decide which of the hypotheses can be rejected or accepted at the first stage and which should be followed up for further testing having combined their p-values from both stages. Unfortunately, no multiple testing procedure is available yet to perform this task meeting pre-specified boundaries on the first-stage p-values in terms of the false discovery rate (FDR) and maintaining a control over the overall FDR at a desired level. Our third goal in this work is to present two procedures, extending the classical Benjamini-Hochberg (BH) procedure and its adaptive version incorporating an estimate of the number of true null hypotheses from single-stage to a two-stage setting. These procedures are theoretically proved to control the overall FDR when the pairs of first- and second-stage p-values are independent and those corresponding to the null hypotheses are identically distributed as a pair (p1, p2) satisfying the p-clud property of Brannath, Posch and Bauer (2002, Journal of the American Statistical Association, 97, 236 -244). We consider two types of combination function, Fisher's and Simes', and present explicit formulas involving these functions towards carrying out the proposed procedures based on pre-determined critical values or through estimated FDR's. Simulations were carried to compare the proposed methods with class BH procedure using first stage data only and full data from both stages respectively. Our simulation studies indicate that the proposed procedures can have significant power improvement over the single-stage BH procedure based on the first stage data, at least under independence, and can continue to control the FDR under some dependence situations. Application of the proposed procedures to a real gene expression data set produces more discoveries compared to the single-stage BH procedure using the first stage data and full data as well.