NEW APPROACHES TO MULTIPLE TESTING OF GROUPED HYPOTHESES

Abstract

Testing multiple hypotheses appearing in non-overlapping groups is a common statistical problem in many modern scientific investigations, with this group formation occurring naturally in many of these investigations. The goal of this dissertation is to explore the current state of knowledge in the area of multiple testing of grouped hypotheses and to present newer and improved statistical methodologies. As the first part of this dissertation, we propose a new Bayesian two-stage multiple testing method controlling false discovery rate (FDR) across all hypotheses. The method decomposes a posterior measure of false discoveries across all hypotheses into within- and between-group components allowing a portion of the overall FDR level to be used to maintain control over within groupfalse discoveries. Such within-group FDR control effectively captures the group structure as well as the dependence, if any, within the groups. The procedure can maintain a tight control over the overall FDR,as shown numerically under two different model assumptions, independent and Markov dependent Bernoulli’s, for the hidden states of the within-group hypotheses. The proposed method in its oracle form is optimal at both within-and between-group levels of its application. We also present a data driven version of the proposed method whose performance in terms of FDR control and power relative to its relevant competitors is examined through simulations. We apply this Bayesian method to a real data application, which is the Adequate Yearly Progress (AYP) study data of California elementary schools (2013) comparing the academic performance for socioeconomically advantaged (SEA) versus socioeconomically disadvantaged (SED) students, and our method has more meaningful discoveries than two other competing methods existing in the literature. The second part of the dissertation is geared towards making contribution to the outstanding problem of developing an FDR controlling frequentist method for multiple testing of grouped hypotheses, which can serve not only as an extension of the classical Benjamini -Hochberg (BH, 1995) method from single to multiple groups but also can be more powerful due to the underlying group structure. We suggest a number of such methods and examine their performances in comparison with the single-group BH method mainly based on simulations. Some possible future directions of research in the proposed area are discussed at the end of this dissertation.