For multiple comparisons in analysis of variance, the practitioners' handbooks generally advocate standard methods such as Bonferroni, or an F-test followed by Tukey's honest significant difference... Show moreFor multiple comparisons in analysis of variance, the practitioners' handbooks generally advocate standard methods such as Bonferroni, or an F-test followed by Tukey's honest significant difference method. These methods are known to be suboptimal compared to closed testing procedures, but improved methods can be complex in the general multigroup set-up. In this note, we argue that the case of three-groups is special: with three groups, closed testing procedures are powerful and easy to use. We describe four different closed testing procedures specifically for the three-group set-up. The choice of method should be determined by assessing which of the comparisons are considered primary and which are secondary, as dictated by subject-matter considerations. We describe how all four methods can be used with any standard software. Show less
In this article, we introduce a novel procedure for improving power of multiple testing procedures (MTPs) of interval hypotheses. When testing interval hypotheses the null hypothesis P-values tend... Show moreIn this article, we introduce a novel procedure for improving power of multiple testing procedures (MTPs) of interval hypotheses. When testing interval hypotheses the null hypothesis P-values tend to be stochastically larger than standard uniform if the true parameter is in the interior of the null hypothesis. The new procedure starts with a set of P-values and discards those with values above a certain pre-selected threshold, while the rest are corrected (scaled-up) by the value of the threshold. Subsequently, a chosen family-wise error rate (FWER) or false discovery rate MTP is applied to the set of corrected P-values only. We prove the general validity of this procedure under independence of P-values, and for the special case of the Bonferroni method, we formulate several sufficient conditions for the control of the FWER. It is demonstrated that this "filtering" of P-values can yield considerable gains of power. Show less
The thesis provides novel statistical (multiple) testing methods based on permutations or other transformations of data. Permutation-based multiple testing methods tend to have relatively high... Show moreThe thesis provides novel statistical (multiple) testing methods based on permutations or other transformations of data. Permutation-based multiple testing methods tend to have relatively high power, because they take into account the observed dependence structure of the data (without requiring assumptions on that structure). This thesis places particular emphasis on confidence upper bounds for the false discovery proportion: the proportion of false positives among all rejected hypotheses. Moreover, a novel semi-parametric method is presented for robust testing in generalized linear models. This test is related to the score test and the permutation test and is based on sign-flipping individual score contributions. Show less
