When a ranking of institutions such as medical centers or universities is based on a numerical measure of performance provided with a standard error, confidence intervals (CIs) should be calculated... Show moreWhen a ranking of institutions such as medical centers or universities is based on a numerical measure of performance provided with a standard error, confidence intervals (CIs) should be calculated to assess the uncertainty of these ranks. We present a novel method based on Tukey's honest significant difference test to construct simultaneous CIs for the true ranks. When all the true performances are equal, the probability of coverage of our method attains the nominal level. In case the true performance measures have no exact ties, our method is conservative. For this situation, we propose a rescaling method to the nominal level that results in shorter CIs while keeping control of the simultaneous coverage. We also show that a similar rescaling can be applied to correct a recently proposed Monte-Carlo based method, which is anticonservative. After rescaling, the two methods perform very similarly. However, the rescaling of the Monte-Carlo based method is computationally much more demanding and becomes infeasible when the number of institutions is larger than 30-50. We discuss another recently proposed method similar to ours based on simultaneous CIs for the true performance. We show that our method provides uniformly shorter CIs for the same confidence level. We illustrate the superiority of our new methods with a data analysis for travel time to work in the United States and on rankings of 64 hospitals in the Netherlands. Show less
Mohamad, D. al; Zwet, E. van; Solari, A.; Goeman, J. 2021
We consider the problem of constructing simultaneous confidence intervals (CIs) for the ranks of n means based on their estimates together with the (known) standard errors of those estimates. We... Show moreWe consider the problem of constructing simultaneous confidence intervals (CIs) for the ranks of n means based on their estimates together with the (known) standard errors of those estimates. We present a generic method based on the partitioning principle in which the parameter space is partitioned into disjoint subsets and then each one of them is tested at level a. The resulting CIs have then a simultaneous coverage of 1 - alpha. We show that any procedure which produces simultaneous CIs for ranks can be written as a partitioning procedure. We present a first example where we test the partitions using the likelihood ratio (LR) test. Then, in a second example we show that a recently proposed method for simultaneous CIs for ranks using Tukey's honest significant difference test has an equivalent procedure based on the partitioning principle. By embedding these two methods inside our generic partitioning procedure, we obtain improved variants. We illustrate the performance of these methods through simulations and real data analysis on hotel ratings. While the novel method that uses the LR test and its variant produce shorter CIs when the number of means is small, the Tukey-based method and its variant produce shorter CIs when the number of means is high. Show less
We present a new general method for constrained likelihood ratio testing which, when few constraints are violated, improves upon the existing approach in the literature that compares the likelihood... Show moreWe present a new general method for constrained likelihood ratio testing which, when few constraints are violated, improves upon the existing approach in the literature that compares the likelihood ratio with the quantile of a mixture of chi-squared distributions; the improvement is in terms of both simplicity and power. The proposed method compares the constrained likelihood ratio statistic against the quantile of only one chi-squared random variable with data-dependent degrees of freedom. The new test is shown to have a valid exact significance level a. It also has more power than the classical approach against alternatives for which the number of violations is not large. We provide more details for testing a simple order mu(1) <= center dot center dot center dot <= mu(p) against all alternatives using the proposed approach and give clear guidelines as to when the new method would be advantageous. A simulation study suggests that for testing a simple order, the new approach is more powerful in many scenarios than the existing method that uses a mixture of chi-squared variables. We illustrate the results of our adaptive procedure using real data on the liquidity preference hypothesis. Show less
