ARTICLE
ABSTRACT
The range of applications of classical multiple testing procedures is limited due to model assumptions, and in many cases classic solutions are non-existent. In such situations non-classical multiple testing procedures allow to control the effect of multiple testing. Although they are popular for computational simplicity and a wide range of applications, marginal multiple testing procedures do not take into account joint distribution of test statistics, which make them more conservative than joint multiple testing procedures. The range of applications of joint procedures introduced by Westfall and Young (1993) is limited due to the subset pivotality requirement. Thus, joint multiple testing procedures suggested by Dudoit and van der Laan (2008) seem very promising. A wide range of applications, the possibility of choosing the Type I error rate and easily accessible software (MTP procedure is implemented in R multtest package) are their obvious advantages. Unfortunately, the results of the analysis of MPT procedure obtained by Werft and Benner (2009) revealed that it does not control FDR in case of numerous sets of hypotheses and small samples. Furthermore, the simulation experiment presented in the article showed that MTP procedure does not control FWER, either.
KEYWORDS
multiple testing, FWER, FDR, resampling, MTP
REFERENCES
[1] Benjamini Y., Hochberg Y., (1995), Controlling the False Discovery Rate: a Practical and Powerful Approach to Multiple Testing, Journal of the Royal Statistical Society, Ser. B, 57 (1), 289–300.
[2] Benjamini Y., Yekutieli D., (2001), The Control of the False Discovery Rate in Multiple Testing Under Dependency, Annals of Statistics, 29, 1165–1188.
[3] Bretz F., Hothorn T, Westfall P., (2011), Multiple Comparisons Using R, Chapman and Hall, Boca Raton.
[4] Denkowska S., (2005), Zastosowanie procedur testowań wielokrotnych opartych na uporządkowanych prawdopodobieństwach testowych do wydzielania jednorodnych podgrup wartości przeciętnych, Przegląd Statystyczny, 1, 115–131.
[5] Denkowska S., (2006), Multiple Testing in a Correlation Matrix, w: Pociecha J., (red.), A Comparative Analysis of the Socio-Economic Consequences of Transition Processes in the Central and Eastern European Countries, Wydawnictwo AE w Krakowie, 117–134.
[6] Denkowska S., (2007a), Testowanie wielokrotne w badaniach ekonomicznych, Folia Oeconomica Cracoviensia, XLV, Wydawnictwo Oddziału PAN, Kraków, 119–135.
[7] Denkowska S., (2007b), Monte Carlo Analysis of the Effectiveness of Multiple Comparison Procedures, Education of Quantitative Mathematical-Statistical Methods, University of Economics in Bratislava, Bratislava, 117–126.
[8] Denkowska S., (2011a), Testowanie jednoczesne przy weryfi kacji ocen parametrów strukturalnych liniowego modelu ekonometrycznego, Zeszyty Naukowe „Metody Analizy Danych”, 873, Wydawnictwo UEK, Kraków, 53–68.
[9] Denkowska S., (2011b), Testowanie wielokrotne przy budowie modelu ekonometrycznego, Zeszyty Naukowe „Metody Analizy Danych”, Wydawnictwo UEK, Kraków, 27–42.
[10] Denuit M., Scaillet O., (2004), Nonparametric Tests for Positive Quadrant Dependence, Journal of Financial Econometrics, 2, 422–450.
[11] Domański Cz., Pruska K., (2000), Nieklasyczne metody statystyczne, PWE, Warszawa.
[12] Domański Cz., Parys D., (2007), Statystyczne metody wnioskowania wielokrotnego, Wydawnictwo UŁ, Łódź.
[13] Dudoit S., van der Laan M., (2008), Multiple Testing Procedures with Applications to Genomics, Springer Series in Statistics.
[14] Hochberg Y., Tamhane A. C., (1987), Multiple Comparison Procedures, John Wiley & Sons, NY.
[15] Holland B., Copenhaver M. D., (1987), An Improved Sequentially Rejective Bonferroni Test Procedure, Biometrics, 43, 417–423.
[16] Lehmann E. L., Romanno J. P., (2005), Generalizations of the Familywise Error Rate, Annals of Statistics, 33 (3), 1138–1154.
[17] Rosenthal R., Rubin D. B., (1983), Ensemble-Adjusted p Values, Psychological Bulletin, 94 (3), 540–541.
[18] Shaffer J. P., (1986), Modifi ed Sequentially Rejective Multiple Test Procedures, Journal of the American Statistical Association, 81, 826–831.
[19] Shaffer J. P., (1995), Multiple Hypothesis Testing, Annual Review of Psychology, 46, 561–84.
[20] Tukey J. W., (1953), The Problem of Multiple Comparisons, w: Braun H. I., (red.), (1994) The Collected Works of John W. Tukey, vol. VIII: Multiple Comparisons: 1948–1983, New York: Chapman & Hall, 1–300.
[21] Westfall, P. H., Tobias R. D., Rom D., Wolfi nger R. D., Hochberg Y., (1999), Multiple Comparisons and Multiple Tests Using the SAS System, SAS Institute Inc., Cary, NC.
[22] Westfall P. H., Young S. S., (1993), Resampling Based Multiple Testing, Wiley, New York.
[23] Werft W., Benner A., (2009), www.iscb2009.info/RSystem/Soubory/Prez%20Monday/S10.4%20Werft.pdf.
[24] Wright S. P., (1992), Adjusted P-values for Simultaneous Inference, Biometrics, 48, 1005–1013.
