MULTIPLE COMPARISONS IN BIOMEDICAL RESEARCH: THE PROBLEM AND ITS SOLUTIONS

Cover Page

Cite item

Full Text

Abstract

One of the most common but rarely discussed problems in Russian biomedical research is a problem of multiple comparisons. When a researcher performs pairwise comparisons of means in several groups the number of tested ststistical hypotheses increases leading to inflation of the alpha-error. In international scientific literature this issue is well-described and several solutions are offered. The aim of this article is to describe the problem of alpha error inflation and present methods for solving the problem of multiple comparisons. The methods suggested in this paper can be applied at the stages of research planning, data analysis and interpretation of the results. Bonferroni, Sidak, Holm-Bonferroni, Holm-Sidak and the Benjamin-Hochberg methods are described in details. We also present user-friendly examples for manual calculations as well as a description of implementation of the suggested solutions using SPSS software.

About the authors

A. N. Narkevich

Voino-Yasenetsky Krasnoyarsk State Medical University

Email: narkevichart@gmail.com
кандидат медицинских наук, доцент, декан медико-психолого-фармацевтического факультета, заведующий кафедрой медицинской кибернетики и информатики, заведующий лабораторией медицинской кибернетики и управления в здравоохранении

K. A. Vinogradov

Voino-Yasenetsky Krasnoyarsk State Medical University

A. M. Grjibovski

Northern State Medical University; West Kazakhstan Marat Ospanov Medical University; Al-Farabi Kazakh National University; M. K. Ammosov North-Eastern Federal University

References

  1. Буненков Н. С., Буненкова Г. Ф., Комок В. В., Гриненко О. А., Немков А. С. SAS Enterprise Guide 6.1 для врачей: сравнение групп // Медицинский академический журнал. 2019. Т. 19, № 4. С. 33-40. DOI: 10/17816/ MAJ17736
  2. Гржибовский А. М. Анализ трех и более независимых групп количественных данных // Экология человека. 2008. № 3. С. 50-58
  3. Гржибовский А. М. Сравнение трех и более независимых групп с использованием непараметрического критерия Краскела - Уоллиса в программе Stata // Экология человека. 2014. № 6. С. 55-58
  4. Гржибовский А. М., Иванов С. В., Горбатова М. А. Сравнение количественных данных трех и более независимых выборок с использованием программного обеспечения Statistica и SPSS: параметрические и непараметрические критерии // Наука и Здравоохранение. 2016. № 4. С. 5-37
  5. Гржибовский А. М., Иванов С. В., Горбатова М. А. Сравнение количественных данных трех и более парных выборок с использованием программного обеспечения Statistica и SPSS: параметрические и непараметрические критерии // Наука и Здравоохранение. 2016. № 5. С. 5-29
  6. Маркевич А. Н., Виноградов К. А. Настольная книга автора медицинской диссертации: пособие. М.: Инфра-М, 2019. 454 с
  7. Benjamini Y., Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: series B (Methodological). 1995, 57 (1 ), pp. 289-300. doi: 10.1111/j.2517-6161.1995.tb02031.x
  8. Bonferroni C. E., Teoria statistica delle classi e calcolo delle probability Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commerciali di Firenze. 1936, 62 p.
  9. Che R., Jack J. R., Motsinger-Reif A. A., Brown C. C. An adaptive permutation approach for genome-wide association study: evaluation and recommendations for use. BioData Mining. 2014, 7, pp. 9. DOI: 10.1 186/1756-0381-7-9
  10. Dunnett C. W. A multiple comparison procedure for comparing several treatments with a control. Journal of the American Statistical Association. 1955, 50, pp. 1096-1121. doi: 10.1080/01621459.1955.10501294
  11. Foulkes A. C., Watson D. S., Griffiths C. E. M., Warren R. B., Huber W., Barnes M. R. Research Techniques Made Simple: Bioinformatics for Genome-Scale Biology. Journal of Investigative Dermatology. 2017, 137, pp. e163-e168. doi: 10.1016/j.jid.2017.07.095
  12. Gao X., Starmer J., Martin E. R. A multiple testing correction method for genetic association studies using correlated single nucleotide polymorphisms. Genetic Epidemiology. 2008, 32 (4), pp. 361-369. DOI: 10.1002/ gepi.20310
  13. Holland B. S., Copenhaver M. D. Improved Bonferroni-type multiple testing procedures. Psychological Bulletin. 1988, 104 (1), pp. 145-149. doi: 10.1037//0033-2909.104.1.145
  14. Holm S. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics. 1979, 6 (2), pp. 65-70.
  15. Johnson R. C., Nelson G. W, Troyer J. L., Lautenberger J. A., Kessing B. D. Accounting for multiple comparisons in a genome-wide association study (GWAS). BMC Genomics. 2010, 11, pp. 724. doi: 10.1186/1471-2164-1 1-724
  16. Keuls M. The use of the «studentized range» in connection with an analysis of variance. Euphytica. 1952, 1 (2), pp. 112-122. doi: 10.1007/bf01908269
  17. Moran M. Arguments for rejecting the sequential Bonferroni in ecological studies. Oikos. 2003, 100 (2), pp. 403-405. doi: 10.1034/j.1600-0706.2003.12010.x
  18. Newman D. The distribution of range in samples from a normal population, expressed in terms of an independent estimate of standard deviation. Biometrika. 1939, 31 (1), pp. 20-30. doi: 10.1093/biomet/31.1-2.20
  19. Rothman K. J. No Adjustments Are Needed for Multiple Comparisons. Epidemiology. 1990, 1 (1), pp. 43-46. doi: 10.1097/00001648-199001000-00010
  20. Seidler J., Vondracek J. I., Saxl I. The life and work of Zbynek Sidak (1933-1999). Applications of Mathematics. 2000, 45 (5), pp. 321. doi: 10.1023/A: 1022238410461. hdl:10338.dmlcz/134443
  21. Shaffer J. P. Multiple Hypothesis Testing. Annual Review of Psychology. 1995, 46 (1), pp. 561-584. doi: 10.1146/annurev.ps.46.020195.003021
  22. Sidak Z. K. Rectangular Confidence Regions for the Means of Multivariate Normal Distributions. Journal of the American Statistical Association. 1967, 62 (318), pp. 626633. doi: 10.1080/01621459.1967.10482935

Copyright (c) 2020 Narkevich A.N., Vinogradov K.A., Grjibovski A.M.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
 


This website uses cookies

You consent to our cookies if you continue to use our website.

About Cookies