Tuberculosis is a growing global health threat made worse by the elusive ability of the Mycobacterium tuberculosis bacteria to become resistant to antibiotics. One group of M. tuberculosis strains, known as the Beijing genotype, is a vital focus of study as it has been believed to be far more likely to develop drug-resistance. However, studies specifically meant to look into whether Beijing genotype bacteria have higher mutation rates in genes that cause drug-resistance are rare and their conclusions have been unclear. One well-regarded study found that the Beijing genotype is no more likely to develop drug-resistance than other strains, whereas another found the opposite. Three earnest attempts were recently made to reanalyze the former study in hope of making the “null” outcome positive and thereby reconciling the two studies.
While disproving a null hypothesis is often a major objective of any scientific endeavor, focusing excessively on it can be detrimental to science. In a new paper published in the journal Tuberculosis, Dr. Qi Zheng, professor in the epidemiology and biostatistics department at the Texas A&M School of Public Health, along with a microbiologist at the Public Health Agency of Sweden, examined the three studies that aimed at revising the seemingly unattractive null outcome. The paper identifies errors in those reanalyses and highlights the need for caution and attention to detail and to be mindful of preconceived notions when researching the Beijing phenotype.
[Photo: Dr. Qi Zhen]
The two preexisting studies were both conducted following the classic Luria-Delbrück protocol, also known as the fluctuation assay. Despite the protocol’s 75-year history of diverse applications in microbial mutation research, analyzing fluctuation assay data is still full of pitfalls. The three reanalyses that Dr. Zheng investigated all contained errors. For example, one reanalysis failed to account for partial plating, a process where only a portion of a cell culture is transferred to a dish coated with an antibiotic, such as rifampicin — a first-line anti-tuberculosis agent. Partial plating must be accounted for because mutant cells are often not evenly distributed in a dish, which is an oft-overlooked source of uncertainty.
Another source of error that Zheng identified was a misunderstanding of variations in the final number of cells in each culture. Two of the three reanalyses mistakenly considered variation between experiments and variation within experiments to be the same, even though they are not related. Between experiment variation is affected by strains of bacteria used, length of the incubation period and size of the cultures studied. However, the data analyst should focus on within experiment variation that can be affected by myriad random factors.
All three reanalyses produced smaller p-values, an indicator of statistical significance, than found in the original study. An imperfect new algorithm was found to be responsible for the lowest p-values in the latest of the reanalyses. “An aversion to null conclusions can perhaps lead researchers to lower their guard, when a positive conclusion is in sight,” said Zheng.
To enhance scientific understanding of the Beijing genotype’s ability to develop drug-resistance, the researchers emphasized that future studies should pay extra attention to experimental details and include more strains from both genotypes. However, they caution that the simple classification of M. tuberculosis into Beijing and non-Beijing genotypes may not be as useful as once thought. Therefore, future studies should pay increasing attention to other characteristics of the Beijing genotype, such as how easily Beijing strains can be transmitted between healthy and infected individuals.
“Taking an unbiased attitude toward the Beijing genotype will likely benefit efforts to better understand M. tuberculosis and tackle the growing global health threat that it presents,” said Dr. Zheng.