Predictive modeling — formulating statistical models to calculate the presence of a variable from other variables in a study — is a vital tool in obesity and nutrition research. Dr. Brandon J. George, statistician in the office of energetics at the University of Alabama at Birmingham, working with Dr. Andrada E. Ivanescu, assistant professor in the department of mathematical sciences at Montclair State University — along with a UAB team that included Dr. Peng Li, former statistician in the office of energetics and currently in the department of biostatistics; Dr. Andrew W. Brown, scientist in the office of energetics and Nutrition Obesity Research Center (NORC); Dr. Dheeraj Raju, assistant professor in the School of Nursing; and Dr. David B. Allison, distinguished professor and director of the office of energetics and NORC — recently observed that “[t]o determine the quality of the model, it is necessary to quantify and report the predictive validity of the derived models. Conducting validation of the predictive measures provides essential information to the research community about the model. Unfortunately, many articles fail to account for the nearly inevitable reduction in predictive ability that occurs when a model derived on one dataset is applied to a new dataset. Under some circumstances, the predictive validity can be reduced to nearly zero.”
In the collaborative overview, the researchers tell why there are reductions in predictive validity; describe the most frequently used metrics to estimate predictive validity (such as “R2, mean squared error, sensitivity, specificity, receiver operating characteristic, [and] concordance index”); and explain the methods employed to estimate predictive validity (such as “cross-validation, bootstrap, adjusted and shrunken R2”). They advise that there are available methods for estimating the anticipated reduction in predictive ability in new samples and note that “this expected reduction should always be reported when new predictive models are introduced.”
“The Importance of Prediction Model Validation and Assessment in Obesity and Nutrition Research” was published online in October in the International Journal of Obesity.
Journal article: http://www.nature.com/ijo/journal/vaop/naam/abs/ijo2015214a.html