In conjunction with National Preparedness Month, University of Kentucky College of Public Health researchers released a set of recommendations for improving the methodology and measures used by the National Health Security Preparedness Index. The Index is a relatively new measurement tool designed to summarize state and national capabilities in preparing for, responding to, and recovering from the health consequences of disasters, epidemics, and other large-scale emergencies. Created with support from the U.S. Centers for Disease Control and Prevention (CDC), an initial version of the Index was released in December 2013, and the second, current, version of the Index was released a year later in December 2014.
Earlier this year, responsibility for managing and improving the Index transitioned from CDC to the Robert Wood Johnson Foundation, which selected a team at the University of Kentucky to serve as the program management office for the Index. University of Kentucky Professor Dr. Glen Mays, who leads this office, observes that “the Index is still a work in progress as the science and practice of preparedness continues to evolve, and particularly as we learn how to measure important preparedness constructs and capabilities such as those outlined in the National Health Security Strategy.” He notes that for these reasons, the Index’s ability to serve as a yardstick in measuring state and national progress toward greater levels of protection and health security has yet to be fully realized.
Earlier this year, the University of Kentucky team began a series of validation studies designed to document current measurement properties of the Index and to identify opportunities for improving the Index as a measurement tool. Dr. Mays says that “the main objective is to offer Index users greater clarity about what the Index numbers really mean, and greater confidence in how the numbers can be compared and used. Right now we can’t say with much confidence whether the national score of 7.8 for surveillance is meaningfully different from the score of 6.7 for community planning.”
The results from validation studies and simulation analyses revealed some important limitations in how Index values are currently calculated. Some of the nearly 200 measures used in the Index have orders of magnitude more influence on aggregate preparedness values than do other measures, due purely to the scaling, aggregation and implicit weighting methods used within the Index. The University of Kentucky team identified some feasible strategies for improving the computational methods and measures used within the Index so that it will offer users greater clarity and greater confidence when Index values are compared across domains, across states, and for the first time next year, when values are compared over time.
Dr. Mays stressed that “the goal is to improve the measurement properties of the Index without diminishing its relevance to preparedness practitioners, so we need to hear from the practice community as well as the scientific community about these recommendations before we finalize them and put them in place.” A report describing the recommended updates to the Index methodology and measures was released by the University of Kentucky in conjunction with National Preparedness Month. A video briefing on the recommendations is also available. The Index is currently inviting public comments regarding these proposed updates to the Index methodology and measures. The public comment period extends throughout National Preparedness Month and ends on September 30. The 2015-16 version of the Index is scheduled to be released at the end of the year.